Geodesic
Principle of Minimizing Empirical Risk and Averaging Aggregate Functions
Itogi nauki i tehniki. Sovremennaâ matematika i eë priloženiâ. Tematičeskie obzory, Proceedings of the International Conference “Actual Problems of Applied Mathematics and Physics,” Kabardino-Balkaria, Nalchik, May 17–21, 2017, Tome 154 (2018), pp. 123-137.

Voir la notice de l'article provenant de la source Math-Net.Ru

In this paper, we propose an extended version of the principle of minimizing empirical risk (ER) based on the use of averaging aggregating functions (AAF) for calculating the ER instead of the arithmetic mean. This is expedient if the distribution of losses has outliers and hence risk assessments are biased. Therefore, a robust estimate of the average risk should be used for optimization the parameters. Such estimates can be constructed by using AAF that which are solutions of the problem of minimizing the penalty function for deviating from the mean value. We also propose an iterative reweighting scheme for the numerical solution of the ER minimization problem. We give examples of constructing a robust procedure for estimating parameters in a linear regression problem and a linear separation problem for two classes based on the use of an averaging aggregating function that replaces the $\alpha$-quantile.
Keywords: empirical risk, averaging function, aggregation function, loss function, iterative reweighing algorithm. 
@article{INTO_2018_154_a14,
     author = {Z. M. Shibzukhov},
     title = {Principle of {Minimizing} {Empirical} {Risk} and {Averaging} {Aggregate} {Functions}},
     journal = {Itogi nauki i tehniki. Sovremenna\^a matematika i e\"e prilo\v{z}eni\^a. Temati\v{c}eskie obzory},
     pages = {123--137},
     publisher = {mathdoc},
     volume = {154},
     year = {2018},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/INTO_2018_154_a14/}
}
TY  - JOUR
AU  - Z. M. Shibzukhov
TI  - Principle of Minimizing Empirical Risk and Averaging Aggregate Functions
JO  - Itogi nauki i tehniki. Sovremennaâ matematika i eë priloženiâ. Tematičeskie obzory
PY  - 2018
SP  - 123
EP  - 137
VL  - 154
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/INTO_2018_154_a14/
LA  - ru
ID  - INTO_2018_154_a14
ER  - 
%0 Journal Article
%A Z. M. Shibzukhov
%T Principle of Minimizing Empirical Risk and Averaging Aggregate Functions
%J Itogi nauki i tehniki. Sovremennaâ matematika i eë priloženiâ. Tematičeskie obzory
%D 2018
%P 123-137
%V 154
%I mathdoc
%U http://geodesic.mathdoc.fr/item/INTO_2018_154_a14/
%G ru
%F INTO_2018_154_a14
Z. M. Shibzukhov. Principle of Minimizing Empirical Risk and Averaging Aggregate Functions. Itogi nauki i tehniki. Sovremennaâ matematika i eë priloženiâ. Tematičeskie obzory, Proceedings of the International Conference “Actual Problems of Applied Mathematics and Physics,” Kabardino-Balkaria, Nalchik, May 17–21, 2017, Tome 154 (2018), pp. 123-137. http://geodesic.mathdoc.fr/item/INTO_2018_154_a14/

[1] Khyuber P., Robastnost v statistike, Mir, M., 1984 | MR

[2] Shibzukhov Z. M., “Agregiruyuschie korrektnye operatsii nad algoritmami”, Dokl. RAN, 462:6 (2015), 649–-652 | DOI | MR | Zbl

[3] Beliakov G. , Sola H., Calvo T., A practical guide to averaging functions, Springer-Verlag, 2016 | MR

[4] Calvo T., Beliakov G., “Aggregation functions based on penalties”, Fuzzy Sets Syst., 161:10 (2010), 1420–1436 | DOI | MR | Zbl

[5] Grabich M., Marichal J.-L., Pap E., “Aggregation Functions”, Encycl. Math. Appl., v. 127, Cambridge Univ. Press, 2009 | MR

[6] Knigma D. P., Ba J., Adam: A method for stochastic optimization, arXiv: 1412.6980 [cs.LG]

[7] Koenker R., Quantile regression, Campridge Univ. Press, N.Y., 2005 | MR | Zbl

[8] Ma Y., Li L., Huang X., Wang S., “Robust support vector machine using least median loss penalty”, IFAC Proc. Vols. 18th IFAC World Congr., 44:1 (2011), 11208–11213

[9] Mesiar R., Komornikova M., Kolesarova A., Calvo T., “Aggregation functions: A revision”, Fuzzy Sets and Their Extensions: Representation, Aggregation and Models, eds. Bustince H., Herrera F., Montero J., Springer-Verlag, Berlin–Heidelberg, 2008 | MR | Zbl

[10] Newey W., Powell J., “Asymmetric least square estimation and testing”, Econometrica, 55:4 (1987), 819–847 | DOI | MR | Zbl

[11] Rousseeuw P. J., “Least median of square regression”, J. Am. Stat. Assoc., 79 (1984), 871–-880 | DOI | MR | Zbl

[12] Rousseeuw P. J., Leroy A. M., Robust regression and outlier detection, Wiley, N.Y., 1987 | MR | Zbl

[13] Schmidt M., Le Roux N., Bach F., Minimizing finite sums with the stochastic average gradient, arXiv: 1309.2388 [math.OC] | MR

[14] Shibzukhov Z. M., “Correct aggregate operations with algorithms”, Pattern Recogn. Image Anal., 24:3 (2014), 377–-382 | DOI

[15] Vapnik V., The nature of statistical learning theory, Springer-Verlag, 2000 | MR | Zbl

[16] Yohai V. J., “High breakdown-point and high effciency robust estimates for regression”, Ann. Stat., 15 (1987), 642–656 | DOI | MR | Zbl