Geodesic
On the principle of minimizing the average empiric risk to solutions of regression problems
News of the Kabardin-Balkar scientific center of RAS, no. 6 (2016), pp. 88-95.

Voir la notice de l'article provenant de la source Math-Net.Ru

In this paper, we propose an extension of the principle of empirical risk minimization to solve the problem of regression. It is based on the use of averaging aggregate functions to calculate the empirical risk instead of the arithmetic mean. Such intermediate risk assessment can be constructed using averaging aggregate functions, which are the solution of the problem of minimizing the penalty function for the deviation from its mean value. Such an approach to represent the average aggregate functions allows, on the one hand, to define a much broader middle class functions. In this paper we propose a new gradient scheme for solving the problem of minimizing the average risk. It is an analog circuit used in the SAG algorithm in the case when the risk is calculated using the arithmetic mean. An illustrative example of the construction of robust procedures for assessment of parameters in a linear regression based on the use of the averaging function average approximating the median is demonstrated.
Keywords: aggregation function/operation, empirical risk, penalty function, gradient descent procedure.
Mots-clés : regression 
@article{IZKAB_2016_6_a13,
     author = {Z. M. Shibzukhov and D. P. Dimitrichenko and M. A. Kazakov},
     title = {On the principle of minimizing the average empiric risk to solutions of regression problems},
     journal = {News of the Kabardin-Balkar scientific center of RAS},
     pages = {88--95},
     publisher = {mathdoc},
     number = {6},
     year = {2016},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/IZKAB_2016_6_a13/}
}
TY  - JOUR
AU  - Z. M. Shibzukhov
AU  - D. P. Dimitrichenko
AU  - M. A. Kazakov
TI  - On the principle of minimizing the average empiric risk to solutions of regression problems
JO  - News of the Kabardin-Balkar scientific center of RAS
PY  - 2016
SP  - 88
EP  - 95
IS  - 6
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/IZKAB_2016_6_a13/
LA  - ru
ID  - IZKAB_2016_6_a13
ER  - 
%0 Journal Article
%A Z. M. Shibzukhov
%A D. P. Dimitrichenko
%A M. A. Kazakov
%T On the principle of minimizing the average empiric risk to solutions of regression problems
%J News of the Kabardin-Balkar scientific center of RAS
%D 2016
%P 88-95
%N 6
%I mathdoc
%U http://geodesic.mathdoc.fr/item/IZKAB_2016_6_a13/
%G ru
%F IZKAB_2016_6_a13
Z. M. Shibzukhov; D. P. Dimitrichenko; M. A. Kazakov. On the principle of minimizing the average empiric risk to solutions of regression problems. News of the Kabardin-Balkar scientific center of RAS, no. 6 (2016), pp. 88-95. http://geodesic.mathdoc.fr/item/IZKAB_2016_6_a13/

[1] V. Vapnik, “The Nature of Statistical Learning Theory”, Information Science and Statistics, 2000, Springer-Verlag | MR

[2] P. J. Rousseeuw, “Least Median of Squares Regression”, Journal of the American Statistical Association, 1984, no. 79, 871–880 | DOI | MR | Zbl

[3] P. J. Rousseeuw, A. M. Leroy, Robust Regression, Outlier Detection, John Wiley and Sons, New York, 1987 | MR | Zbl

[4] R. Mesiar, M. Komornikova, A. Kolesarova, T. Calvo, “Aggregation functions: A revision”, Fuzzy Sets and Their Extensions: Representation, Aggregation and Models, eds. In H. Bustince, F. Herrera, J. Montero, Springer, Berlin, Heidelberg, 2008 | MR | Zbl

[5] M. Grabich, J. L. Marichal, E. Pap, “Aggregation Functions”, Series: Encyclopedia of Mathematics and its Applications, v. 127, Cambridge University Press, 2009 | MR

[6] G. Sola H. Beliakov, T. Calvo, A Practical Guide to Averaging Functions, Springer, 2016, 329 pp. | MR

[7] T. Calvo, G. Beliakov, “Aggregation functions based on penalties”, Fuzzy Sets and Systems, 161:10 (2010), 1420–1436 | DOI | MR | Zbl

[8] Z. M. Shibzukhov, “Correct Aggregate Operations with Algorithms”, Pattern Recognition and Image Analysis, 24:3 (2014), 377–382 | DOI

[9] Z. M. Shibzukhov, “Aggregation correct operations on algorithms”, Doklady Mathematics, 91:3 (2015), 391–393 | DOI | MR | Zbl

[10] N. Le Roux, M. Schmidt, F. Bach, 2012 http://arxiv.org/abs/1202.6258

[11] M. Schmidt, N. Le Roux, F. Bach, 2013 http://arxiv.org/abs/1309.2388 | MR

[12] Shalev-Shwartz, T. Zhang, “Stochastic dual coordin ate ascent methods for regularized loss minimization”, Journal of Machine Learning Research, 14 (2013), 567–599 | MR | Zbl