Principle of Minimizing Empirical Risk and Averaging Aggregate Functions

Z. M. Shibzukhov

Itogi nauki i tehniki. Sovremennaâ matematika i eë priloženiâ. Tematičeskie obzory, Proceedings of the International Conference “Actual Problems of Applied Mathematics and Physics,” Kabardino-Balkaria, Nalchik, May 17–21, 2017, Tome 154 (2018), pp. 123-137

Résumé

In this paper, we propose an extended version of the principle of minimizing empirical risk (ER) based on the use of averaging aggregating functions (AAF) for calculating the ER instead of the arithmetic mean. This is expedient if the distribution of losses has outliers and hence risk assessments are biased. Therefore, a robust estimate of the average risk should be used for optimization the parameters. Such estimates can be constructed by using AAF that which are solutions of the problem of minimizing the penalty function for deviating from the mean value. We also propose an iterative reweighting scheme for the numerical solution of the ER minimization problem. We give examples of constructing a robust procedure for estimating parameters in a linear regression problem and a linear separation problem for two classes based on the use of an averaging aggregating function that replaces the $\alpha$-quantile.

Détail

Keywords: empirical risk, averaging function, aggregation function, loss function, iterative reweighing algorithm.