Geodesic
On the properties of bias-variance decomposition for kNN regression
The Bulletin of Irkutsk State University. Series Mathematics, Tome 43 (2023), pp. 110-121

Voir la notice de l'article provenant de la source Math-Net.Ru

When choosing the optimal complexity of the method for constructing decision functions, an important tool is the decomposition of the quality criterion into bias and variance. It is generally assumed (and in practice this is most often true) that with increasing complexity of the method, the bias component monotonically decreases, and the variance component increases. The conducted research shows that in some cases this behavior is violated. In this paper, we obtain an expression for the variance component for the kNN method for the linear regression problem in the formulation when the “explanatory” features are random variables. In contrast to the well-known result obtained for non-random “explanatory” variables, in the considered case, the variance may increase with the growth of $k$.
Keywords: machine learning, $k$-nearest neighbors algorithm, overfitting.
Mots-clés : bias-variance decomposition 
@article{IIGUM_2023_43_a7,
     author = {Victor M. Nedel'ko},
     title = {On the properties of bias-variance decomposition for {kNN} regression},
     journal = {The Bulletin of Irkutsk State University. Series Mathematics},
     pages = {110--121},
     publisher = {mathdoc},
     volume = {43},
     year = {2023},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/IIGUM_2023_43_a7/}
}
TY  - JOUR
AU  - Victor M. Nedel'ko
TI  - On the properties of bias-variance decomposition for kNN regression
JO  - The Bulletin of Irkutsk State University. Series Mathematics
PY  - 2023
SP  - 110
EP  - 121
VL  - 43
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/IIGUM_2023_43_a7/
LA  - en
ID  - IIGUM_2023_43_a7
ER  - 
%0 Journal Article
%A Victor M. Nedel'ko
%T On the properties of bias-variance decomposition for kNN regression
%J The Bulletin of Irkutsk State University. Series Mathematics
%D 2023
%P 110-121
%V 43
%I mathdoc
%U http://geodesic.mathdoc.fr/item/IIGUM_2023_43_a7/
%G en
%F IIGUM_2023_43_a7
Victor M. Nedel'ko. On the properties of bias-variance decomposition for kNN regression. The Bulletin of Irkutsk State University. Series Mathematics, Tome 43 (2023), pp. 110-121. http://geodesic.mathdoc.fr/item/IIGUM_2023_43_a7/