Geodesic
On the maximization of quadratic weighted kappa
The Bulletin of Irkutsk State University. Series Mathematics, Tome 23 (2018), pp. 36-45 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

An analytical expression for the optimal estimation of the numerical dependence by the criterion of a quadratic weighted kappa and also the expression for the optimal value of this criterion were obtained. It is shown that the optimal decision function is obtained from the regression function by a linear transformation. The coefficients of this transformation can be found from the condition of equality of mathematical expectations and variances of the predicted value and its estimate. The quadratic weighted kappa coefficient was originally proposed as an alternative to the correlation coefficient to reflect the strength of dependence between two characteristics, but recently it has been widely used as a criterion for the quality of the forecast in the problem of recovery of dependencies (regression analysis). At the same time, the properties of this coefficient in this context are still poorly understood. The properties of the quadratic weighted kappa criterion revealed in the work allow us to conclude that the expediency of using it as a criterion for the quality of the decision function in most cases raises doubts. This criterion provides a solution that is actually based on the regression function, but the variance of the forecast is artificially made equal to the variance of the original value. This distorts the forecast without improving the statistical properties of the decision function.
Keywords: quadratic weighted kappa, Cohen's kappa, least squares, machine learning.
Mots-clés : regression 
@article{IIGUM_2018_23_a2,
     author = {V. M. Nedel'ko},
     title = {On the maximization of quadratic weighted kappa},
     journal = {The Bulletin of Irkutsk State University. Series Mathematics},
     pages = {36--45},
     year = {2018},
     volume = {23},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/IIGUM_2018_23_a2/}
}
TY  - JOUR
AU  - V. M. Nedel'ko
TI  - On the maximization of quadratic weighted kappa
JO  - The Bulletin of Irkutsk State University. Series Mathematics
PY  - 2018
SP  - 36
EP  - 45
VL  - 23
UR  - http://geodesic.mathdoc.fr/item/IIGUM_2018_23_a2/
LA  - ru
ID  - IIGUM_2018_23_a2
ER  - 
%0 Journal Article
%A V. M. Nedel'ko
%T On the maximization of quadratic weighted kappa
%J The Bulletin of Irkutsk State University. Series Mathematics
%D 2018
%P 36-45
%V 23
%U http://geodesic.mathdoc.fr/item/IIGUM_2018_23_a2/
%G ru
%F IIGUM_2018_23_a2
V. M. Nedel'ko. On the maximization of quadratic weighted kappa. The Bulletin of Irkutsk State University. Series Mathematics, Tome 23 (2018), pp. 36-45. http://geodesic.mathdoc.fr/item/IIGUM_2018_23_a2/

[1] Berikov V. B., Lbov G. S., “Bayes estimates for recognition quality on finite sets of events”, Doklady Mathematics, 71:3 (2005), 327–330 | MR

[2] Vityaev E. E., “Semantic Probabilistic Inference of Predictions”, The Bulletin of Irkutsk State University. Series Mathematics, 21 (2017), 33–50 (In Russian) | DOI

[3] Genrikhov I. E., Djukova E. V., Zhuravlyov V. I., “About full regression decision trees”, Machine learning and data analysis, 2:1 (2016), 116–126 (In Russian) | DOI

[4] Kovalevskii A. P., Shatalin E. V., “The Choice of a Regression Model of the Body Weight on the Height via an Empirical Bridge”, Tomsk State University Journal of Mathematics and Mechanics, 2015, no. 5(37), 35–47 (In Russian) | DOI

[5] Linke Yu. Yu., “Asymptotic properties of one-step weighted M-estimators with application to some regression problems”, Teor. Veroyatnost. i Primenen., 62:3 (2017), 468–498 (In Russian) | DOI

[6] Nedelko V. M., “Some aspects of estimating a quality of decision functions construction methods”, Tomsk state university. Journal of control and computer science, 2013, no. 3(24), 123–132 (In Russian)

[7] Nedel'ko V. M., “Estimation of feature importance for quantile regression”, Machine learning and data analysis, 3:2 (2017), 151–159 (In Russian) | DOI

[8] Nedel'ko V. M., “Regression models in the classification problem”, Siberian Journal of Industrial Mathematics, 17:1 (2014), 86–98 (In Russian)

[9] Hermann Brenner, Ulrike Kliebsch, “Dependence of Weighted Kappa Coefficients on the Number of Categories”, Epidemiology, 7:2 (1996), 199–202 | DOI

[10] Cohen Jacob, “A coefficient of agreement for nominal scales”, Educational and Psychological Measurement, 20:1 (1960), 37–46 | DOI | MR

[11] A. R. Feinstein, D. V. Cicchetti, “High agreement but low Kappa. I: The problems of two paradoxes”, Journal of Clinical Epidemiology, 43:6 (1990), 543–549 | DOI

[12] K. L. Gwet, “Kappa Statistic is not Satisfactory for Assessing the Extent of Agreement between Raters”, Statistical Methods for Inter-Rater Reliability Assessment, 2002, no. 1 | MR

[13] J. Ludbrook, “Statistical Techniques for Comparing Measurers And Methods Of Measurement: A Critical Review”, Clinical and Experimental Pharmacology and Physiology, 29 (2002), 527–536 | DOI

[14] S. Vanbelle, A. Albert, “A note on the linearly weighted kappa coefficient for ordinal scales”, Statistical Methodology, 6:2 (2009), 157–163 | DOI | MR

[15] D. Vaughn, D. Justice, On The Direct Maximization of Quadratic Weighted Kappa, 2015, arXiv: 1509.07107v1