Geodesic
Relative cost curves: An alternative to AUC and an extension to 3-class problems
Kybernetika, Tome 50 (2014) no. 5, pp. 647-660
Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

Performance evaluation of classifiers is a crucial step for selecting the best classifier or the best set of parameters for a classifier. Receiver Operating Characteristic (ROC) curves and Area Under the ROC Curve (AUC) are widely used to analyse performance of a classifier. However, the approach does not take into account that misclassification for different classes might have more or less serious consequences. On the other hand, it is often difficult to specify exactly the consequences or costs of misclassifications. This paper is devoted to Relative Cost Curves (RCC) - a graphical technique for visualising the performance of binary classifiers over the full range of possible relative misclassification costs. This curve provides helpful information to choose the best set of classifiers or to estimate misclassification costs if those are not known precisely. In this paper, the concept of Area Above the RCC (AAC) is introduced, a scalar measure of classifier performance under unequal misclassification costs problem. We also extend RCC to multicategory problems when misclassification costs depend only on the true class.
Performance evaluation of classifiers is a crucial step for selecting the best classifier or the best set of parameters for a classifier. Receiver Operating Characteristic (ROC) curves and Area Under the ROC Curve (AUC) are widely used to analyse performance of a classifier. However, the approach does not take into account that misclassification for different classes might have more or less serious consequences. On the other hand, it is often difficult to specify exactly the consequences or costs of misclassifications. This paper is devoted to Relative Cost Curves (RCC) - a graphical technique for visualising the performance of binary classifiers over the full range of possible relative misclassification costs. This curve provides helpful information to choose the best set of classifiers or to estimate misclassification costs if those are not known precisely. In this paper, the concept of Area Above the RCC (AAC) is introduced, a scalar measure of classifier performance under unequal misclassification costs problem. We also extend RCC to multicategory problems when misclassification costs depend only on the true class.
DOI : 10.14736/kyb-2014-5-0647
Classification : 62A10, 62N05, 93E12
Keywords: classifier; performance evaluation; misclassification costs; cost curves; ROC curves; AUC 
@article{10_14736_kyb_2014_5_0647,
     author = {Montvida, Olga and Klawonn, Frank},
     title = {Relative cost curves: {An} alternative to {AUC} and an extension to 3-class problems},
     journal = {Kybernetika},
     pages = {647--660},
     year = {2014},
     volume = {50},
     number = {5},
     doi = {10.14736/kyb-2014-5-0647},
     mrnumber = {3301852},
     zbl = {1305.93195},
     language = {en},
     url = {http://geodesic.mathdoc.fr/articles/10.14736/kyb-2014-5-0647/}
}
TY  - JOUR
AU  - Montvida, Olga
AU  - Klawonn, Frank
TI  - Relative cost curves: An alternative to AUC and an extension to 3-class problems
JO  - Kybernetika
PY  - 2014
SP  - 647
EP  - 660
VL  - 50
IS  - 5
UR  - http://geodesic.mathdoc.fr/articles/10.14736/kyb-2014-5-0647/
DO  - 10.14736/kyb-2014-5-0647
LA  - en
ID  - 10_14736_kyb_2014_5_0647
ER  - 
%0 Journal Article
%A Montvida, Olga
%A Klawonn, Frank
%T Relative cost curves: An alternative to AUC and an extension to 3-class problems
%J Kybernetika
%D 2014
%P 647-660
%V 50
%N 5
%U http://geodesic.mathdoc.fr/articles/10.14736/kyb-2014-5-0647/
%R 10.14736/kyb-2014-5-0647
%G en
%F 10_14736_kyb_2014_5_0647
Montvida, Olga; Klawonn, Frank. Relative cost curves: An alternative to AUC and an extension to 3-class problems. Kybernetika, Tome 50 (2014) no. 5, pp. 647-660. doi: 10.14736/kyb-2014-5-0647

[1] Bradley, A. P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30 (1997), 1145-1159. | DOI

[2] Drummond, C., Holte, R. C.: Cost curves: An improved method for visualizing classifier performance. Machine Learning 65 (2006) 95-130. | DOI

[3] Fawcett, T.: An introduction to roc analysis. Pattern Recognition Lett. 27 (2006), 861-874. | DOI

[4] Hand, D. J.: Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning 77 (2009), 103-123. | DOI

[5] Hand, D. J., Till, R. J.: A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 45 (2001), 171-186. | DOI | Zbl

[6] Hanley, J. A.: Receiver operating characteristic (ROC) methodology: the state of the art. Critical Reviews in Diagnostic Imaging 29 (1989), 307-335.

[7] Hernández-Orallo, J., Flach, P., Ferri, C.: Brier curves: a new cost-based visualisation of classifier performance. In: Proc. 28th International Conference on Machine Learning (ICML-11) (L. Getoor and T. Scheffer, eds.), ACM, New York 2011, pp. 585-592.

[8] Klawonn, F., Höppner, F., May, S.: An alternative to ROC and AUC analysis of classifiers. In: Advances in Intelligent Data Analysis X, (J. Gama, E. Bradley, and J. Hollmén, eds.), Springer, Berlin 2011, p. 210-221.

[9] Krzanowski, W. J., Hand, D. J.: ROC Curves for Continuous data. Chapman and Hall, London 2009. | MR | Zbl

[10] Li, J., Fine, J. P.: ROC analysis with multiple classes and multiple tests: methodology and its application in microarray studies. Biostatistics 9 (2008), 566-576. | DOI | Zbl

[11] Murphy, P. M., Aha, D. W.: Uci repository of machine learning databases. 1992. Avaible: http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)

[12] Mossman, D.: Three-way ROCs. Medical Decision Making 19 (1999), 78-89. | DOI

Cité par Sources :