Multimodal ensemble neural network system for skin cancer detection on heterogeneous dermatological data

U. A. Lyakhova   ; P. A. Lyakhov

Vestnik Sankt-Peterburgskogo universiteta. Prikladnaâ matematika, informatika, processy upravleniâ, Tome 20 (2024) no. 2, pp. 231-243

Résumé

Today, skin cancer is one of the leading causes of death in the world. Diagnosing skin cancer early is critical to increasing potential survival. Therefore, it is relevant to develop high-precision intelligent auxiliary diagnostic systems for detecting skin cancer in the early stages. Ensemble learning is one of the current and promising methods for increasing the accuracy of intelligent classification systems by reducing the dispersion and variability of predictions of individual components of the overall system. The work proposes an ensemble intelligent system for analyzing heterogeneous dermatological data based on multimodal neural networks. The accuracy of the developed ensemble system was 85.92 %, which is 1.85 percentage points higher than the average accuracy of individual multimodal architectures for classifying heterogeneous dermatological data. The developed system can be used as a high-precision auxiliary diagnostic tool to help make a medical decision, which will increase the chance of early detection of pigmented oncological pathologies.

Détail

Keywords: multimodal neural network, ensemble neural network, machine learning, heterogeneous data, dermatological images, pigmented skin lesions, skin cancer, melanoma.

Bibliographie

[1] Apalla Z., Lallas A., Sotiriou E., Lazaridou E., Ioannides D., “Epidemiological trends in skin cancer”, Dermatol. Pract. Concept., 7:2 (1885), 1

[2] Sinz C., Tschandl P., Rosendahl C., Akay B. N., Argenziano G., Blum A., Kittler H., “Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin”, Journal of Acad. Dermatol., 77:6 (2017), 1100–1109 | DOI

[3] Lodha S., Saggar S., Celebi J. T., Silvers D. N., “Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting”, Journal of Cutan Pathol., 35:4 (2008), 349–352 | DOI

[4] Haggenmüller S., Maron R. C., Hekler A., Utikal J. S., Barata C., Barnhill R. L., Brinker T. J., “Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts”, Eur. Journal of Cancer., 156 (2021), 202–216 | DOI

[5] Polikar R., Zhang C., Ma Y., “Ensemble Learning”, Ensemble Machine Learning, 2012, 1–34 | MR

[6] Qureshi A. S., Roos T., “Transfer learning with ensembles of deep neural networks for skin cancer detection in imbalanced data sets”, Neural Process Lett., 55:4 (2023), 4461–4479 | DOI

[7] Raza R., Zulfiqar F., Tariq S., Anwar G. B., Sargano A. B., Habib Z., “Melanoma classification from dermoscopy images using ensemble of convolutional neural networks”, Mathematics, 10:1 (2021), 26–43 | DOI

[8] Kausar N., Hameed A., Sattar M., Ashraf R., Imran A. S., Abidin M. Z. U., Ali A., “Multiclass skin cancer classification using ensemble of fine-tuned deep learning models”, Applied Sciences, 11:22 (2021), 10593–10608 | DOI

[9] Lu Y., Zhang L., Wang B., Yang J., “Feature ensemble learning based on sparse autoencoders for image classification”, Proceedings of the International Joint Conference on Neural Networks, 2014, 1739–1745

[10] Tang E. K., Suganthan P. N., Yao X., “An analysis of diversity measures”, Machine Learning, 65:1 (2006), 247–271 | DOI | Zbl

[11] Baltrušaitis T., Ahuja C., Morency L. P., “Multimodal machine learning”, IEEE Trans. Pattern Anal. Mach. Intell., 41:2 (2019), 423–443 | DOI

[12] Liu K., Li Y., Xu N., Natarajan P., Learn to combine modalities in multimodal deep learning, 2018, arXiv: 1805.11730

[13] Kurtansky N. R., Dusza S. W., Halpern A. C., Hartman R. I., Geller A. C., Marghoob A. A., Marchetti M. A., “An epidemiologic analysis of melanoma overdiagnosis in the United States, 1975-2017”, Journal of Investigative Dermatology, 142:7 (2022), 1804–1811 | DOI

[14] Höhn J., Hekler A., Krieghoff-Henning E., Kather J. N., Utikal J. S., Meier F., Brinker T. J., “Integrating patient data into skin cancer classification using convolutional neural networks: systematic review”, Journal of Medical Internet Research, 23:7 (2021), 20708–20723

[15] Sriwong K., Bunrit S., Kerdprasop K., Kerdprasop N., “Dermatological classification using deep learning of skin image and patient background knowledge”, International Journal of Machine Learning and Computing, 9:6 (2019), 862–867 | DOI

[16] Siegel J. A., Korgavkar K., Weinstock M. A., “Current perspective on actinic keratosis: a review”, British Journal of Dermatology, 177:2 (2017), 350–358 | DOI

[17] Lyakhov P. A., Lyakhova U. A., Nagornov N. N., “System for the recognizing of pigmented skin lesions with fusion and analysis of heterogeneous data based on a multimodal neural network”, Cancers, 14 (2022), 1819–1836 | DOI

[18] Chicco D., Jurman G., “The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation”, BMC Genomics, 21:1 (2020), 1–13 | DOI

[19] Harangi B., Baran A., Hajdu A., “Classification of skin lesions using an ensemble of deep neural networks”, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2018, 2575–2578

[20] Akter M. S., Shahriar H., Sneha S., Cuzzocrea A., “Multi-class skin cancer classification architecture based on deep convolutional neural network”, 2022 IEEE International Conference on Big Data. Proceedings, 2022, 5404–5413 | DOI

[21] Keerthana D., Venugopal V., Nath M. K., Mishra M., “Hybrid convolutional neural networks with SVM classifier for classification of skin cancer”, Biomedical Engineering Advances, 5 (2023), 100069–100103 | DOI