Using the Mask-RCNN convolutional neural network to automate the construction of two-dimensional solid vertebral models

A. S. Beskrovny ; L. V. Bessonov ; D. V. Ivanov ; I. V. Kirillova ; L. Yu. Kossovich

Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, Tome 20 (2020) no. 4, pp. 502-516.

Résumé

Biomechanical modeling requires the construction of an accurate solid model of the object under study based on the data of a particular patient. This problem can be solved manually using modern software packages for medical data processing or using computer-aided design systems. This approach is used by many researchers and allows you to create accurate solid models, but is time consuming. In this regard, the automation of the construction of solid models suitable for performing biomechanical calculations is an urgent task and can be carried out using neural network technologies. This study presents the implementation of one of the methods for processing computed tomography data in order to create two-dimensional accurate solid models of vertebral bodies in a sagittal projection. An artificial neural network Mask-RCNN was used for automatic recognition of vertebrae. The assessment of the quality of the automatic recognition performed by the neural network was carried out on the basis of comparison with the Sörensen measure with manual segmentation performed by practitioners. Application of the method makes it possible to significantly speed up the process of modeling bone structures of the spine in 2D mode. The implemented technique was used in the development of a solid-state model module, which is included in the SmartPlan Ortho 2D medical decision support system developed at Saratov State University within the framework of the Advanced Research Foundation project.

Bibliographie

[1] Kazhanov I. V., Mikityuk S. I., Dol' A.V., Ivanov D. V., Kharlamov A. V., Petrov A. V., Kossovich L. Yu., Manukovskiy V. A., “Biomechanical Modeling of Options for Internal Fixation of Unilateral Fractures of the Sacrum”, Traumatology and Orthopedics of Russia, 26:2 (2020), 79–90 (in Russian) | DOI | MR

[2] Kudjashev A. L., Hominec V. V., Teremshonok A. V., Korostelev K. E., Nagornyj E. B., Dol A. V., Ivanov D. V., Kirillova I. V., Kossovich L. Yu., “Biomechanical background for the formation of proximal junctional kyphosis after the transpedicular fixation of the lumbarian spine”, Russian Journal of Biomechanics, 21:3 (2017), 270–278 | DOI

[3] Donnik A. M., Ivanov D. V., Kossovich L. Ju., Levchenko K. K., Kireev S. I., Morozov K. M., Ostrovskij N. V., Zareckov V. V., Lihachev S. V., “Creation of 3D Solid Models of the Spine with Transpedicular Fixation Using Specialized Software”, Izv. Saratov Univ. (N. S.), Ser. Math. Mech. Inform., 19:4 (2019), 424–438 (in Russian) | DOI | MR

[4] Stewart R. D., Fermin I., Opper M., “Region growing with pulse-coupled neural networks: An alternative to seeded region growing”, IEEE Trans. on Neural Networks, 13:6 (2002), 1557–1562 | DOI

[5] Chandhok C., “A Novel Approach to Image Segmentation using Artificial Neural Networks and K-Means Clustering”, International Journal of Engineering Research and Applications, 2:3 (2012), 274–279 | DOI

[6] Belim S. V., Larionov S. B., “Image segmentation algorithm using an artificial neural network without using other images”, Radio Engineering, 2017, no. 3, 43–53 (in Russian) | DOI

[7] Dol D., Dol A., Bessonov L., Ivanov. D., Beskrovny A., Falkovich A., Ostrovsky N., “Methods of constructing an outline simple closed contour for modeling functional spine unit on CT slice”, Progress in Biomedical Optics and Imaging, Proceedings of SPIE, 11229, 2020, 112291Q | DOI

[8] Medvedev D. G., “Algorithm for highlighting the outline of an object with fuzzy edges”, Actual problems of the humanities and natural sciences, 2:1 (2014), 56–61 (in Russian)

[9] Withey D. J., Koles Z. J., “Medical Image Segmentation: Methods and Software”, Joint Meeting of the 6th International Symposium on Noninvasive Functional Source Imaging of the Brain and Heart and the International Conference on Functional Biomedical Imaging (Hangzhou, 2007), 140–143 | DOI

[10] Beskrovny A. S., Makhankov A. V., Bessonov L. V., Lemeshkin M. O., “Application of artificial neural network technologies to vertebral segmentation according on CT data”, Progress in Biomedical Optics and Imaging, Proceedings of SPIE, 11229, 2020, 112291Y | DOI

[11] Kolesnikova A. S., Bessonov L. V., Luneva A D., Dmitriev P. O., Matershev I. V., Kurchatkin A. A., Zolotov V. S., Sidorenko D. A., Chuvashkin V. K., Varjuhin A. A., Gushhina S. G., “Development of the approach for improvements method of active circles”, Information Technologies for the Physician, 3 (2018), 61–72 (in Russian)

[12] Long J., Shelhamer E., Darrell T., “Fully convolutional networks for semantic segmentation”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Boston, Massachusetts, 2015), 3431–3440 | DOI

[13] Badrinarayanan V., Kendall A., Cipolla R., “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation”, IEEE Trans. Pattern Anal. Mach. Intell., 39:12 (2017), 2481–2495 | DOI

[14] Kamnitsas K., Ferrante E., Parisot S., Ledig C., Nori A. V., Criminisi A., Rueckert D., Glocker B., “DeepMedic for Brain Tumor Segmentation”, Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, BrainLes: 2016, Lecture Notes in Computer Science, 10154, eds. A. Crimi, B. Menze, O. Maier, M. Reyes, S. Winzeck, H. Handels, Springer, Cham, 138–149 | DOI

[15] Noh H., Hong S., Han B., “Learning deconvolution network for semantic segmentation”, Proc. IEEE Int. Conf. Comput. Vis. (ICCV) (Santiago, 2015), 2015, 1520–1528 | DOI

[16] Chaurasia A., Culurciello E., “LinkNet: Exploiting encoder representations for efficient semantic segmentation”, IEEE Visual Communications and Image Processing (VCIP) (St. Petersburg, FL, 2017), 2017, 1–4 | DOI

[17] He K., Gkioxari G., Dollár P., Girshick R., “Mask R-CNN”, IEEE International Conference on Computer Vision (ICCV) (Venice, 2017), 2017, 2980–2988 | DOI

[18] He K., Zhang X., Ren S., Sun J., “Deep Residual Learning for Image Recognition”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Las Vegas, NV, 2016), 2016, 770–778 | DOI

[19] Samarskij A. A., Gulin A. V., Numerical methods, Nauka, M., 1989, 432 pp. (in Russian)

[20] Yao A. D., Cheng D. L., Pan I., Kitamura F., “Deep Learning in Neuroradiology: A Systematic Review of Current Algorithms and Approaches for the New Wave of Imaging Technology”, Radiology: Artificial Intelligence, 2:2 (2020), 6 | DOI

[21] Ivanov D. V., Kirillova I. V., Kossovich L. Yu., Bessonov L. V., Petraikin A. V., Dol A. V., Ahmad E. S., Morozov S. P., Vladzymyrskyy A. V., Sergunova K. A., Kharlamov A. V., “Influence of Convolution Kernel and Beam-Hardening Effect on the Assessment of Trabecular Bone Mineral Density Using Quantitative Computed Tomography”, Izv. Saratov Univ. (N. S.), Ser. Math. Mech. Inform., 20:2 (2020), 205–219 | DOI | MR

[22] Sekuboyina A., Bayat A., Husseini M. E., Löffler M., Rempfler M., Kukacka J., Tetteh G., Valentinitsch A., Payer C., Urschler M., Chen M., Cheng D., Lessmann N., Hu Y., Wang T., Yang D., Xu D., Ambellan F., Zachow S., Jiang T., Ma X., Angerman Ch., Wang X., Wei Q. Brown K., Wolf M., Kirszenberg A., Puybareauq É., Menze B. H., Kirschke J., VerSe: A Vertebrae Labelling and Segmentation Benchmark, 2020, 30 pp., arXiv: 2001.09193 [cs.CV]