An intelligent multimodal framework for identifying children with autism spectrum disorder

Chen, Jingying ; Liao, Mengyi ; Wang, Guangshuai ; Chen, Chang

International Journal of Applied Mathematics and Computer Science, Tome 30 (2020) no. 3, pp. 435-448.

Résumé

Early identification can significantly improve the prognosis of children with autism spectrum disorder (ASD). Yet existing identification methods are costly, time consuming, and dependent on the manual judgment of specialists. In this study, we present a multimodal framework that fuses data on a child’s eye fixation, facial expression, and cognitive level to automatically identify children with ASD, to improve the identification efficiency and reduce costs. The proposed methodology uses an optimized random forest (RF) algorithm to improve classification accuracy and then applies a hybrid fusion method based on the data source and time synchronization to ensure the reliability of the classification results. The classification accuracy of the framework was 91

Détail

Keywords: autism spectrum disorder, eye fixation, facial expression, cognitive level, improved random forest
Mots-clés : spektrum zaburzeń autystycznych, wyraz twarzy, poziom poznawczy, las losowy

Bibliographie

[1] Achenbach, T. and Rescorla, L. (2000). Manual for the ASEBA Preschool Forms Profiles, University of Vermount, Burlington, VA.

[2] Al-Jarrah, O.Y., Yoo, P.D., Muhaidat, S., Karagiannidis, G.K. and Taha, K. (2015). Efficient machine learning for big data: A review, Big Data Research 2(3): 87–93.

[3] Amaral, D.G., Schumann, C.M. and Nordahl, C.W. (2008). Neuroanatomy of autism, Trends in Neurosciences 31(3): 137–145.

[4] Ashwin, C., Hietanen, J.K. and Baron-Cohen, S. (2015). Atypical integration of social cues for orienting to gaze direction in adults with autism, Molecular Autism 6(1): 5–14.

[5] Baron-Cohen, S., Jolliffe, T., Mortimore, C. and Robertson, M. (1997). Another advanced test of theory of mind: Evidence from very high functioning adults with autism or Asperger syndrome, Journal of Child Psychology and Psychiatry 38(7): 813–822.

[6] Bernier, R., Mao, A. and Yen, J. (2011). Diagnosing autism spectrum disorders in primary care, Practitioner 255(1745): 27–30.

[7] Chitategmark, M. (2016). Social attention allocation in ASD: A review and meta-analysis of eye-tracking studies, Review Journal of Autism Developmental Disorders 3(3): 209–223.

[8] Christensen, D.L., Baio, J., Braun, K.V.N., Bilder, D., Charles, J., Constantino, J.N., Daniels, J., Durkin, M.S., Fitzgerald, R.T. and Kurziusspencer, M. (2016). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2012, Morbidity and Mortality Weekly Report—Surveillance Summaries 65(3): 1–23.

[9] Constantino, J.N., Kennon-Mcgill, S., Weichselbaum, C., Marrus, N. and Jones, W. (2017). Infant viewing of social scenes is under genetic control and is atypical in autism, Nature 547(7663): 340–344.

[10] Drimalla, H., Landwehr, N., Baskow, I., Behnia, B. and Scheffer, T. (2018). Detecting autism by analyzing a simulated social interaction, Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Berlin, Germany, pp. 193–208.

[11] Durkin, M., Maenner, M.J., Meaney, F.J., Levy, S.E. and DiGuiseppi, C. (2010). Socioeconomic inequality in the prevalence of autism spectrum disorder: Evidence from a US cross-sectional study, PLoS ONE 5(7): e11551.

[12] Eack, S.M., Mazefsky, C.A. and Minshew, N.J. (2015). Misinterpretation of facial expressions of emotion in verbal adults with autism spectrum disorder, Autism 19(3): 308–315.

[13] Gan, Y.L., Chen, J.Y. and Xu, L.H. (2019). Facial expression recognition boosted by soft label with a diverse ensemble, Pattern Recognition Letters 125(4): 105–112.

[14] Greene, D.J., Colich, N., Iacoboni, M., Zaidel, E., Bookheimer, S.Y. and Dapretto,M. (2011). Atypical neural networks for social orienting in autism spectrum disorders, Neuroimage 56(1): 354–362.

[15] Halim, A., Ford, G., Eric, G. and Wall Dennis, P. (2018). Machine learning approach for early detection of autism by combining questionnaire and home video identification, Journal of the American Medical Informatics Association 25(8): 1000–1007.

[16] Hubert, B., Wicker, B., Moore, D.G., Monfardini, E., Duverger, H., Da Fonseca, D. and Deruelle, C. (2007). Brief report: Recognition of emotional and non-emotional biological motion in individuals with autistic spectrum disorders, Journal of Autism and Developmental Disorders 37(7): 1386–1392.

[17] Jaiswal, S., Valstar, M.F., Gillott, A. and Daley, D. (2017). Automatic detection of ADHD and ASD from expressive behaviour in RGBD data, IEEE International Conference on Automatic Face Gesture Recognition, Washington, DC, USA, pp. 762–769.

[18] Jiang, M., Sunday, M., Francis and Srishyla, D. (2019). Classifying individuals with ASD through facial emotion recognition and eye-tracking, Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Berlin, Germany, pp. 6063–6068.

[19] Kantavat, P., Kijsirikul, B., Songsiri, P., Fukui, K.-I. and Numao, M. (2018). Efficient decision trees for multi-class support vector machines using entropy and generalization error estimation, International Journal of Applied Mathematics Computer Science 28(4): 705–717, DOI: 10.2478/amcs-2018-0054.

[20] Kerrianne, E., Morrison, A.E. and Pinkham, S.K. (2019). Psychometric evaluation of social cognitive measures for adults with autism, Autism Research 12(5): 766–778.

[21] Liu, W., Li, M. and Yi, L. (2016). Identifying children with autism spectrum disorder based on their face processing abnormality: A machine learning framework, Autism Research 9(8): 888–898.

[22] Manfredonia, J., Bangerter, A., Manyakov, N.V., Ness, S., Lewin, D., Skalkin, A., Boice, M., Goodwin, M. S., Dawson, G. and Hendren, R. (2018). Automatic recognition of posed facial expression of emotion in individuals with autism spectrum disorder, Journal of Autism and Developmental Disorders 27(10): 1–15.

[23] Müller and Frith, U. (2005). Autism-explaining the enigma, Kindheit Und Entwicklung 14(4): 257.

[24] Parkhi, O., Vedaldi, A. and Zisserman, A. (2015). Deep face recognition, British Machine Vision Conference, Swansea, UK, p. 6.

[25] Poria, S., Cambria, E., Bajpai, R. and Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion, Information Fusion 37(2): 98–125.

[26] Remington, A., Swettenham, J., Campbell, R. and Coleman, M. (2009). Selective attention and perceptual load in autism spectrum disorder, Psychological Science 20(11): 1388–1393.

[27] Rozga, A., Mumaw, M., King, T. and Robins, D.L. (2009). Lack of emotion-specific facial mimicry responses among high-functioning individuals with an autism spectrum disorder (poster), International Meeting for Autism Research, Chicago, IL, USA, pp. S43–S44.

[28] Rundo, L., Militello, C., Russo, G., Garufi, A., Vitabile, S. and Gilardi, M. (2017a). Automated prostate gland segmentation based on an unsupervised fuzzy c-means clustering technique using multispectral T1w and T2wMR imaging, Information 8(2): 1–28.

[29] Rundo, L., Stefano, A., Militello, C., Russo, G., Sabini, M.G. and Arrigo, C. (2017b). A fully automatic approach for multimodal PET and MR image segmentation in Gamma Knife treatment planning, Computer Methods Programs in Biomedicine 144(3): 77–96.

[30] Samad, M.D., Diawara, N., Bobzien, J.L., Harrington, J.W., Witherow, M.A. and Iftekharuddin, K.M. (2018). A feasibility study of autism behavioral markers in spontaneous facial, visual, and hand movement response data, IEEE Transactions on Neural Systems Rehabilitation Engineering PP(99): 1–1.

[31] Sasson, N.J. (2006). The development of face processing in autism, Journal of Autism Developmental Disorders 36(3): 381–394.

[32] Sasson, N.J., Elison, J.T., Turner-Brown, L.M., Dichter, G.S. and Bodfish, J.W. (2011a). Brief report: Circumscribed attention in young children with autism, Journal of Autism Developmental Disorders 41(2): 242–247.

[33] Sasson, N.J., Pinkham, A.E., Carpenter, K.L. and Belger, A. (2011b). The benefit of directly comparing autism and schizophrenia for revealing mechanisms of social cognitive impairment, Journal of Neurodevelopmental Disorders 3(2): 87–100.

[34] Sasson, N., Tsuchiya, N., Hurley, R., Couture, S.M., Penn, D.L., Adolphs, R. and Piven, J. (2007). Orienting to social stimuli differentiates social cognitive impairment in autism and schizophrenia, Neuropsychologia 45(11): 2580–2588.

[35] Serra, A., Galdi, P. and Tagliaferri, R. (2018). Machine learning for bioinformatics and neuroimaging, Wiley Interdisciplinary Reviews: Data Mining Knowledge Discovery 8(5): e1248.

[36] Shaddy, D.J. (2006). Visual scanning and pupillary responses in young children with autismspectrum disorder, Journal of Clinical Experimental Neuropsychology 28(7): 1238–1256.

[37] Tariq, Q., Daniels, J. and Schwartz, J.N. (2018). Mobile detection of autism through machine learning on home video: A development and prospective validation study, PLoS Medicine 15(11): e1002705.

[38] Traynor, J.M., Gough, A., Duku, E., Shore, D.I. and Hall, G.B.C. (2019). Eye tracking effort expenditure and autonomic arousal to social and circumscribed interest stimuli in autism spectrum disorder, Journal of Autism and Developmental Disorders 49(1): 1988–2002.

[39] Trevisan, D.A., Hoskyn, M. and Birmingham, E. (2018). Facial expression production in autism: A meta-analysis, Autism Research 11(2): 1586–1601.

[40] Wang, G.S., Chen, J.Y. and Zhang, K. (2018). The perception of emotional facial expressions by children with autism using hybrid multiple factorial design and eye-tracking, Chinese Science Bulletin 63(31): 3204–3216, (in Chinese).

[41] Wang, Y. and Chen, W. (2010). Broken mirror theory of autism, Advances in Psychological Science 18(2): 297–305.

[42] Xu, L., Fu, H.Y., Goodarzi, M., Cai, C.B., Yin, Q.B., Wu, Y., Tang, B.C. and She, Y. B. (2018). Stochastic cross validation, Chemometrics Intelligent Laboratory Systems 175(4): 74–81.

[43] Xu, M., Shen, J. and Yu, H.Y. (2017). A review on data-driven healthcare decision-making support, Industrial Engineering and Management 21(1): 1–13.

[44] Yi, H., Song, X.F., Jiang, B., Liu, Y.F. and Zhou, Z.H. (2013). Fault diagnosis based on self-tuning support vector machine in sample unbalance condition, Transactions of Beijing Institute of Technology 33(4): 394–398.

[45] Yi, L., Feng, C., Quinn, P.C., Ding, H., Li, J., Liu, Y. and Lee, K. (2014). Do individuals with and without autism spectrum disorder scan faces differently? A new multi-method look at an existing controversy, Autism Research 7(1): 72–83.

[46] Zhao, S., Uono, S., Yoshimura, S., Kubota, Y. and Toichi, M. (2017). Atypical gaze cueing pattern in a complex environment in individuals with ASD, Journal of Autism Developmental Disorders 47(7): 1978–1986.

[47] Zhong, S.S., Li, X. and Zhang, Y.J. (2019). Fault diagnosis of civil aero-engine driven by unbalanced samples based on DBN, Journal of Aerospace Power 34(3): 708–716.

[48] Zunino, A., Morerio, P. and Cavallo, A. (2018). Video gesture analysis for autism spectrum disorder detection, 24th International Conference on Pattern Recognition (ICPR), Beijing, China, pp. 3421–3426.

[49] Zwaigenbaum, L., Bryson, S., Lord, C., Rogers, S., Carter, A., Carver, L., Chawarska, K., Constantino, J., Dawson, G. and Dobkins, K. (2009). Clinical assessment and management of toddlers with suspected autism spectrum disorder: Insights from studies of high-risk infants, Pediatrics 123(5): 1383–1391.