Combined 2D/3D approach for improving the accuracy of face recognition systems using deep learning

K. A. Dorofeev ; A. N. Ruchay

Čelâbinskij fiziko-matematičeskij žurnal, Tome 7 (2022) no. 4, pp. 490-504.

Résumé

Face recognition systems using color images are widely known. However, their main problem is the instability to various lighting conditions, emotional and facial expressions, overlaps and rotation angles. The article proposed a new approach that combines the processing of two-dimensional data on a color image and three-dimensional data on a point cloud or depth map. The main attention was paid to the architecture of neural networks, the quality and accuracy of recognition. Some combined methods for the face recognition using machine learning and deep learning have been proposed. A comparative analysis of the results of experiments in terms of the recognition accuracy on open face databases was carried out. The best combined method was chosen. The main goal is to build a reliable, accurate combined biometric face recognition system that is resistant to complex external factors, such as facial expressions, scale changes, lighting, partial overlap with foreign objects, large rotation angles.

Détail

Keywords: information security, biometrics, face recognition, neural networks, machine learning, deep learning, identification, authentication, multi-biometrics.

Bibliographie

[1] Callaway S., Cheng J., Contratti A., Fu D., Gelivi H., Wachulec J., Purohit S., “Comparative analysis of image processing algorithms for airport security”, 2020 IEEE MIT Undergraduate Research Technology Conference (URTC), 2020 | Zbl

[2] Bansal M., Sharma D., “Facial recognition system for security resolutions in smart city”, International Journal of Advanced Research in Engineering and Technology, 11:10 (2020), 146–151

[3] Praveen G., Dakala J., Face recognition: Challenges and issues in smart city/environments, International Conference on Communication Systems and Networks, 2020, 791–793

[4] Taigman Y., Yang M., Ranzato M., Wolf L., “Deepface: Closing the gap to human-level performance in face verification”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2014, 1701–1708

[5] He R., Zhang X., Ren S., Sun J., “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification”, Proceedings of IEEE International Conference on Computer Vision, 2015, 1026–1034 | Zbl

[6] Sun Y., Wang X., Tang X., “Deeply learned face representations are sparse, selective, and robust”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015, 2892–2900 | MR

[7] Goodfellow I., Bengio Y., Courville A., Deep Learning, MIT Press, Cambridge, MA, 2016 | MR | Zbl

[8] Guo Y., Liu Y., Oerlemans A., Lao S., Wu S., Lew M. S., “Deep learning for visual understanding: A review”, Neurocomputing, 187 (2016), 27–48 | DOI

[9] Rawat W., Wang Z., “Deep convolutional neural networks for image classification: A comprehensive review”, Neural Computation, 2017 | MR

[10] Schroff F., Kalenichenko D., Philbin J., “Facenet: A unified embedding for face recognition and clustering”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015, 815–823

[11] Sun Y., Liang D., Wang X., Tang X., “Deepid3: Face recognition with very deep neural networks”, 2015, arXiv: 1502.00873

[12] Zhou E., Cao Z., Yin Q., Naive-deep face recognition: Touching the limit of LFW benchmark or not?, 2015, arXiv: 1501.04690

[13] Li B. Y., Mian A., Liu W., Krishna A., “Using kinect for face recognition under varying poses, expressions, illumination and disguise”, IEEE Workshop on Applications of Computer Vision (WACV), 2013

[14] Abbad A., Abbad K., Tairi H., “3D Face recognition: Multi-scale strategy based on geometric and local descriptors”, Computers Electrical Engineering, 70 (2018), 525–537 pp.

[15] Dorofeev K., Ruchay A., Kober A., Kober V., “3D face recognition using depth filtering and deep convolutional neural network”, Applications of Digital Image Processing, XLII (2019), 11137 pp.

[16] Ruchay A., Dorofeev K., Kalschikov V., “A switching morphological algorithm for depth map recovery”, Analysis of Images, Social Networks and Texts: 8th International Conference, 2019

[17] Dorofeev K., Ruchay A., “Design of autonomous mobile systems for face recognition based on a DCNN with compression and pruning”, Applications of Digital Image Processing, XLIII (2020), 11510

[18] Kim D., Hernandez M., Choi J., Medioni G., “Deep 3d face identification”, 2017, arXiv: 1703.10714

[19] Gilani S. Z., Mian A., “Learning from millions of 3d scans for large-scale 3d face recognition”, 2018, arXiv: 1711.05942

[20] Cai Y., Lei Y., Yang M., You Z., Shan S., “A fast and robust 3d face recognition approach based on deeply learned face representation”, Neurocomputing, 363 (2019), 375–397 | DOI

[21] Chui M., Cheng H., Wang C., Lai S., High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network, Department of Computer Science, National Tsing Hua University, Taiwan, Microsoft AI R Center, 2018

[22] Lee Y.-C., Chen J., Tseng C. W., Lai S.-H., “Accurate and robust face recognition from RGB-D images with a deep learning approach”, Proceedings of the British Machine Vision Conference (BMVC), 2016

[23] Xiong X., Wen X., Huang C., “Improving RGB-D face recognition via transfer learning from a pretrained 2D network”, International Symposium on Benchmarking, Measuring and Optimization, 2019

[24] Jiang L., Zhang J., Deng B., “Robust RGB-D face recognition using attribute-aware loss”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019

[25] Charles R., Hao S., Kaichun M., Leonidas J., PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, Stanford University, Stanford, 2017

[26] Zhang Z., Da F., Yu Y., Data-free point cloud network for 3d face recognition, 2019, arXiv: 1911.04731

[27] Feng Y., Zhang Z., Zhao X., Ji R., Gao Y., “Group-view convolutional neural networks for 3d shape recognition”, Conference on Computer Vision and Pattern Recognition (CVPR), 2018

[28] Jackson A. S., Bulat A., Argyriou V., Tzimiropoulos G., “Large pose 3D face reconstruction from a single image via direct volumetric CNN regression”, International Conference on Computer Vision (ICCV), 2017

[29] Savran A., Alyuz N., Dibeklioglu H., Celiktutan O., Gokberk B., Sankur B., Akarun L., “Bosphorus database for 3D face analysis”, Workshop on Biometrics and Identity Management, 2008

[30] Cao Q., Shen L., Xie W., Parkhi O., Zisserman A., “VGGFace2: A dataset for recognising faces across pose and age”, IEEE Conference on Automatic Face and Gesture Recognition (F), 2018

[31] Guo Y., Zhang L., Hu Y., He X., Gao J., “MS-Celeb-1M: A dataset and benchmark for large scale face recognition”, European Conference on Computer Vision (ECCV), 2016