Training Viola–Jones detectors for 3D objects based on fully synthetic data for use in rescue missions with UAV

S. A. Usilin   ; V. V. Arlazarov   ; N. S. Rokhlin   ; S. A. Rudyka   ; S. A. Matveev   ; A. A. Zatsarinnyy

Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ, Matematičeskoe modelirovanie i programmirovanie, Tome 13 (2020) no. 4, pp. 94-106

Résumé

In this paper, the problem of training the Viola–Jones detector for 3D objects is considered on the example of an inflatable life raft PSN-10. The detector is trained on a fully synthetic training dataset. The paper discusses in detail the methods of modelling an inflatable life raft, water surface, various weather conditions. As a feature space, we use edge Haar-like features, which allow training the detector that is resistant to various lighting conditions. To increase the computational efficiency, the L1 norm is used to calculate the magnitude of the image gradient. The performance of the trained detector is estimated on real data obtained during the rescue operation of the trawler “Dalniy Vostok”. The proposed method for training the Viola–Jones detectors can be successfully used as a component of hardware and software “assistants” of the UAV.

Détail

Keywords: machine learning, object detection, Viola–Jones, rescue mission.
Mots-clés : classification, 3D object, UAV

Bibliographie

[1] Hongyang Yu, Guorong Li, Weigang Zhang, Qingming Huang, Dawei Du, Qi Tian, Nicu Sebe, “The Unmanned Aerial Vehicle Benchmark: Object Detection, Tracking and Baseline”, International Journal of Computer Vision, 128:5 (2020), 1141–1159 | DOI

[2] Dumin D., Dinh T. D., Pham V. D., Kirichek R., “Application of Installed Systems of GSM-Device Detection on UAVs for Searching Victim in Result of Emergency Situations”, Information Technologies and Telecommunications, 6:2 (2018), 62–69 | MR

[3] Matveev S. A., Rudyka S. A., Petrov Yu.V., Zhdanov A. S., “Onboard Complex of Information Support of Search and Rescue Operations in Arctic”, Issues of Radio Electronics, 2019, no. 6, 30–37 (in Russian) | DOI

[4] Garmash V. N., Korobochkin D. M., Matveev S. A., Petrov Yu.V., Rudyka S. A., Sukhov T. M., “Complexing Information from Different Sources in the On-Board Systems Search and Rescue Operations”, Issues of Radio Electronics, 2018, no. 7, 30–37 (in Russian) | DOI

[5] Matveev S. A., Bizov A. N., Bistrov S.Yu., Garmash V. N., Isenko S. I., Korobochkin D. M., Petrov Yu.V., Rudika S. A., Strahov S.Yu., Sircev A. N., “Helicopter System that Provide Information Support for Safety of Flights and Conduct Search and Rescue Operations”, Bulletin of the Kyrgyz-Russian Slavic University, 18:12 (2018), 60–64

[6] Leira F. S., Johansen T. A., Fossen T. I., “Automatic Detection, Classification and Tracking of Objects in the Ocean Surface from UAVs Using a Thermal Camera”, IEEE Aerospace Conference (Big Sky, USA, 2015), 1–10

[7] Du Dawei, Yunkai Qi, Hongyang Yu, Yifan Yang, Kaiwen Duan, Guorong Li, Weigang Zhang, Qingming Huang, Qi Tuan, “The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking”, Proceedings of the European Conference on Computer Vision, 2018, 370–386

[8] Matalov D. P., Usilin S. A., Arlazarov V. V., “Single-Sample Augmentation Framework for Training Viola–Jones Classifiers”, 12th International Conference on Machine Vision (Munich, Germany, 2020), 11433, 1–9 | DOI

[9] Emelyanov S. O., Ivanova A. A., Shvets E. A., Nikolaev D. P., “Methods of Training Data Augmentation in the Task of Image Classification”, Sensory Systems, 32:3 (2018), 236–245 | DOI

[10] Arlazarov V. V., Slavin O. A., Uskov A. V., Janiszewski I. M., “Modelling the Flow of Character Recognition Results in Video Stream”, Bulletin of the South Ural State University: Mathematical Modelling, Programming and Computer Software, 11:2 (2018), 14–28 | DOI | Zbl

[11] Bulatov K. B., “A Method to Reduce Errors of String Recognition Based on Combination of Several Recognition Results with Per-Character Alternatives”, Bulletin of the South Ural State University: Mathematical Modelling, Programming and Computer Software, 12:3 (2019), 74–88 | DOI | Zbl

[12] Chernyshova Y. S., Sheshkus A. V., Arlazarov V. V., “Two-Step CNN Framework for Text Line Recognition in Camera-Captured Images”, IEEE Access, 2020, no. 8, 32587–32600 | DOI

[13] Gayer A. V., Chernyshova Y. S., Sheshkus A. V., “Artificial Training Data Generation for the Task of Character Recognition of Fields of Russian Passport”, Sensory Systems, 32:3 (2018), 230–235 | DOI

[14] Danielczuk M., Matl M., Gupta S., Li A., Lee A., Mahler J., Goldberg K., “Segmenting Unknown 3D Objects from Real Depth Images using Mask R-CNN Trained on Synthetic Data”, International Conference on Robotics and Automation, 2019, 7283–7290 | DOI

[15] Akimov A. V., Sirota A. A., “Synthetic Data Generation Models and Algorithms for Training Image Recognition Algorithms Using the Viola–Jones Framework”, Computer Optics, 40:6 (2016), 911–918 | DOI

[16] Mogelmose A., Trivedi M. M., Moeslund T. B., “Learning to Detect Traffic Signs: Comparative Evaluation of Synthetic and Real-World Datasets”, Proceedings of the 21st International Conference on Pattern Recognition, 2012, 3452–3455

[17] Afanasyev I. I., Laptev V. N., Pirogov V. P., “Analysis of the Rescue Assets Range of the Russian Navy”, Scientific Bulletin of the Volsk Military Institute of Material Support: Military Scientific Journal, 2015, no. 2, 150–154

[18] Viola P., Jones M., “Rapid Object Detection Using a Boosted Cascade of Simple Features”, Computer Vision and Pattern Recognition, 2001, no. 1, 511–518

[19] Viola P., Jones M., “Robust Real-Time Object Detection”, International Journal of Computer Vision, 2001, no. 4, 34–47

[20] Papageorgiou C. P., Oren M., Poggio T., “A General Framework for Object Detection”, Sixth International Conference Computer Vision, 1998, 555–562

[21] Lewis J. P., “Fast Template Matching”, Proceedings Vision Interface, 1995, 120–123

[22] Kotov A. A., Usilin S. A., Gladilin S. A., Nikolaev D. P., “Construction of Robust Features for Detection and Classification of Objects without Characteristic Brightness Contrasts”, Journal of Information Technologies and Computing Systems, 2014, no. 1, 53–60

[23] Matalov D. P., Usilin S. A., Arlazarov V. V., “Modification of the Viola–Jones Approach for the Detection of the Government Seal Stamp of the Russian Federation”, Eleventh International Conference on Machine Vision, 2019, 11041 | DOI

[24] Home of the Blender Project – Free and Open 3D Creation Software, , 2020 https://www.blender.org/