A hierarchy of finite state machines as a scenario player in interactive training of pilots in flight simulators

Bach, Małgorzata ; Werner, Aleksandra ; Mrozik, Magda ; Cyran, Krzysztof A.

International Journal of Applied Mathematics and Computer Science, Tome 31 (2021) no. 4, pp. 713-727.

Résumé

The paper presents the concept of a control unit, i.e., a scenario player, for interactive training pilots in flight simulators. This scenario player is modelled as a hierarchy of finite state machines. Such an approach makes it possible to separate the details of an augmented reality display device which is used in training, from the core module of the system, responsible for contextual organization of the content. Therefore, the first contribution of this paper is the mathematical model of the scenario player as a universal formulation of the self-trained control unit for interactive learning systems, which is applicable in a variety of situations not limited solely to flight simulator related procedures. The second contribution is an experimental verification achieved by extensive simulations of the model, which proves that the proposed approach is capable to properly self-organize details of the context information by tracing preferences of the end users. For that latter purpose, the original algorithm is derived from statistical analysis, including Bayesian inference. The whole approach is illustrated by a real application of training the preflight procedure for the captain of the Boeing 737 aircraft in a flight simulator.

Détail

Keywords: Bayesian inference, deterministic Moore machine, flight simulator, finite state machine, scenario player
Mots-clés : wnioskowanie Bayesa, automat Moore'a, symulator lotu, automat skończony

Bibliographie

[1] [1] Adrego da Rocha, A.M. (1999). Synthesis and simulation of reprogrammable control units from hierarchical specification, PhD thesis, University of Aveiro.

[2] [2] Barkalov, A., Titarenko, L. and Mielcarek, K. (2020). Improving characteristics of LUT-based Mealy FSMs, International Journal of Applied Mathematics Computer Science 30(4): 745–759, DOI: 10.34768/amcs-2020-0055.

[3] [3] Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, Springer, New York, DOI: 10.1007/978-1-4757-4286-2.

[4] [4] Bernard, M. (2002). Examining a Metric for Predicting the Accessibility of Information within Hypertext Structures, PhD thesis, Wichita State University, Wichita.

[5] [5] Bower, M., Howe, C., McCredie, N., Robinson, A. and Grover, D. (2014). Augmented reality in education-Cases, places, and potentials, Educational Media International 51(1): 1–15.

[6] [6] Brown, L. (2017). The next generation classroom: Transforming aviation training with augmented reality, National Training Aircraft Symposium NTAS 2.0, Daytona Beach, USA, https://commons.erau.edu/ntas/2017/presentations/40/.

[7] [7] Camilleri,M. (2018). Travel Marketing, Tourism Economics and the Arline Product, Springer Nature, Cham, Chapter 12, pp. 191–204, DOI: 10.1007/978-3-319-49849-2-12.

[8] [8] Caudell, T. and Mizell, D.W. (1992). Augmented reality: An application of heads-up display technology to manual manufacturing processes, Proceedings of the 25th Hawaii International Conference on System Sciences, Kauai, USA, Vol. 2, pp. 659–669

[9] [9] Cerqueira, C. and Kirner, C. (2012). Developing educational applications with a nonprogramming augmented reality authoring tool, World Conference on Educational Multimedia, Hypermedia and Telecommunications (EDMEDIA), Denver, USA, pp. 2816–2825.

[10] [10] Cox, J. (2020). Training, Checking and Recency, Safety Resources of the EASA Community, Section: Air Operations, European Union Aviation Safety Agency, Cologne, https://www.easa.europa.eu/community/content/training-checking-and-recency.

[11] [11] Cyran, K.A., Moczulski, W., Myszor, D., Paszkuta, M., Ruranski, A., Kalisch, M., Cyran, J., Adamczyk, M. and Timofiejczuk, A. (2018a). Immersive human-machine interface for controlling the operation of the Telerescuer robot, International Journal of Advances in Computer Science and Its Applications 8(1): 322–326.

[12] [12] Cyran, K. A., Paszkuta, M., Myszor, D., Rohn, T., Drosik, T., Adamczyk, M. and Moczulski, W. (2018b). UAV-based anti-smog monitoring of the quality of exhausts from private chimneys in urban areas, Towards a Circular Economy: 7th International Symposium and Environmental Exhibition, Vienna, Austria.

[13] [13] Dhaliwal, A. (2019). Augmented Reality for the Aviation Industry: Getting Ready for Take Off, Atheer, Santa Clara, https://content.atheerair.com/hubfs/eBooks/Aviation-eBook.pdf?t=1541468052547.

[14] [14] Eschen, H., Kötter, T., Rodeck, R., Harnisch, M. and Schüppstuhl, T. (2018). Augmented and virtual reality for inspection and maintenance processes in the aviation industry, Procedia Manufacturing 19: 156–163, DOI: 10.1016/j.promfg.2018.01.022.

[15] [15] FAA (2020). Benefit-Cost Analysis, Federal Aviation Administration, Washington, https://www.faa.gov/regulations_policies/policy_guidance/benefit_cost/.

[16] [16] Feiner, S., Macintyre, B. and Seligmann, D. (1993). Knowledge-based augmented reality, Communications of the ACM 36(7): 53–62, DOI:10.1145/159544.159587.

[17] [17] Ferdania, D.F., Irawati, Garminia, H., Akhmaloka and Rachmansyah, K.A. (2021). Minimal state automata for detecting a β globin gene mutation, International Journal of Applied Mathematics and Computer Science 31(2): 337–351, DOI: 10.34768/amcs-2021-0023.

[18] [18] Gagné, R.M. (1965). The Conditions of Learning, Holt, Rinehart and Winston, New York.

[19] [19] Giantamidis, G., Tripakis, S. and Basagianis, S. (2019). Learning Moore machines from input-output traces, International Journal on Software Tools for Technology Transfer 23: 1–29, DOI: 10.1007/s10009-019-00544-0.

[20] [20] Goel, A. (2018). Augmented reality in aviation: Changing the face of the sector through training and simulated experience, eLearning Industry, https://elearningindustry.com/augmented-reality-in-aviation-changing-face-sector-training-simulated-experience.

[21] [21] Grzegorczyk, T., Śliwinski, R. and Kaczmarek, J. (2019). Attractiveness of augmented reality to consumers, Technology Analysis and Strategic Management 31(11): 1257–1269(13), DOI: 10.1080/09537325.2019.1603368.

[22] [22] Haritos, T. and Macchiarella, N.D. (2005). A mobile application of augmented reality for aerospace maintenance training, 24th Digital Avionics Systems Conference, Washington, USA, Vol. 1, pp. 5.B.3–5.1.

[23] [23] Hejase, M., Oguz, A., Kurt, A., Ozguner, U. and Redmill, K. (2016). A hierarchical hybrid state system based controller design approach for an autonomous UAS mission, 16th AIAA Aviation Technology, Integration, and Operations Conference, Washington, USA, DOI: 10.2514/6.2016-3294.

[24] [24] Kearns, S.K., Mavin, T.J. and Hodge, S. (2020). Engaging the Next Generation of Aviation Professionals, Routledge, London.

[25] [25] Khan, T., Johnston, K. and Ophoff, J. (2019). The impact of an augmented reality application on learning motivation of students, Advances in Human-Computer Interaction 2019: 1–14, Article ID: 7208494.

[26] [26] Ledermann, F. and Schmalstieg, D. (2005). April: A high-level framework for creating augmented reality presentations, IEEE Virtual Reality, Bonn, Germany, pp. 187–194, DOI: 10.1109/VR.2005.1492773.

[27] [27] Lee, K. (2012). Augmented reality in education and training, TechTrends 56(2): 13–22, DOI: 10.1007/s11528-012-0559-3.

[28] [28] Markets and Markets (2019). Augmented and virtual reality (AR VR) market in aviation global forecast to 2025 by technology (AR, VR), function (training, operations), component (hardware, software), application (on-board, off-board), product, vertical, and region, Market Research Report AS 4553, Markets and Markets, https://www.marketsandmarkets.com/pdfdownloadNew.asp?id=230427667.

[29] [29] Moir, I. and Seabridge, A. (2008). Aircraft Systems: Mechanical, Electrical, and Avionics Subsystems Integration, 3rd Edn, Wiley, Chichester, DOI: 10.1002/9780470770931.

[30] [30] Myers, P.L., Starr, A.W. and Mullins, K. (2018). Flight simulator fidelity, training transfer, and the role of instructors in optimizing learning, International Journal of Aviation, Aeronautics, and Aero-Space 5(1), Article 6.

[31] [31] Neumann, U. and Majoros, A. (1998). Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance, IEEE 1998 Virtual Reality Annual International Symposium, Atlanta, USA, pp. 4–11, DOI: 10.1109/VRAIS.1998.658416.

[32] [32] Olsson, M. (2016). Behavior Trees for Decision-Making in Autonomous Driving, Master thesis, KTH Royal Institute of Technology, Stockholm.

[33] [33] Page, R.L. (2004). Brief history of flight simulation, Semantic Scholar, Corpus ID: 211478463, DOI: 10.1.1.132.5428.

[34] [34] Parkinson, S.R., Hill, M.D., Sisson, N. and Viera, C. (1988). Effects of breadth, depth and number of responses on computer menu search performance, International Journal of Man-Machine Studies 28(6): 683–692.

[35] [35] Parsons, D. (2020). Full flight simulators incorporate VR for next generation of pilots, Aviation Today, https://www.aviationtoday.com/2019/08/01/training-brain-mind/.

[36] [36] Plummer, D., Karamouzis, F., Alvarez, G., Hill, J., Sallam, R., Daigler, J., Hunter, R., Litan, A., Resnick, M., Prentice, B., Natis, Y. and Gaughan, D. (2019). Gartner’s Top Strategic Predictions for 2020 and Beyond: Technology Changes the Human Condition, Gartner, Inc., Stamford, https://emtemp.gcom.cloud/ngw/globalassets/en/doc/documents/450595-top-strategic-predictions-for-2020-and-beyond.pdf.

[37] [37] Rabbath, C. (2013). A finite-state machine for collaborative airlift with a formation of unmanned air vehicles, Journal of Intelligent Robotic Systems 70: 233–253.

[38] [38] Rypulak, A. (2017). Using a virtual reality environment to teach practical skills of aviation personnel, EduAkcja, https://eduakcja.eu/files/pdf/143.pdf, (in Polish).

[39] [39] Safi, M., Chung, J. and Pradhan, P. (2019). Review of augmented reality in aerospace industry, Aircraft Engineering and Aerospace Technology 91(9): 1187–1194, DOI: 10.1108/AEAT-09-2018-0241.

[40] [40] Schaffernak, H., Moesl, B., Vorraber, W. and Koglbauer, I.V. (2020). Potential augmented reality application areas for pilot education: An exploratory study, Education Sciences 10(4): 86, DOI: 10.3390/educsci10040086.

[41] [41] Singh, V. and Singh, A. (2018). Learn-as-you-go: Feedback-driven result ranking and query refinement for interactive data exploration, Procedia Computer Science 125: 550–559.

[42] [42] Sklyarov, V., da Rocha, A.A. and de Ferrari, A.B. (1998). Synthesis of Reconfigurable Control Devices Based on Object-Oriented Specifications, Springer, Boston, DOI: 10.1007/978-1-4757-4419-4_7.

[43] [43] Spagnolo, C., Sumsurooah, S., Hill, C. and Bozhko, S. (2018). Finite state machine control for aircraft electrical distribution system, Journal of Engineering 2018(13): 506–511, DOI: 10.1049/joe.2018.0039.

[44] [44] Stańczyk, U., Cyran, K. and Pochopień, B. (2007). Theory of Logic Circuits. Vol. 2: Circuit Design and Analysis, Silesian University of Technology, Gliwice, (in Polish).

[45] [45] Turner, R., Hooda, S., Gersh, J. and Cancro, G. (2008). ExecSpec: Visually designing and operating a finite state machine-based spacecraft autonomy system Proceedings of the 9th International Symposium on Artificial Intelligence, Robotics and Automation for Space, Pasadena, USA, DOI: 10.1.1.520.1445.

[46] [46] Valenta, V. (2018). Effects of airline industry growth on pilot training, Magazine of Aviation Development 6(4): 52–56.

[47] [47] Velichko, M. (2020). How virtual and augmented reality are used in aviation training and other use cases, Jasoren, https://jasoren.com/how-virtual-and-augmented-reality-are-used-in-aviation-training-and-other-use-cases.

[48] [48] Wang, D., Yang, S., Wang, L. and Liu, W. (2016). Hardware-in-the-loop simulation for aircraft electric power system, 2016 International Conference on Electrical Systems for Aircraft, Railway, Ship Propulsion and Road Vehicles/International Transportation Electrification Conference (ESARS-ITEC), Toulouse, France, pp. 1–5.

[49] [49] Yon, Z. (2015). Modeling and simulation of controllers of aircraft power supply system based on finite state machine, Semantic Scholar: Engineering, Corpus ID: 114484490.

[50] [50] Young, J.D. (2015). Development of a Finite State Machine for a Small Unmanned Aircraft System Using Experimental Design, Student graduate works, AFIT-ENS-MS-15-M-146, Wright-Patterson Air Force Base, Dayton, https://scholar.afit.edu/etd/136.

[51] [51] Zajac,W., Andrzejewski, G., Krzywicki, K. and Królikowski, T. (2019). Finite state machine based modelling of discrete control algorithm in lad diagram language with use of new generation engineering software, Procedia Computer Science 159(2019): 2560–2569.

[52] [52] Zaphiris, P. (2000). Depth vs. breath in the arrangement of web links, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Diego, USA, pp. 453–456.

[53] [53] Zazula, A., Myszor, D., Antemijczuk, O. and Cyran, K. (2013). Flight simulators-From electromechanical analogue computers to modern laboratory of flying, Advances in Science and Technology Research Journal 7(17): 51–55, DOI: 10.5604/20804075.1036998.

[54] [54] Zhang, J., Sheng, Y., Hao, W., Wang, P.P., Tian, P., Miao, K. and Pickering, C.K. (2010). A context-aware framework supporting complex ubiquitous scenarios with augmented reality enabled, 5th International Conference on Pervasive Computing and Applications, Maribor, Slovenia, pp. 69–74, DOI: 10.1109/ICPCA.2010.5704077.

[55] [55] Zhu, J., Ong, S. and Nee, A. (2015). A context-aware augmented reality assisted maintenance system, International Journal of Computer Integrated Manufacturing 28(2): 213–225, DOI: 10.1080/0951192X.2013.874589.