Using neural networks with data quantization for time series analysis in LHC superconducting magnets

Wielgosz, Maciej ; Skoczeń, Andrzej

International Journal of Applied Mathematics and Computer Science, Tome 29 (2019) no. 3, pp. 503-515.

Résumé

The aim of this paper is to present a model based on the recurrent neural network (RNN) architecture, the long short-term memory (LSTM) in particular, for modeling the work parameters of Large Hadron Collider (LHC) superconducting magnets. High-resolution data available in the post mortem database were used to train a set of models and compare their performance for various hyper-parameters such as input data quantization and the number of cells. A novel approach to signal level quantization allowed reducing the size of the model, simplifying the tuning of the magnet monitoring system and making the process scalable. The paper shows that an RNN such as the LSTM or a gated recurrent unit (GRU) can be used for modeling high-resolution signals with the accuracy of over 0.95 and a small number of parameters, ranging from 800 to 1200. This makes the solution suitable for hardware implementation, which is essential in the case of monitoring the performance critical and high-speed signal of LHC superconducting magnets.

Détail

Keywords: Large Hadron Collider, LSTM architecture, signal modelling
Mots-clés : Wielki Zderzacz Hadronów, architektura LSTM, modelowanie sygnału

Bibliographie

[1] Apollinari, G., Béjar, A.I., Brüning, O., Fessia, P., Lamont, M., Rossi, L. and Tavian, L. (2017). High-Luminosity Large Hadron Collider (HL-LHC): Technical Design Report V.0.1, Yellow Reports: Monographs, CERN, Geneva, http://cds.cern.ch/record/2284929.

[2] Bordry, F., Denz, R., Mess, K.-H., Puccio, B., Rodriguez-Mateos, F. and Schmidt, R. (2001). Machine protection for the LHC: Architecture of the beam and powering interlock system, LHC Project Report 521, CERN, Geneva, https://cds.cern.ch/record/531820.

[3] Borland, M. (1998). A brief introduction to the SDDS toolkit, Technical report, Argonne National Laboratory, Lemont, IL, https://ops.aps.anl.gov/SDDSIntroTalk/slides.html.

[4] Brüning, O. and Collier, P. (2007). Building a behemoth, Nature 448(7151): 285–289.

[5] Chang, A.X.M., Martini, B. and Culurciello, E. (2015). Recurrent neural networks hardware implementation on FPGA, arXiv 1511.05552 [cs.NE].

[6] Chen, X., Liu, X., Wang, Y., Gales, M.J.F. and Woodland, P.C. (2016). Efficient training and evaluation of recurrent neural network language models for automatic speech recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing 24(11): 2146–2157.

[7] Chollet, F. (2015). Keras, GitHub repository, https://github.com/fchollet/keras.

[8] Chong, Y.S. and Tay, Y.H. (2017). Abnormal event detection in videos using spatiotemporal autoencoder, in F. Cong et al. (Eds), Advances in Neural Networks, ISNN 2017, Springer International Publishing, Cham, pp. 189–196.

[9] Chung, J., Gülçehre, Ç., Cho, K. and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv 1412.3555[cs.NE].

[10] Chung, J., Gülçehre, Ç., Cho, K. and Bengio, Y. (2015). Gated feedback recurrent neural networks, arXiv 1502.02367[cs.NE].

[11] Ciapala, E., Rodríguez-Mateos, F., Schmidt, R. and Wenninger, J. (2002). The LHC post-mortem system, Technical Report LHC-PROJECT-NOTE-303, CERN, Geneva, http://cds.cern.ch/record/691828.

[12] Evans, L. and Bryant, P. (2008). LHC Machine, Journal of Instrumentation 3(08): S08001.

[13] FCC Study (2019). Future Circular Collide, Conceptual Design Report, Yellow Reports: Monographs, CERN, Geneva, https://fcc-cdr.web.cern.ch/, (in press).

[14] Graves, A. (2012). Neural Networks, Springer, Berlin/Heidelberg.

[15] Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R. and Schmidhuber, J. (2015). LSTM: A search space odyssey, ArXiv 1503.04069 [cs.NE].

[16] Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., Yang, H. and Dally, W. B.J. (2017). ESE: Efficient speech recognition engine with sparse LSTM on FPGA, Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA’17), Monterey, CA, USA, pp. 75–84.

[17] Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks, in F. Pereira et al. (Eds), Advances in Neural Information Processing Systems 25, Curran Associates, Inc., Red Hook, NY, pp. 1097–1105.

[18] Lauckner, R.J. (2001). What data is needed to understand failures during LHC operation, CERN-SL-2001-003, CERN, Geneva, pp. 278–283, https://cds.cern.ch/record/567214.

[19] LeCun, Y. (2015). Deep learning of convolutional networks, 2015 IEEE Hot Chips 27 Symposium (HCS), Cupertino, CA, USA, pp. 1–95.

[20] Lee, M., Hwang, K., Park, J., Choi, S., Shin, S. and Sung, W. (2016). FPGA-based low-power speech recognition with recurrent neural networks, ArXiv 1610.00552 [cs.CL].

[21] Malhotra, P., Vig, L., Shroff, G. and Agarwal, P. (2015). Long short term memory networks for anomaly detection in time series, 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015, Bruges, Belgium, pp. 89–94.

[22] Marchi, E., Vesperini, F., Eyben, F., Squartini, S. and Schuller, B. (2015a). A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional LSTM neural networks, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, pp. 1996–2000.

[23] Marchi, E., Vesperini, F., Weninger, F., Eyben, F., Squartini, S. and Schuller, B. (2015b). Non-linear prediction with LSTM recurrent neural networks for acoustic novelty detection, 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, pp. 1–7.

[24] Morton, J., Wheeler, T.A. and Kochenderfer, M.J. (2016). Analysis of recurrent neural networks for probabilistic modelling of driver behaviour, IEEE Transactions on Intelligent Transportation Systems PP(99): 1–10.

[25] Pouladi, F., Salehinejad, H. and Gilani, A.M. (2015). Recurrent neural networks for sequential phenotype prediction in genomics, 2015 International Conference on Developments of E-Systems Engineering (DeSE), Dubai, UAE, pp. 225–230.

[26] Schmidt, R. (2016). Machine protection and interlock systems for circular machines—Example for LHC, Yellow Report CERN-2016-002, CERN, Geneva, pp. 319–341.

[27] Skoczeń, A. and Skała, A. (2009). Commissioning of quench protection instruments in the LHC superconducting circuits, Electrical Review 85(7): 73–77, (in Polish).

[28] Steckert, J. and Skoczeń, A. (2017). Design of FPGA-based radiation tolerant quench detectors for LHC, Journal of Instrumentation 12(04): T04005–T04005.

[29] Strecht, P., Cruz, L., Soares, C., Mendes-Moreira, J. and Abreu, R. (2015). A comparative study of regression and classification algorithms for modelling students’ academic performance, Proceedings of the 8th International Conference on Educational Data Mining, EDM 2015, Madrid, Spain, pp. 392–395.

[30] Wenninger, J. (2016). Machine protection and operation for LHC, Yellow Report CERN-2016-002, CERN, Geneva, pp. 377–401.

[31] Wielgosz, M., Mertik, M., Skoczeń, A. and Matteis, E.D. (2018a). The model of an anomaly detector for HiLumi LHC magnets based on recurrent neural networks and adaptive quantization, Engineering Applications of Artificial Intelligence 74: 166–185.

[32] Wielgosz, M. and Skoczeń, A. (2018). Recurrent neural networks with grid data quantization for modeling LHC superconducting magnets behavior, Contemporary Computational Science, AGH-UST Press, Cracow, p. 240.

[33] Wielgosz, M. and Skoczeń, A. (2020). Recurrent neural networks with grid data quantization for modeling LHC superconducting magnets behavior, in P. Kulczycki et al. (Eds), Information Technology, Systems Research and Computational Physics, Advances in Intelligent Systems and Computing, Vol. 945, Springer, Cham, (in press).

[34] Wielgosz, M., Skoczeń, A. and De Matteis, E. (2018b). Protection of superconducting industrial machinery using RNN-based anomaly detection for implementation in smart sensor, Sensors 18(11), Article ID: 3933.

[35] Wielgosz, M., Skoczeń, A. and Mertik, M. (2017). Using LSTM recurrent neural networks for detecting anomalous behavior of LHC superconducting magnets, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 867: 40–50.

[36] Wielgosz, M., Skoczeń, A. and Wiatr, K. (2018c). Looking for a correct solution of anomaly detection in the LHC machine protection system, International Conference on Signals and Electronic Systems (ICSES), Cracow, Poland, pp. 257–262.