Room for Uncertainty in Remaining Useful Life Estimation for Turbofan Jet Engines

I. A. Serenko ; Y. V. Dorn ; S. R. Singh ; A. V. Kornaev

Russian journal of nonlinear dynamics, Tome 20 (2024) no. 5, pp. 933-944.

Résumé

This work addresses uncertainty quantification in machine learning, treating it as a hidden parameter of the model that estimates variance in training data, thereby enhancing the interpretability of predictive models. By predicting both the target value and the certainty of the prediction, combined with deep ensembling to study model uncertainty, the proposed method aims to increase model accuracy. The approach was applied to the well-known problem of Remaining Useful Life (RUL) estimation for turbofan jet engines using NASA’s dataset. The method demonstrated competitive results compared to other commonly used tabular data processing methods, including k-nearest neighbors, support vector machines, decision trees, and their ensembles. The proposed method is based on advanced techniques that leverage uncertainty quantification to improve the reliability and accuracy of RUL predictions.

Détail

Keywords: machine learning, analysis of sequences, uncertainty quantification, recurrent neural networks, rotor machines, remaining useful life

Bibliographie

[1] Prince, S. J. D., Understanding Deep Learning, MIT Press, Cambridge, Mass., 2023, 544 pp.

[2] Goodfellow, I., Bengio, Y., and Courville, A., Deep Learning, MIT Press, Cambridge, Mass., 2016, 800 pp. | MR | Zbl

[3] Recurrent Neural Networks: Design and Applications, eds. L. R. Medsker, L. C. Jain, CRC, Boca Raton, Fla., 2000, 416 pp. | MR

[4] Ahmed, S., Nielsen, I. E., Tripathi, A., Siddiqui, S., Ramachandran, R. P., and Rasool, G., “Transformers in Time-Series Analysis: A Tutorial”, Circuits Syst. Signal Process., 42:12 (2023), 7433–7466 | DOI

[5] Mathew, V., Toby, T., Singh, V., Rao, B. M., and Kumar, M. G., “Prediction of Remaining Useful Lifetime (RUL) of Turbofan Engine Using Machine Learning”, Proc. of the IEEE Internat. Conf. on Circuits and Systems (ICCS, Thiruvananthapuram, India, Dec 2017), 306–311

[6] Kendall, A. and Gal, Y., What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, Proc. of the 31st Conf. on Neural Information Processing Systems (NIPS, Long Beach, Calif., USA, Dec 2017), 5580–5590

[7] Bentéjac, C., Csörgő, A., and Martínez-Muñoz, G., “A Comparative Analysis of Gradient Boosting Algorithms”, Artif. Intell. Rev., 54:3 (2021), 1937–1967 | DOI

[8] Peng, C., Chen, Y., Chen, Q., Tang, Z., Li, L., and Gui, W., “A Remaining Useful Life Prognosis of Turbofan Engine Using Temporal and Spatial Feature Fusion”, Sensors, 21:2 (2021), Art. 418, 20 pp.

[9] Mitici, M., de Pater, I., Barros, A., and Zeng, Z., “Dynamic Predictive Maintenance for Multiple Components Using Data-Driven Probabilistic RUL Prognostics: The Case of Turbofan Engines”, Reliab. Eng. Syst. Safety, 234 (2023), Art. 109199, 14 pp. | DOI

[10] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I., “Attention Is All You Need”, Proc. of the 31st Conf. on Neural Information Processing Systems (NIPS, Long Beach, Calif., USA, 2017), 11 pp.

[11] Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., and Wang, Y., “Transformer in Transformer”, Proc. of the 35th Conf. on Neural Information Processing Systems (NeurIPS'2021), 15908–15919

[12] Che, C., Zhang, P., Zhu, M., Qu, Y., and Jin, B., “Constrained Transformer Network for ECG Signal Processing and Arrhythmia Classification”, BMC Med. Inform. Decis. Mak., 21:1 (2021), Art. 184, 13 pp. | DOI

[13] Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S.-Y., and Sainath, T., “Deep Learning for Audio Signal Processing”, IEEE J. Sel. Topics Signal Process., 13:2 (2019), 206–219 | DOI

[14] Stebakov, I., Kornaev, A., Popov, S., and Savin, L., “Fault Diagnosis Systems for Rotating Machines Operating with Fluid-Film Bearings”, Proc. Inst. Mech. Eng. J: J. Eng. Tribol., 236:10 (2022), 1917–1931 | DOI

[15] Zschech, P., Bernien, J., and Heinrich, K., “Towards a Taxonomic Benchmarking Framework for Predictive Maintenance: The Case of NASA's Turbofan Degradation”, Proc. of the 40th Internat. Conf. on Information Systems (ICIS, Munich, Germany, Dec 2019), 15 pp. | Zbl

[16] Thakkar, U. and Chaoui, H., “Remaining Useful Life Prediction of an Aircraft Turbofan Engine Using Deep Layer Recurrent Neural Networks”, Actuators, 11:3 (2022), Art. 67, 17 pp. | DOI

[17] Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., Shahzad, M., Yang, W., Bamler, R., and Zhu, X. X., “A Survey of Uncertainty in Deep Neural Networks”, Artif. Intell. Rev., 56:suppl. 1 (2023), 1513–1589 | DOI

[18] van Amersfoort, J., Smith, L., Teh, Y. W., and Gal, Y., “Uncertainty Estimation Using a Single Deep Deterministic Neural Network”, Proc. of the 37th Internat. Conf. on Machine Learning (ICML'2020), 9690–9700

[19] Gal, Y. and Ghahramani, Z., “Dropout As a Bayesian Approximation: Representing Model Uncertainty in Deep Learning”, Proc. of the 33rd Internat. Conf. on Machine Learning (ICML, New York City, NY, USA, Jun 2016), 1050–1059

[20] Ashukha, A., Lyzhov, A., Molchanov, D., and Vetrov, D., Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning, , 2020, 30 pp. arXiv:2002.06470 [stat.ML]

[21] Ganaie, M. A., Hu, M., Malik, A. K., Tanveer, M., and Suganthan, P. N., “Ensemble Deep Learning: A Review”, Eng. Appl. Artif. Intell., 115 (2022), Art. 105151, 18 pp. | DOI

[22] Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., Makarenkov, V., and Nahavandi, S., “A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges”, Inf. Fusion, 76 (2021), 243–297 | DOI

[23] Le, M. T., Diehl, F., Brunner, Th., and Knoll, A., “Uncertainty Estimation for Deep Neural Object Detectors in Safety-Critical Applications”, Proc. of the 21st Internat. Conf. on Intelligent Transportation Systems (ITSC, Maui, H.I., USA, Nov 2018), 3873–3878

[24] Kornaev, A., Kornaeva, E., Ivanov, O., Pershin, I., and Alukaev, D., Awareness of Uncertainty in Classification Using a Multivariate Model and Multi-Views, , 2024, 11 pp. arXiv:2404.10314 [cs.CV]

[25] Meronen, L., Trapp, M., and Solin, A., “Periodic Activation Functions Induce Stationarity”, Proc. of the 35th Conf. on Neural Information Processing Systems (NeurIPS, Dec 2021), 1673–1685

[26] Kirstein, C., Predictive Maintenance NASA Turbofan Regression, , 2022 www.kaggle.com

[27] Prince, S. J. D., Computer Vision: Models, Learning, and Inference, Cambridge Univ. Press, Cambridge, 2012, 598 pp. | Zbl

[28] Bishop, Ch. M., Pattern Recognition and Machine Learning, Information Science and Statistics, Springer, New York, 2006, xx, 778 pp. | MR | Zbl