Integration of missing data imputation tools for time series in real-time mode into a relational DBMS

A. A. Yurtin

Vestnik Ûžno-Uralʹskogo gosudarstvennogo universiteta. Seriâ Vyčislitelʹnaâ matematika i informatika, Tome 14 (2025) no. 1, pp. 30-46

Résumé

The article addresses the problem of integrating time series imputation into relational database management systems (RDBMS). A method called ImputeDB is proposed, which enables the real-time integration of neural network-based imputation models into the PostgreSQL RDBMS. The imputation of missing values is carried out through triggers (stored functions automatically executed by the RDBMS kernel when new data is inserted). When a trigger is activated, missing values are replaced by synthetic ones generated by a neural network model. Using the proposed method, a database application programmer can integrate the process of imputing missing values into the standard time series processing pipeline within the PostgreSQL RDBMS, without relying on external services. The proposed approach includes a set of components implemented as user-defined functions (UDFs) in Python and PL/Python: Trigger Constructor, Model Manager, Model Storage, and Imputer. The Trigger Constructor is used to create triggers that automatically perform imputation of missing values in inserted data. The Model Manager is responsible for training neural network models, while the Model Storage is used to save these models in a file-based repository. The Imputer, in turn, synthesizes missing values using the trained models. Experiments were conducted to evaluate the performance of the ImputeDB method. The experiments measured the processing time of data insertion with automatic gap imputation as a function of the time series dimensionality. Experiments were performed under two scenarios: single and multiple insertions. Neural network-based imputation models with various architectures, including recurrent neural networks, autoencoders, and transformers, were employed. The experimental results demonstrated that under conditions of increasing time series dimensionality and rising overhead from network requests and data transfer, ImputeDB exhibits superior performance. Specifically, the system achieved an efficiency gain of 22.5% compared to another approach, while maintaining the accuracy of the employed imputation methods.

Détail

Keywords: time series, DBMS, PostgreSQL, missing value imputation, neural networks.