Performance analysis of least squares algorithm for multivariable stochastic systems

Wang, Ziming ; Xing, Yiming ; Zhu, Xinghua

Kybernetika, Tome 59 (2023) no. 1, pp. 28-44

Abstract (VO)

In this paper, we consider the parameter estimation problem for the multivariable system. A recursive least squares algorithm is studied by minimizing the accumulative prediction error. By employing the stochastic Lyapunov function and the martingale estimate methods, we provide the weakest possible data conditions for convergence analysis. The upper bound of accumulative regret is also provided. Various simulation examples are given, and the results demonstrate that the convergence rate of the algorithm depends on the parameter dimension and output dimension.

Détail

DOI : 10.14736/kyb-2023-1-0028

Classification : 93A10, 93E12, 93E24
Keywords: least squares; martingale theory; non-persistent excitation

Bibliographie

[1] Chen, H.-F., Guo, L.: Strong consistency of recursive identification by no use of persistent excitation condition. Acta Math. Appl. Sinica 2 (1985), 2, 133-145. | DOI

[2] Chen, H.-F., Guo, L.: Continuous-time stochastic adaptive tracking—robustness and asymptotic properties. SIAM J. Control Optim. 28 (1990), 3, 513-527. | DOI | MR

[3] Chen, H.-F., Guo, L.: Identification and Stochastic Adaptive Control. Volume 5. Springer Science Business Media, 1991. | MR

[4] Doyle, J. C., Francis, B. A., Tannenbaum, A. R.: Feedback Control Theory. Courier Corporation, 2013. | MR

[5] Durrett, R.: Probability: Theory and Examples. Vol. 49. Cambridge University Press, 2019. | MR

[6] Gan, D., Liu, Z. X.: On the stability of Kalman filter with random coefficients. IFAC-PapersOnLine 53 (2020), 2, 2397-2402. | DOI

[7] Guo, L.: Stability of recursive stochastic tracking algorithms. SIAM J. Control Optim. 32 (1994), 5, 1195-1225, 1994. | DOI | MR

[8] Guo, L.: Convergence and logarithm laws of self-tuning regulators. Automatica 31 (1995), 3, 435-450. | DOI | MR

[9] Guo, L.: Time-Varying Stochastic Systems, Stability and Adaptive Theory. Second Edition. Science Press, Beijing 2020.

[10] Guo, L., Ljung, L., Priouret, P.: Performance analysis of the forgetting factor rls algorithm. Int. J. Adaptive Control Signal Process. 7 (1993), 6, 525-537. | DOI | MR

[11] Haykin, S.: Radar signal processing. IEEE Signal Process. Magazine 2 (1993), 2, 2-18.

[12] Lai, T. L.: Asymptotically efficient adaptive control in stochastic regression models. Advances Appl. Math. 7 (1986), 1, 23-45. | DOI | MR

[13] Lai, T. L., Wei, Ch. Z.: Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. Ann. Statist. 10 (1982), 1, 154-166. | DOI | MR

[14] Li, J., Ding, F., Yang, G.: Maximum likelihood least squares identification method for input nonlinear finite impulse response moving average systems. Math. Comput. Modell. 55 (2012), 3-4, :442-450. | MR

[15] Li, J., Stoica, P.: MIMO Radar Signal Processing. John Wiley and Sons, 2008.

[16] Liu, Y., Ding, F.: Convergence properties of the least squares estimation algorithm for multivariable systems. Applied Math. Modell. 37 (2013), 1-2, 476-483. | DOI | MR

[17] Marelli, D., Fu, M.: A continuous-time linear system identification method for slowly sampled data. IEEE Trans. Signal Process. 58 (2010), 5, 2521-2533. | DOI | MR

[18] Moore, J. B.: On strong consistency of least squares identification algorithms. Automatica 14 (1978), 5, 505-509. | DOI

[19] Niedzwiecki, M., Guo, L.: Nonasymptotic results for finite-memory wls filters. In: Proc. 28th IEEE Conference on Decision and Control, Vol. 2, 1989, pp. 1785-1790. | MR

[20] Richards, F. S. G.: A method of maximum-likelihood estimation. J. Roy. Statist. Soc,: Series B (Methodological) 23 (1961), 2, 469-475. | DOI | MR

[21] Sen, A., Sinha, N. K.: On-line estimation of the parameters of a multivariable system using matrix pseudo-inverse. Int. J. Systems Sci. 7 (1976), 4, 461-471. | DOI | MR

[22] Subudhi, B., Jena, D.: Nonlinear system identification of a twin rotor mimo system. In: IEEE Region 10 Conference 2009, pp. 1-6.

[23] Vaezi, M., Izadian, A.: Piecewise affine system identification of a hydraulic wind power transfer system. IEEE Trans. Control Systems Technol. 23 (2015), 6, :2077-2086. | DOI

[24] Wang, W., Ding, F., Dai, J.: Maximum likelihood least squares identification for systems with autoregressive moving average noise. Appl. Math. Modell. 36 (2012), 5, 1842-1853. | DOI | MR

[25] Vogels, T. P., Rajan, K., Abbott, L. F.: Neural network dynamics. Ann. Rev. Neurosci. 28 (2005), 357-376. | DOI | MR

[26] Widrow, B., Stearns, S. D.: Adaptive Signal Processing. Prentice-Hall Englewood Cliffs, NJ 1985.

[27] Zhang, Y.: Unbiased identification of a class of multi-input single-output systems with correlated disturbances using bias compensation methods. Math. Computer Modell. 53 (2011), 9-10, 1810-1819. | DOI | MR

[28] Zhang, Y., Cui, G.: Bias compensation methods for stochastic systems with colored noise. Appl. Math- Modell. 35 (2011), 4, 1709-1716. | DOI | MR