Intermittent estimation for finite alphabet finitarily Markovian processes with exponential tails

Morvai, Gusztáv ; Weiss, Benjamin

Kybernetika, Tome 57 (2021) no. 4, pp. 628-646

Abstract (VO)

We give some estimation schemes for the conditional distribution and conditional expectation of the the next output following the observation of the first $n$ outputs of a stationary process where the random variables may take finitely many possible values. Our schemes are universal in the class of finitarily Markovian processes that have an exponential rate for the tail of the look back time distribution. In addition explicit rates are given. A necessary restriction is that the scheme proposes an estimate only at certain stopping times, but these have density one so that one rarely fails to give an estimate.

Détail

DOI : 10.14736/kyb-2021-4-0628

Classification : 60G10, 60G25, 62G05
Keywords: nonparametric estimation; stationary processes

Bibliographie

[1] Algoet, P.: The strong law of large numbers for sequential decisions under uncertainty. IEEE Trans. Inform. Theory 40 (1994), 609-633. | DOI

[2] Algoet, P.: Universal schemes for learning the best nonlinear predictor given the infinite past and side information. IEEE Trans. Inform. Theory 45 (1999), 1165-1185. | DOI | Zbl

[3] Bailey, D. H.: Sequential Schemes for Classifying and Predicting Ergodic Processes. Ph.D. Thesis, Stanford University, 1976.

[4] Csiszár, I., Talata, Zs.: Context tree estimation for not necessarily finite memory processes via BIC and MDL. IEEE Trans. Inform. Theory 52 (2006), 3, 1007-1016. | DOI

[5] Györfi, L., Morvai, G., Yakowitz, S.: Limits to consistent on-line forecasting for ergodic time series. IEEE Trans. Inform. Theory 44 (1998), 886-892. | DOI | MR | Zbl

[6] Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 (1963), 13-30. | DOI

[7] Kalikow, S., Katznelson, Y., Weiss, B.: Finitarily deterministic generators for zero entropy systems. Israel J. Math. 79 (1992), 33-45. | DOI

[8] Maker, Ph. T.: The ergodic theorem for a sequence of functions. Duke Math. J. 6 (1940), 27-30.

[9] Morvai, G.: Guessing the output of a stationary binary time series. In: Foundations of Statistical Inference (Y. Haitovsky, H. R.Lerche, and Y. Ritov, eds.), Physika-Verlag, pp. 207-215, 2003.

[10] Morvai, G., Yakowitz, S., Algoet, P.: Weakly convergent nonparametric forecasting of stationary time series. IEEE Trans. Inform. Theory 43 (1997), 483-498. | DOI

[11] Morvai, G., Weiss, B.: Forecasting for stationary binary time series. Acta Appl. Math. 79 (2003), 25-34. | DOI

[12] Morvai, G., Weiss, B.: Intermittent estimation of stationary time series. Test 13 (2004), 525-542. | DOI

[13] Morvai, G., Weiss, B.: Inferring the conditional mean. Theory Stochast. Process. 11 (2005), 1-2, 112-120. | Zbl

[14] Morvai, G., Weiss, B.: Prediction for discrete time series. Probab. Theory Related Fields 132 (2005), 1-12. | DOI

[15] Morvai, G., Weiss, B.: Limitations on intermittent forecasting. Statist. Probab. Lett. 72 (2005), 285-290. | DOI

[16] Morvai, G., Weiss, B.: On classifying processes. Bernoulli 11 (2005), 523-532. | DOI

[17] Morvai, G., Weiss, B.: Order estimation of Markov chains. IEEE Trans. Inform. Theory 51 (2005), 1496-1497. | DOI

[18] Morvai, G., Weiss, B.: Forward estimation for ergodic time series. Ann. I. H. Poincaré Probab. Statist. 41 (2005), 859-870. | DOI

[19] Morvai, G., Weiss, B.: On estimating the memory for finitarily Markovian processes. Ann. I. H. Poincaré PR 43 (2007), 15-30. | DOI

[20] Morvai, G., Weiss, B.: On sequential estimation and prediction for discrete time series. Stoch. Dyn. 7 (2007), 4, 417-437. | DOI | Zbl

[21] Morvai, G., Weiss, B.: Estimating the lengths of memory words. IEEE Trans. Inform. Theory 54 (2008), 8, 3804-3807. | DOI | Zbl

[22] Morvai, G., Weiss, B.: On universal estimates for binary renewal processes. Annals Appl. Probab. 18 (2008), 5, 1970-1992. | DOI | Zbl

[23] Morvai, G., Weiss, B.: Estimating the residual waiting time for binary stationary time series. Proc. ITW2009, Volos 2009, pp. 67-70.

[24] Morvai, G., Weiss, B.: A note on prediction for discrete time series. Kybernetika 48 (2012), 4, 809-823.

[25] Morvai, G., Weiss, B.: Universal tests for memory words. IEEE Trans. Inform. Theory 59 (2013), 6873-6879. | DOI

[26] Morvai, G., Weiss, B.: Inferring the residual waiting time for binary stationary time series. Kybernetika 50 (2014), 869-882. | DOI | Zbl

[27] Morvai, G., Weiss, B.: A versatile scheme for predicting renewal times. Kybernetika 52 (2016), 348-358. | DOI

[28] Morvai, G., Weiss, B.: Universal rates for estimating the residual waiting time in an intermittent way. Kybernetika 56, (2020), 4, 601-616. | DOI

[29] Morvai, G., Weiss, B.: On universal algorithms for classifying and predicting stationary processes. Probab. Surveys 18 (2021), 77-131. | DOI

[30] Morvai, G., Weiss, B.: Consistency, integrability and asymptotic normality for some intermittent estimators. ALEA, Lat. Am. J. Probab. Math. Stat. 18 (2021), 1643-1667. | DOI

[31] Ryabko, B. Ya.: Prediction of random sequences and universal coding. Problems Inform. Trans. 24 (1988), 87-96. | DOI | Zbl

[32] Ryabko, D.: Asymptotic Nonparametric Statistical Analysis of Stationary Time Series. Springer, Cham 2019.

[33] Shields, P. C.: The Ergodic Theory of Discrete Sample Paths. In: Graduate Studies in Mathematics. American Mathematical Society 13, Providence 1996. | Zbl

[34] Suzuki, J.: Universal prediction and universal coding. Systems Computers Japan 34 (2003), 6, 1-11. | DOI

[35] Takahashi, H.: Computational limits to nonparametric estimation for ergodic processes. IEEE Trans. Inform. Theory 57 (2011), 6995-6999. | DOI