Critical dimension in the semiparametric Bernstein--von~Mises theorem

Maxim E. Panov ; Vladimir G. Spokoiny

Trudy Matematicheskogo Instituta imeni V.A. Steklova, Stochastic calculus, martingales, and their applications, Tome 287 (2014), pp. 242-266.

Résumé

The classical parametric and semiparametric Bernstein–von Mises (BvM) results are reconsidered in a nonclassical setup allowing finite samples and model misspecification. In the parametric case and in the case of a finite-dimensional nuisance parameter, we establish an upper bound on the error of Gaussian approximation of the posterior distribution of the target parameter; the bound depends explicitly on the dimension of the full and target parameters and on the sample size. This helps to identify the so-called critical dimension $p_n$ of the full parameter for which the BvM result is applicable. In the important special i.i.d. case, we show that the condition "$p_n^3/n$ is small" is sufficient for the BvM result to be valid under general assumptions on the model. We also provide an example of a model with the phase transition effect: the statement of the BvM theorem fails when the dimension $p_n$ approaches $n^{1/3}$.

Bibliographie

[1] Barron A., Schervish M.J., Wasserman L., “The consistency of posterior distributions in nonparametric problems”, Ann. Stat., 27:2 (1996), 536–561 | MR

[2] Bickel P.J., Kleijn B.J.K., “The semiparametric Bernstein–von Mises theorem”, Ann. Stat., 40:1 (2012), 206–237 | DOI | MR | Zbl

[3] Bochkina N., Green P.J., “The Bernstein–von Mises theorem and nonregular models”, Ann. Stat., 42:5 (2014), 1850–1878 | DOI | MR | Zbl

[4] Bontemps D., “Bernstein–von Mises theorems for Gaussian regression with increasing number of regressors”, Ann. Stat., 39:5 (2011), 2557–2584 | DOI | MR | Zbl

[5] Boucheron S., Gassiat E., “A Bernstein–von Mises theorem for discrete probability distributions”, Electron. J. Stat., 3 (2009), 114–148 | DOI | MR | Zbl

[6] Boucheron S., Massart P., “A high-dimensional Wilks phenomenon”, Probab. Theory Relat. Fields., 150:3–4 (2011), 405–433 | DOI | MR | Zbl

[7] Castillo I., “A semiparametric Bernstein–von Mises theorem for Gaussian process priors”, Probab. Theory Relat. Fields, 152:1–2 (2012), 53–99 | DOI | MR | Zbl

[8] Castillo I., Nickl R., “Nonparametric Bernstein–von Mises theorems in Gaussian white noise”, Ann. Stat., 41:4 (2013), 1999–2028 | DOI | MR | Zbl

[9] Castillo I., Rousseau J., A general Bernstein–von Mises theorem in semiparametric models, E-print, 2013, arXiv: 1305.4482v1 [math.ST]

[10] Cheng G., Kosorok M.R., “General frequentist properties of the posterior profile distribution”, Ann. Stat., 36:4 (2008), 1819–1853 | DOI | MR | Zbl

[11] Chernozhukov V., Hong H., “An MCMC approach to classical estimation”, J. Econom., 115:2 (2003), 293–346 | DOI | MR | Zbl

[12] Cox D.D., “An analysis of Bayesian inference for nonparametric regression”, Ann. Stat., 21:2 (1993), 903–923 | DOI | MR | Zbl

[13] Freedman D., “On the Bernstein–von Mises theorem with infinite-dimensional parameters”, Ann. Stat., 27:4 (1999), 1119–1140 | MR | Zbl

[14] Ghosal S., “Asymptotic normality of posterior distributions in high-dimensional linear models”, Bernoulli, 5:2 (1999), 315–331 | DOI | MR | Zbl

[15] Ghosal S., “Asymptotic normality of posterior distributions for exponential families when the number of parameters tends to infinity”, J. Multivariate Anal., 74:1 (2000), 49–68 | DOI | MR | Zbl

[16] Johnstone I.M., “High dimensional Bernstein–von Mises: simple examples”, Borrowing strength: Theory powering applications—a Festschrift for Lawrence D. Brown, Inst. Math. Stat. Collect., 6, Inst. Math. Stat., Beachwood, OH, 2010, 87–98 | MR

[17] Kim Y., “The Bernstein–von Mises theorem for the proportional hazard model”, Ann. Stat., 34:4 (2006), 1678–1700 | DOI | MR | Zbl

[18] Kim Y., Lee J., “A Bernstein–von Mises theorem in the nonparametric right-censoring model”, Ann. Stat., 32:4 (2004), 1492–1512 | DOI | MR | Zbl

[19] Kleijn B.J.K., van der Vaart A.W., “Misspecification in infinite-dimensional Bayesian statistics”, Ann. Stat., 34:2 (2006), 837–877 | DOI | MR | Zbl

[20] Kleijn B.J.K., van der Vaart A.W., “The Bernstein–von Mises theorem under misspecification”, Electron. J. Stat., 6 (2012), 354–381 | DOI | MR | Zbl

[21] Le Cam L., Yang G.L., Asymptotics in statistics: Some basic concepts, Springer Ser. Stat., Springer, New York, 1990 | DOI | MR

[22] Leahu H., “On the Bernstein–von Mises phenomenon in the Gaussian white noise model”, Electron. J. Stat., 5 (2011), 373–404 | DOI | MR | Zbl

[23] Polzehl J., Spokoiny V., “Propagation–separation approach for local likelihood estimation”, Probab. Theory Relat. Fields, 135:3 (2006), 335–362 | DOI | MR | Zbl

[24] Rivoirard V., Rousseau J., “Bernstein–von Mises theorem for linear functionals of the density”, Ann. Stat., 40:3 (2012), 1489–1523 | DOI | MR | Zbl

[25] Schwartz L., “On Bayes procedures”, Z. Wahrscheinlichkeitstheor. Verw. Geb., 4:1 (1965), 10–26 | DOI | MR | Zbl

[26] Shen X., “Asymptotic normality of semiparametric and nonparametric posterior distributions”, J. Amer. Stat. Assoc., 97:457 (2002), 222–235 | DOI | MR | Zbl

[27] Spokoiny V., “Parametric estimation. Finite sample theory”, Ann. Stat., 40:6 (2012), 2877–2909, arXiv: 1111.3029 [math.ST] | DOI | MR | Zbl

[28] Spokoiny V., Zhilova M., “Sharp deviation bounds for quadratic forms”, Math. Methods Stat., 22:2 (2013), 100–113, arXiv: 1302.1699 [math.PR] | DOI | MR | Zbl

[29] van der Vaart A.W., Asymptotic statistics, Cambridge Ser. Stat. Probab. Math., 3, Cambridge Univ. Press, Cambridge, 2000 | MR