Model selection for regression on a random design

Baraud, Yannick 

ESAIM: Probability and Statistics, Tome 6 (2002), pp. 127-146

Résumé

We consider the problem of estimating an unknown regression function when the design is random with values in ${ℝ}^k$. Our estimation procedure is based on model selection and does not rely on any prior information on the target function. We start with a collection of linear functional spaces and build, on a data selected space among this collection, the least-squares estimator. We study the performance of an estimator which is obtained by modifying this least-squares estimator on a set of small probability. For the so-defined estimator, we establish nonasymptotic risk bounds that can be related to oracle inequalities. As a consequence of these, we show that our estimator possesses adaptive properties in the minimax sense over large families of Besov balls ${ℬ}_{\alpha ,l,\infty }\left(R\right)$ with $R>0$, $l\ge 1$ and $\alpha >{\alpha }_l$ where ${\alpha }_l$ is a positive number satisfying $1/l-1/2\le {\alpha }_l<1/l$. We also study the particular case where the regression function is additive and then obtain an additive estimator which converges at the same rate as it does when $k=1$.

Détail

DOI : 10.1051/ps:2002007

Classification : 62G07, 62J02
Keywords: nonparametric regression, least-squares estimators, penalized criteria, minimax rates, Besov spaces, model selection, adaptive estimation