Geodesic
The LASSO estimator: Distributional properties
Kybernetika, Tome 54 (2018) no. 4, pp. 778-797
Cet article a éte moissonné depuis la source Czech Digital Mathematics Library

Voir la notice de l'article

The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.
The least absolute shrinkage and selection operator (LASSO) is a popular technique for simultaneous estimation and model selection. There have been a lot of studies on the large sample asymptotic distributional properties of the LASSO estimator, but it is also well-known that the asymptotic results can give a wrong picture of the LASSO estimator's actual finite-sample behaviour. The finite sample distribution of the LASSO estimator has been previously studied for the special case of orthogonal models. The aim in this work is to generalize the finite sample distribution properties of LASSO estimator for a real and linear measurement model in Gaussian noise. In this work, we derive an expression for the finite sample characteristic function of the LASSO estimator, we then use the Fourier slice theorem to obtain an approximate expression for the marginal probability density functions of the one-dimensional components of a linear transformation of the LASSO estimator.
DOI : 10.14736/kyb-2018-4-0778
Classification : 60E05, 62E15, 62G05, 62J05
Keywords: linear regression; LASSO; characteristic function; finite sample probability distribution function; Fourier-Slice theorem; Cramer–Wold theorem 
@article{10_14736_kyb_2018_4_0778,
     author = {Jagannath, Rakshith and Upadhye, Neelesh S.},
     title = {The {LASSO} estimator: {Distributional} properties},
     journal = {Kybernetika},
     pages = {778--797},
     year = {2018},
     volume = {54},
     number = {4},
     doi = {10.14736/kyb-2018-4-0778},
     mrnumber = {3863256},
     zbl = {06987034},
     language = {en},
     url = {http://geodesic.mathdoc.fr/articles/10.14736/kyb-2018-4-0778/}
}
TY  - JOUR
AU  - Jagannath, Rakshith
AU  - Upadhye, Neelesh S.
TI  - The LASSO estimator: Distributional properties
JO  - Kybernetika
PY  - 2018
SP  - 778
EP  - 797
VL  - 54
IS  - 4
UR  - http://geodesic.mathdoc.fr/articles/10.14736/kyb-2018-4-0778/
DO  - 10.14736/kyb-2018-4-0778
LA  - en
ID  - 10_14736_kyb_2018_4_0778
ER  - 
%0 Journal Article
%A Jagannath, Rakshith
%A Upadhye, Neelesh S.
%T The LASSO estimator: Distributional properties
%J Kybernetika
%D 2018
%P 778-797
%V 54
%N 4
%U http://geodesic.mathdoc.fr/articles/10.14736/kyb-2018-4-0778/
%R 10.14736/kyb-2018-4-0778
%G en
%F 10_14736_kyb_2018_4_0778
Jagannath, Rakshith; Upadhye, Neelesh S. The LASSO estimator: Distributional properties. Kybernetika, Tome 54 (2018) no. 4, pp. 778-797. doi: 10.14736/kyb-2018-4-0778

[1] Austin, C. D., Moses, R., Ash, J., Ertin, E.: On the relation between sparse reconstruction and parameter estimation with model order selection. IEEE J. Selected Topics Signal Process. 4 (2010), 3, 560-570. | DOI

[2] Babacan, S., Molina, R., Katsaggelos, A.: Bayesian compressive sensing using laplace priors. IEEE Trans. Image Process. 19 (2010), 1, 53-63. | DOI | MR

[3] Baraniuk, R., Candes, E., Nowak, R., R., M., Vetterli: Compressive sampling. IEEE Signal Processing Magazine 25 (2008), 2, 12-13. | DOI

[4] Ben-Tal, A., Nemirovskiaei, A. S.: Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. Society for Industrial and Applied Mathematics, Philadelphia 2001. | DOI | MR

[5] Boufounos, P., Duarte, M. F., Baraniuk, R. G.: Sparse signal reconstruction from noisy compressive measurements using cross validation. In: IEEE/SP 14th Workshop on Statistical Signal Processing, 2007, pp. 299-303. | DOI | MR

[6] Candes, E.: The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique 346 (2008), 9-10, 589-592. | DOI | MR

[7] Chen, S. S., Donoho, D. L., Saunders, M. A.: Atomic decomposition by basis pursuit. SIAM Rev. 43 (2001), 1, 129-159. | DOI | MR

[8] Cuesta-Albertos, J. A., Fraiman, R., Ransford, T.: A sharp form of the Cramér-wold theorem. J. Theoret. Probab. 20 (2007), 2, 201-209. | DOI | MR

[9] Donoho, D.: Compressed sensing. IEEE Trans. Inform. Theory 52 (2006), 4, 1289-1306. | DOI | MR

[10] Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Statist. 32 (2004), 407-499. | DOI | MR

[11] Eldar, Y.: Generalized sure for exponential families: Applications to regularization. IEEE Trans. Signal Process. 57 (2009), 2, 471-481. | DOI | MR

[12] Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 (2001), 456, 1348-1360. | DOI | MR

[13] Grant, M., Boyd, S.: {CVX}: Matlab software for disciplined convex programming, version 2.1.

[14] Kabaila, P.: The effect of model selection on confidence regions and prediction regions. Econometr. Theory 11 (1995), 537-549. | DOI | MR

[15] Kay, S. M.: Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall, Inc., Upper Saddle River, NJ 1993.

[16] Knight, K., Fu, W.: Asymptotics for lasso-type estimators. Ann. Statist. 28 (2000), 5, 1356-1378. | DOI | MR

[17] Krim, H., Viberg, M.: Two decades of array signal processing research: the parametric approach. IEEE Signal Processing Magazine 13 (1996), 4, 67-94. | DOI

[18] Leeb, H., Pötscher, B. M.: Model selection and inference: Facts and fiction. Econometr. Theory 21 (2005), 21-59. | DOI | MR

[19] Lockhart, R., Taylor, J., Tibshirani, R. J., Tibshirani, R.: A significance test for the lasso. Ann. Statist. 42 (2014), 2, 413-468. | DOI | MR

[20] Lopes, M. E.: Estimating unknown sparsity in compressed sensing. CoRR 2012, abs/1204.4227.

[21] Ng, R.: Fourier slice photography. ACM Trans. Graph. 24 (2005), 3, 735-744. | DOI

[22] Panahi, A., Viberg, M.: Fast candidate points selection in the lasso path. IEEE Signal Process. Lett. 19 (2012), 2, 79-82. | DOI

[23] Pötscher, B. M., Leeb, H.: On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. J. Multivar. Anal. 100 (2009), 9, 2065-2082. | DOI | MR

[24] Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc., Ser. B 58 (1994), 267-288. | MR

[25] Zou, H.: The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 (2006), 476, 1418-1429. | DOI | MR

Cité par Sources :