Geodesic
Regression model based on convex combinations best correlated with response
Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, Tome 55 (2015) no. 3, pp. 530-544 
A. A. Dokukin; O. V. Senko. Regression model based on convex combinations best correlated with response. Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, Tome 55 (2015) no. 3, pp. 530-544. http://geodesic.mathdoc.fr/item/ZVMMF_2015_55_3_a15/
@article{ZVMMF_2015_55_3_a15,
     author = {A. A. Dokukin and O. V. Senko},
     title = {Regression model based on convex combinations best correlated with response},
     journal = {\v{Z}urnal vy\v{c}islitelʹnoj matematiki i matemati\v{c}eskoj fiziki},
     pages = {530--544},
     year = {2015},
     volume = {55},
     number = {3},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/ZVMMF_2015_55_3_a15/}
}
TY  - JOUR
AU  - A. A. Dokukin
AU  - O. V. Senko
TI  - Regression model based on convex combinations best correlated with response
JO  - Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki
PY  - 2015
SP  - 530
EP  - 544
VL  - 55
IS  - 3
UR  - http://geodesic.mathdoc.fr/item/ZVMMF_2015_55_3_a15/
LA  - ru
ID  - ZVMMF_2015_55_3_a15
ER  - 
%0 Journal Article
%A A. A. Dokukin
%A O. V. Senko
%T Regression model based on convex combinations best correlated with response
%J Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki
%D 2015
%P 530-544
%V 55
%N 3
%U http://geodesic.mathdoc.fr/item/ZVMMF_2015_55_3_a15/
%G ru
%F ZVMMF_2015_55_3_a15

Voir la notice de l'article provenant de la source Math-Net.Ru

A new regression method based on constructing optimal convex combinations of simple linear regressions of the least squares method (LSM regressions) built from original regressors is presented. It is shown that, in fact, this regression method is equivalent to a modification of the LSM including the additional requirement of the coincidence of the sign of the regression parameter with that of the correlation coefficient between the corresponding regressor and the response. A method for constructing optimal convex combinations based on the concept of nonexpandable irreducible ensembles is described. Results of experiments comparing the developed method with the known glmnet algorithm are presented, which confirm the efficiency of the former.

[1] Tibshirani R., “Regression shrinkage and selection via the lasso”, J. Roy. Stat. Soc., 58 (1996), 267–288 | MR | Zbl

[2] Zou H., Hastie T., Efron B., Hastie T., “Regularization and variable selection via the elastic net”, J. Roy. Stat. Soc., 67:2 (2005), 301–320 | MR | Zbl

[3] Efron B., Hastie T., Jonnstone I., Tibshirani R., “Least angle regression”, Annals of Statistics, 32:2 (2004), 407–499 | MR | Zbl

[4] Zhuravlev Yu. I., Ryazanov V. V., Senko O. V., Raspoznavanie. Matematicheskie metody. Programmnaya sistema. Prakticheskie primeneniya, Fazis, M., 2006

[5] Kuznetsov V. A., Senko O. V. et al., “Recognition of fuzzy systems by method of statistically weighed syndromes and its using for immunological and hematological norm and chronic pathology”, Chemical Physics, 15:1 (1996), 81–100

[6] Breiman L., Random forests — random features, Technical 567, , Statistics department. University of California, Berkley, September, 1999 http://www.boosting.org

[7] Kuncheva L. I., Combining pattern classifiers. Methods and algorithms, Wiley Interscience, New Jersey, 2004 | MR | Zbl

[8] Zhuravlev Yu. I., Kuznetsova A. V., Ryazanov V. V., Senko O. V., Botvin M. A., “The use of pattern recognition methods in tasks of biomedical diagnostics and forecasting”, Pattern Recognition and Image Analysis, 18:2 (2008), 195–200 | MR

[9] Brown G., Wyatt J. L., Tino P., “Managing diversity in regression ensembles”, J. Machine Learning Research, 6 (2005), 1621–1650 | MR | Zbl

[10] Senko O. V., “An optimal ensemble of predictors in convex correcting procedures”, Pattern Recognition and Image Analysis, 19:3 (2009), 465–468

[11] Senko O., Dokukin A., “Optimal forecasting based on convex correcting procedures”, New Trends in Classification and Data Mining, ITHEA, Sofia, 2010, 62–72

[12] Dokukin A. A., Senko O. V., “Optimalnye vypuklye korrektiruyuschie protsedury v zadachakh vysokoi razmernosti”, Zh. vychisl. matem. i matem. fiz., 51:9 (2011), 1751–1760 | MR | Zbl

[13] Senko O., Dokukin A., “Correlation maximization in regression models based on convex combinations”, Information Theories and Applications, 18:3 (2011), 224–231

[14] Friedman J. H., Hastie T., Tibshirani R., “Regularization paths for generalized linear models via coordinate descent”, J. Statistical Software, 33:1 (2010), 1–22