Geodesic
Computationally efficient algorithm for Gaussian Process regression in case of structured samples
Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, Tome 56 (2016) no. 4, pp. 507-522 
M. Belyaev; E. Burnaev; E. Kapushev. Computationally efficient algorithm for Gaussian Process regression in case of structured samples. Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki, Tome 56 (2016) no. 4, pp. 507-522. http://geodesic.mathdoc.fr/item/ZVMMF_2016_56_4_a0/
@article{ZVMMF_2016_56_4_a0,
     author = {M. Belyaev and E. Burnaev and E. Kapushev},
     title = {Computationally efficient algorithm for {Gaussian} {Process} regression in case of structured samples},
     journal = {\v{Z}urnal vy\v{c}islitelʹnoj matematiki i matemati\v{c}eskoj fiziki},
     pages = {507--522},
     year = {2016},
     volume = {56},
     number = {4},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/ZVMMF_2016_56_4_a0/}
}
TY  - JOUR
AU  - M. Belyaev
AU  - E. Burnaev
AU  - E. Kapushev
TI  - Computationally efficient algorithm for Gaussian Process regression in case of structured samples
JO  - Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki
PY  - 2016
SP  - 507
EP  - 522
VL  - 56
IS  - 4
UR  - http://geodesic.mathdoc.fr/item/ZVMMF_2016_56_4_a0/
LA  - ru
ID  - ZVMMF_2016_56_4_a0
ER  - 
%0 Journal Article
%A M. Belyaev
%A E. Burnaev
%A E. Kapushev
%T Computationally efficient algorithm for Gaussian Process regression in case of structured samples
%J Žurnal vyčislitelʹnoj matematiki i matematičeskoj fiziki
%D 2016
%P 507-522
%V 56
%N 4
%U http://geodesic.mathdoc.fr/item/ZVMMF_2016_56_4_a0/
%G ru
%F ZVMMF_2016_56_4_a0

Voir la notice de l'article provenant de la source Math-Net.Ru

Surrogate modeling is widely used in many engineering problems. Data sets often have Cartesian product structure (for instance factorial design of experiments with missing points). In such case the size of the data set can be very large. Therefore, one of the most popular algorithms for approximation-Gaussian Process regression-can be hardly applied due to its computational complexity. In this paper a computationally efficient approach for constructing Gaussian Process regression in case of data sets with Cartesian product structure is presented. Efficiency is achieved by using a special structure of the data set and operations with tensors. Proposed algorithm has low computational as well as memory complexity compared to existing algorithms. In this work we also introduce a regularization procedure allowing to take into account anisotropy of the data set and avoid degeneracy of regression model.

[1] Forrester A., Sobester A., Keane A., Engineering design via surrogate modelling: a practical guide, John Wiley Sons, 2008

[2] Rasmussen C. E., Williams C., Gaussian processes for machine learning, MIT Press, 2006 | MR | Zbl

[3] Rasmussen C. E., Ghahramani Z., “Infinite mixtures of gaussian process experts”, Advances in neural information processing systems 14, v. 2, 2002, 881–888

[4] Quinñonero-Candela J., Rasmussen C. E., “A unifying view of sparse approximate Gaussian process regression”, J. Mach. Learn. Res., 6 (2005), 1939–1959 | MR

[5] Montgomery D. C., Design and analysis of experiments, John Wiley Sons, 2006

[6] Charles J. S., Hansen M., Kooperberg C., Truong Y. K., “Polynomial splines and their tensor products in extended linear modeling”, Ann. Statist., 25 (1997), 1371–1470 | DOI | MR | Zbl

[7] Dietrich C. R., Newsam G. N., “Fast and exact simulation of stationary gaussian processes through circulant embedding of the covariance matrix”, SIAM J. Sci. Comput., 18 (1997), 1088–1107 | DOI | MR | Zbl

[8] Zimmerman D. L., “Computationally efficient restricted maximum likelihood estimation of generalized covariance functions”, Mathematical Geology, 21:7 (1989), 655–672 | DOI | MR

[9] Chan G., Wood A. T. A., “Algorithm AS 312: an algorithm for simulating stationary Gaussian random fields”, J. Royal Statistical Society: Series C (Applied Statistics), 46:1 (1997), 171–181 | DOI | Zbl

[10] Rendall T. C. S., Allen C. B., “Multi-dimensional aircraft surface pressure interpolation using radial basis functions”, Proc. IMechE Part G: Aerospace Engineering, 222 (2008), 483–495 | DOI

[11] Shiryaev A. N., Veroyatnost-1: elementarnaya teoriya veroyatnostei, matematicheskie osnovaniya, predelnye teoremy, Izd-vo MTsNMO, M., 2011

[12] Tamara G. K., Brett W. B., “Tensor decompositions and applications”, SIAM Rev., 51:3 (2009), 455–500 | DOI | MR | Zbl

[13] Horn R. A., Johnson C. R., Topics in matrix analysis, Cambridge University Press, 1994 | MR | Zbl

[14] Rasmussen C. E., Nickisch H., “Gaussian processes for machine learning (GPML) Toolbox”, J. Mach. Learn. Res., 11 (2010), 3011–3015 | MR | Zbl

[15] Nocedal J., Wright S. J., Numerical optimization, Second Edition, Springer, New York, 2006 | MR | Zbl

[16] Evoluationary computation pages — the function testbed, http://www.tik.ee.ethz.ch/sop/download/supplementary/testproblems/

[17] System optimization — testproblems, http://www.tik.ee.ethz.ch/sop/download/supplementary/testproblems/

[18] Snelson E., Ghahramani Z., “Sparse Gaussian processes using pseudo-inputs”, Advances in neural information processing systems, 18 (2005), 1257–1264

[19] Friedman J. H., “Multivariate adaptive regression splines”, The annals of statistics, 1991, 1–67 | DOI | MR

[20] Dolan E. D., Moré J. J., “Benchmarking optimization software with performance profiles”, Math. Programming, 91:2 (2002), 201–213 | DOI | MR | Zbl