Geodesic
Accelerated gradient sliding for minimizing a sum of functions
Doklady Rossijskoj akademii nauk. Matematika, informatika, processy upravleniâ, Tome 492 (2020), pp. 85-88.

Voir la notice de l'article provenant de la source Math-Net.Ru

We propose a new way of justifying the accelerated gradient sliding of G. Lan, which allows one to extend the sliding technique to a combination of an accelerated gradient method with an accelerated variance reduction method. New optimal estimates for the solution of the problem of minimizing a sum of smooth strongly convex functions with a smooth regularizer are obtained.
Keywords: accelerated gradient sliding of G. Lan, accelerated variance reduction methods, smooth strongly convex functions. 
@article{DANMA_2020_492_a17,
     author = {D. M. Dvinskikh and S. S. Omelchenko and A. V. Gasnikov and A. I. Turin},
     title = {Accelerated gradient sliding for minimizing a sum of functions},
     journal = {Doklady Rossijskoj akademii nauk. Matematika, informatika, processy upravleni\^a},
     pages = {85--88},
     publisher = {mathdoc},
     volume = {492},
     year = {2020},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/DANMA_2020_492_a17/}
}
TY  - JOUR
AU  - D. M. Dvinskikh
AU  - S. S. Omelchenko
AU  - A. V. Gasnikov
AU  - A. I. Turin
TI  - Accelerated gradient sliding for minimizing a sum of functions
JO  - Doklady Rossijskoj akademii nauk. Matematika, informatika, processy upravleniâ
PY  - 2020
SP  - 85
EP  - 88
VL  - 492
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/DANMA_2020_492_a17/
LA  - ru
ID  - DANMA_2020_492_a17
ER  - 
%0 Journal Article
%A D. M. Dvinskikh
%A S. S. Omelchenko
%A A. V. Gasnikov
%A A. I. Turin
%T Accelerated gradient sliding for minimizing a sum of functions
%J Doklady Rossijskoj akademii nauk. Matematika, informatika, processy upravleniâ
%D 2020
%P 85-88
%V 492
%I mathdoc
%U http://geodesic.mathdoc.fr/item/DANMA_2020_492_a17/
%G ru
%F DANMA_2020_492_a17
D. M. Dvinskikh; S. S. Omelchenko; A. V. Gasnikov; A. I. Turin. Accelerated gradient sliding for minimizing a sum of functions. Doklady Rossijskoj akademii nauk. Matematika, informatika, processy upravleniâ, Tome 492 (2020), pp. 85-88. http://geodesic.mathdoc.fr/item/DANMA_2020_492_a17/

[1] Gasnikov A.V., Sovremennye chislennye metody optimizatsii. Metod universalnogo gradientnogo spuska, MFTI, M., 2018

[2] Allen-Zhu Z., Hazan E., “Optimal Black-Box Reductions between Optimization Objectives”, Advances in Neural Information Processing Systems, 2016, 1614–1622

[3] Beznosikov A., Gorbunov E., Gasnikov A., “Derivative-Free Method for Decentralized Distributed Non-Smooth Optimization”, IFAC 2020. The 21st World Congress of the International Federation of Automatic Control

[4] Bottou L., Curtis F.E., Nocedal J., “Optimization Methods for Large-Scale Machine Learning”, Siam Review, 60:2 (2018), 223–311 | DOI | MR | Zbl

[5] Dvurechensky P., Gasnikov A., Tiurin A., Randomized Similar Triangles Method: A Unifying Framework for Accelerated Randomized Optimization Methods (Coordinate Descent, Directional Search, Derivative-Free Method), arXiv: 1707.08486

[6] Hazan E., Lecture Notes: Optimization for Machine Learning, arXiv: 1909.03550

[7] Ivanova A., Gasnikov A., Dvurechensky P., Dvinskikh D., Tyurin A., Vorontsova E., Pasechnyuk D., Oracle Complexity Separation in Convex Optimization, arXiv: 2002.02706

[8] Ivanova A., Grishchenko D., Gasnikov A., Shulgin E., Adaptive Catalyst for Smooth Convex Optimization, arXiv: 1911.11271

[9] Lan G., Lectures on Optimization. Methods for Machine Learning https://pwp.gatech.edu/guanghui-lan/publications/

[10] Lan G., Li Z., Zhou Y., “A Unified Variance-Reduced Accelerated Gradient Method for Convex Optimization”, Advances in Neural Information Processing Systems, 2019, 10462–10472

[11] Lin H., Mairal J., Harchaoui Z., “Catalyst Acceleration for First-Order Convex Optimization: from Theory to Practice”, J. Machine Learning Research, 18:1 (2017), 7854–7907 | MR

[12] Nesterov Yu., “Gradient Methods for Minimizing Composite Functions”, Math. Prog., 140:1 (2013), 125–161 | DOI | MR | Zbl

[13] Nesterov Yu., “Smooth Minimization of Non-Smooth Function”, Math. Program, 103:1 (2005), 127–152 | DOI | MR | Zbl

[14] Shalev-Shwartz S., Ben-David S., Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, Cambridge, 2014 | Zbl

[15] Shalev-Shwartz S., Shamir O., Srebro N., Sridharan K., “Stochastic Convex Optimization”, COLT 2009