The Lezanski -- Polyak -- Lojasiewicz inequality and the convergence of~the~gradient projection algorithm

M. V. Balashov

Izvestiya of Saratov University. Mathematics. Mechanics. Informatics, Tome 23 (2023) no. 1, pp. 4-10.

Résumé

We consider the Lezanski – Polyak – Lojasiewicz inequality for a real-analytic function on a real-analytic compact manifold without boundary in finite-dimensional Euclidean space. This inequality emerged in 1963 independently in works of three authors: Lezanski and Lojasiewicz from Poland and Polyak from the USSR. The inequality is appeared to be a very useful tool in the convergence analysis of the gradient methods, firstly in unconstrained optimization and during the past few decades in problems of constrained optimization. Basically, it is applied for a smooth in a certain sense function on a smooth in a certain sense manifold. We propose the derivation of the inequality from the error bound condition of the power type on a compact real-analytic manifold. As an application, we prove the convergence of the gradient projection algorithm of a real analytic function on a real analytic compact manifold without boundary. Unlike known results, our proof gives explicit dependence of the error via parameters of the problem: the power in the error bound condition and the constant of proximal smoothness first of all. Here we significantly use a technical fact that a smooth compact manifold without boundary is a proximally smooth set.

Bibliographie

[1] Lee J. M., Manifolds and differential geometry, Graduate Studies in Mathematics, 107, AMS Providence, Rhode Island, 2009, 671 pp. | DOI | MR

[2] Lojasiewicz S., “Sur le probléme de la division”, Studia Mathematica, 18 (1959), 87–136 | DOI | MR

[3] Balashov M. V., Polyak B. T., Tremba A. A., “Gradient projection and conditional gradient methods for constrained nonconvex minimization”, Numerical Functional Analysis and Optimization, 41:7 (2020), 822–849 | DOI | MR

[4] Karimi H., Nutini J., Schmidt M., “Linear convergence of gradient and proximal-gradient methods under the Polyak – Lojasiewicz condition”, Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2016, Lecture Notes in Computer Science, 9851, eds. Frasconi P., Landwehr N., Manco G., Vreeken J., Springer, Cham, 2016, 795–811 | DOI

[5] Schneider R., Uschmajew A., “Convergence results for projected line-search methods on varieties of low-rank matricies via Lojasiewicz inequality”, SIAM Journal on Optimization, 25:1 (2015), 622–646 | DOI | MR

[6] Merlet B., Nguyen T. N., “Convergence to equilibrium for discretizations of gradient-like flows on Riemannian manifolds”, Differential Integral Equations, 26:5/6 (2013), 571–602 | DOI | MR

[7] Vial J.-Ph., “Strong and weak convexity of sets and functions”, Mathematics of Operations Research, 8:2 (1983), 231–259 | DOI | MR

[8] Balashov M. V., “The gradient projection algorithm for smooth sets and functions in nonconvex case”, Set-Valued and Variational Analysis, 29 (2021), 341–360 | DOI | MR

[9] Balashov M. V., “Stability of minimization problems and the error bound condition”, Set-Valued and Variational Analysis, 30 (2022), 1061–1076 | DOI | MR

[10] Ivanov G. E., Weakly Convex Sets and Functions: Theory and Application, Fizmatlit, M., 2006, 351 pp. (in Russian) | MR

[11] Balashov M. V., Kamalov R. A., “The gradient projection method with Armijo's step size on manifolds”, Computational Mathematics and Mathematical Physics, 61:1 (2021), 1776–1786 | DOI | MR

[12] Adly S., Nacry F., Thibault L., “Preservation of prox-regularity of sets with applications to constrained optimization”, SIAM Journal on Optimization, 26:1 (2016), 448–473 | DOI | MR

[13] Polyak B. T., Introduction to Optimization, Optimization Software, New York, 1987, 464 pp. | MR

[14] Balashov M. V., Tremba A. A., “Error bound conditions and convergence of optimization methods on smooth and proximally smooth manifolds”, Optimization: A Journal of Mathematical Programming and Operations Research, 71:3 (2020), 711–735 | DOI | MR