Geodesic
Le­ven­berg–Mar­quardt method for unconstrained optimization
Vestnik rossijskih universitetov. Matematika, Tome 24 (2019) no. 125, pp. 60-74 Cet article a éte moissonné depuis la source Math-Net.Ru

Voir la notice de l'article

We propose and study the Levenberg–Marquardt method globalized by means of linesearch for unconstrained optimization problems with possibly nonisolated solutions. It is well-recognized that this method is an efficient tool for solving systems of nonlinear equations, especially in the presence of singular and even nonisolated solutions. Customary globalization strategies for the Levenberg–Marquardt method rely on linesearch for the squared Euclidean residual of the equation being solved. In case of unconstrained optimization problem, this equation is formed by putting the gradient of the objective function equal to zero, according to the Fermat principle. However, these globalization strategies are not very adequate in the context of optimization problems, as the corresponding algorithms do not have “preferences” for convergence to minimizers, maximizers, or any other stationary points. To that end, in this work we considers a different technique for globalizing convergence of the Levenberg–Marquardt method, employing linesearch for the objective function of the original problem. We demonstrate that the proposed algorithm possesses reasonable global convergence properties, and preserves high convergence rate of the Levenberg–Marquardt method under weak assumptions.
Keywords: unconstrained optimization problem; nonisolated solutions; Levenberg–Marquardt method; globalization of convergence. 
@article{VTAMU_2019_24_125_a5,
     author = {A. F. Izmailov and A. S. Kurennoy and P. I. Stetsyuk},
     title = {Le\-ven\-berg{\textendash}Mar\-quardt method for unconstrained optimization},
     journal = {Vestnik rossijskih universitetov. Matematika},
     pages = {60--74},
     year = {2019},
     volume = {24},
     number = {125},
     language = {ru},
     url = {http://geodesic.mathdoc.fr/item/VTAMU_2019_24_125_a5/}
}
TY  - JOUR
AU  - A. F. Izmailov
AU  - A. S. Kurennoy
AU  - P. I. Stetsyuk
TI  - Le­ven­berg–Mar­quardt method for unconstrained optimization
JO  - Vestnik rossijskih universitetov. Matematika
PY  - 2019
SP  - 60
EP  - 74
VL  - 24
IS  - 125
UR  - http://geodesic.mathdoc.fr/item/VTAMU_2019_24_125_a5/
LA  - ru
ID  - VTAMU_2019_24_125_a5
ER  - 
%0 Journal Article
%A A. F. Izmailov
%A A. S. Kurennoy
%A P. I. Stetsyuk
%T Le­ven­berg–Mar­quardt method for unconstrained optimization
%J Vestnik rossijskih universitetov. Matematika
%D 2019
%P 60-74
%V 24
%N 125
%U http://geodesic.mathdoc.fr/item/VTAMU_2019_24_125_a5/
%G ru
%F VTAMU_2019_24_125_a5
A. F. Izmailov; A. S. Kurennoy; P. I. Stetsyuk. Le­ven­berg–Mar­quardt method for unconstrained optimization. Vestnik rossijskih universitetov. Matematika, Tome 24 (2019) no. 125, pp. 60-74. http://geodesic.mathdoc.fr/item/VTAMU_2019_24_125_a5/

[1] K. Levenberg, “A method for the solution of certain non-linear problems in least squares”, Quarterly of Appl. Math., 2 (1944), 164–168 | DOI | MR | Zbl

[2] D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters”, J. SIAM, 11 (1963), 431–441 | MR | Zbl

[3] N. Yamashita, M. Fukushima, “On the rate of convergence of the Levenberg–Marquardt method”, Computing, 15 (2001), 237–249 | MR

[4] J.-Y. Fan, Y.-X. Yuan, “On the quadratic convergence of the Levenberg–Marquardt method”, Computing, 74 (2005), 23–39 | DOI | MR | Zbl

[5] P. E. Gill, W. Murray, M. H. Wright, Practical Optimization, Academic Press, San Diego, 1981 | MR | Zbl

[6] R. B. Schnabel, E. Eskow, “A new modified Cholesky factorization”, SIAM J. Sci. Statist. Comput., 11 (1990), 1136–1158 | DOI | MR | Zbl

[7] S. H. Cheng, N. J. Higham, “A modified Cholesky algorithm based on a symmetric indefinite factorization”, SIAM J. Matrix. Anal. Appl., 9 (1998), 1097–1110 | DOI | MR

[8] J. Nocedal and S.J. Wright, Numerical Optimization, 2, Springer-Verlag, Heidelberg–New York–Berlin, 2006 | MR | Zbl

[9] D. Bertsekas, Uslovnaya optimizatsiya i metody mnozhitelei Lagranzha, Radio i svyaz, M., 1987 ; D. P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, 1982 | MR | MR | Zbl

[10] A. Fischer, “Local behavior of an iterative framework for generalized equations with nonisolated solutions”, Math. Program., 94 (2002), 91–124 | DOI | MR | Zbl

[11] A. F. Izmailov, M. V. Solodov, E. I. Uskov, “Globalizing stabilized SQP by smooth primal-dual exact penalty function”, J. Optim. Theory Appl., 169 (2016), 148–178 | DOI | MR | Zbl

[12] K. Ueda, N. Yamashita, “Convergence properties of the regularized Newton method for the unconstrained nonconvex optimization”, Appl. Math. Optim., 62 (2010), 27–46 | DOI | MR | Zbl

[13] C. Shen, X. Chen, Y. Liang, “A regularized Newton method for degenerate unconstrained optimization problems”, Optim. Lett., 6 (2012), 1913–1933 | DOI | MR | Zbl