New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Hemici, Youcef Elhamam ; Khelladi, Samia ; Benterki, Djamel

Kybernetika, Tome 60 (2024) no. 4, pp. 535-552

Abstract (VO)

The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of $\beta _{k}^{RMIL}$ and $\beta _{k}^{HS}$. We compute the convex parameter $\theta _{k}$ using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant.

Détail

DOI : 10.14736/kyb-2024-4-0535

Classification : 65K05, 90C26, 90C30
Keywords: unconstrained optimization; conjugate gradient method; descent direction; line search; image restoration

Bibliographie

[1] Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10 (2008), 147-161. | DOI | MR

[2] Andrei, N.: Nonlinear conjugate gradient methods for unconstrained optimization. Springer Optimization and its Applications, Romania 2020. | MR

[3] Hanachi, S. Ben, Sellami, B., Belloufi, M.: New iterative conjugate gradient method for nonlinear unconstrained optimization. RAIRO - Oper. Res. 56 (2022), 2315-2327. | DOI | MR

[4] Cai, J. F., Chan, R., Morini, B.: Minimization of an edge-preserving regularization functional by conjugate gradient type methods. Image Processing Based on Partial Differential Equations. Mathematics and Visualization. Springer, Berlin, Heidelberg 2007. | MR

[5] Chan, R. H., Ho, C. W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14 (2005), 10, 1479-1485. | DOI | MR

[6] Dai, Y. H., Yuan, Y.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10 (1999), 1, 177-182. | DOI | MR | Zbl

[7] Dai, Y. H., Yuan, Y.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103 (2001), 33-47. | DOI | MR

[8] Delladji, S., Belloufi, M., Sellami, B.: New hybrid conjugate gradient method as a convex combination of FR and BA methods. J. Inform. Optim. Sci. 42 (2021), 3, 591-602.

[9] Djordjevic, S.: New hybrid conjugate gradient method as a convex combination of LS and FR methods. Acta Math. Scientia 39B (2019), 1, 214-228. | DOI | MR

[10] Djordjevic, S.: New hybrid conjugate gradient method as a convex combination of HS and FR methods. J. Appl. Math. Comput. 2 (2018), 9, 366-378. | MR

[11] Dolan, E. D., Moré, J. J.: Benchmarking optimization software with performance profiles. Math. Program. 91 (2002), 201-213. | DOI | MR | Zbl

[12] Fletcher, R.: Practical Methods of Optimization. Unconstrained Optimization. Wiley, New York 1987. | MR

[13] Fletcher, R., Reeves, C. M.: Function minimization by conjugate gradients. Comput. J. 7 (1964), 2, 149-154. | DOI | MR | Zbl

[14] Hager, W. W., Zhang, H.: A survey of nonlinear conjugate gradient methods. Pacific J. Optim. 2 (2006), 35-58. | MR

[15] Hestenes, M. R., Steifel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49 (1952), 6, 409-436. | DOI | MR

[16] Jiang, X., Liao, W., Yin, J., Jian, J.: A new family of hybrid three-term conjugate gradient methods with applications in image restoration. Numer. Algor. 91 (2022), 161-191. | DOI | MR

[17] Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithm. Part 1: Theory. J. Optim. Theory Appl. 69 (1991), 1, 129-137. | DOI | MR

[18] Ma, G., Lin, H., Jin, W., Han, D.: Two modified conjugate gradient methods for unconstrained optimization with applications in image restoration problems. J. Appl. Math. Comput. 68 (2022), 4733-4758. | DOI | MR

[19] Malik, M., Sulaiman, I. M., Abubakar, A. B., Ardaneswari, G., Sukono, A.: A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection. AIMS Math. 8 (2023), 1-28. | DOI | MR

[20] Mtagulwa, P., Kaelo, P.: A convergent modified HS-DY hybrid conjugate gradient method for unconstrained optimization problems. J. Inform. Optim. Sci. 40 (2019), 1, 97-113. | MR

[21] Polak, E., Ribiere, G.: Note sur la convergence des mé thodes de directions conjuguées. Rev. Française Inform. Recherche Opertionelle 16 (1969), 35-43. | MR

[22] Polyak, B. T.: The conjugate gradient method in extreme problems. U.S.S.R. Comput. Math. Phys. 9 (1969), 94-112. | DOI

[23] Powell, M. J. D.: Restart procedures of the conjugate gradient method. Math. Program. 2 (1977), 241-254. | DOI | MR

[24] Rivaie, M., Mustafa, M., Leong, W. J., Ismail, M.: A new class of nonlinear conjugate gradient coefficients with global convergence properties. Appl. Math. Comput. 218 (2012), 22, 11323-11332. | DOI | MR

[25] Rivaie, M., Mustafa, M., Abdelrhaman, A.: A new class of non linear conjugate gradient coefficients with exact and inexact line searches. Appl. Math. Comput. 268 (2015), 1152-1163. | DOI | MR

[26] Sellami, B., Chaib, Y.: A new family of globally convergent conjugate gradient methods. Ann. Oper. Res. Springer 241 (2016), 497-513. | DOI | MR

[27] Sellami, B., Chaib, Y.: New conjugate gradient method for unconstrained optimization. RAIRO Oper. Res. 50 (2016), 1013-1026. | DOI | MR

[28] Yang, X., Luo, Z., Dai, X.: A global convergence of LS-CD hybrid conjugate gradient method. Adv. Numer. Anal. 2013 (2013), 5 pp. | MR

[29] Zoutendijk, G.: Nonlinear programming, computational methods. In: Integer and Nonlinear Programming (J. Abadie, ed.), 1970, pp. 37-86. | MR