New Douglas-Rashford Splitting Algorithms for Generalized DC Programming with Applications in Machine Learning

Aug 5, 2026

Upload by Noah

This work introduces new Douglas-Rachford splitting algorithms to solve generalized DC programming problems in real Hilbert spaces, where the objective combines two proper convex (possibly nonsmooth) terms and a convex smooth term with a subtractive structure. The methods exploit proximal properties of the nonsmooth component and incorporate a fast control parameter to improve convergence. Convergence to critical points is established under reasonable assumptions for nonconvex optimization. Numerical experiments on machine learning examples demonstrate efficiency and improved performance over DCA and ADMM.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages32

Reading Time1 h 21 min

AccessFree to download

Introduction

Contribution

Organization

arXiv :2404 . 14800v1 [math .OC] 23 Apr 2024 New Douglas-Rashford Splitting Algorithms for Generalized DC Programming with Applications in Machine Learning Yonghong Yao ∗ , Lateef O. Jolaoso†, Yekini Shehu‡, Jen-Chih Yao§ April 24, 2024 Abstract In this work, we propose some new Douglas-Rashford splitting algorithms for solving a class of generalized DC (di􀀋erence of convex functions) in real Hilbert spaces. The proposed methods leverage the proximal properties of the nonsmooth component and a fasten control parameter which improves the convergence rate of the algorithms. We prove the convergence of these methods to the critical points of nonconvex optimization under reasonable conditions. We evaluate the performance and e􀀋ectiveness of our methods through experimentation with three practical examples in machine learning. Our 􀀌ndings demonstrated that our methods o􀀋er e􀀎ciency in problem-solving and outperform state-of-the-art techniques like the DCA (DC Algorithm) and ADMM. Keywords: Douglas-Rachford splitting algorithm; DC programming; Nonconvex optimization; Machine learning. 2010 MSC classi􀀌cation: 65K05, 90C26, 90C30 . 1 Introduction Let us consider the following class of generalized DC programming in a real Hilbert space H: min{p(x) = f(x) + g(x) − h(x)} , (1) x∈H where f, g : H → ( −∞ , +∞ ] are proper, convex, and lower semicontinuous (not necessarily smooth) functions, and h : H → ( −∞ , +∞ ] is a convex and smooth ∗ School of Mathematical Sciences, Tiangong University, Tianjin 300387, China; and Center for Advanced Information Technology, Kyung Hee University, Seoul 02447, South Korea; e-mail: [yyhtgu@hotmail.com](yyhtgu@hotmail.com) †School of Mathematical Sciences, University of Southampton, SO17 1BJ, United Kingdom; e-mail: [l.o.jolaoso@soton.ac.uk](l.o.jolaoso@soton.ac.uk). ‡(Corresponding Author) School of Mathematical Sciences, Zhejiang Normal University, Jinhua 321004, People’s Republic of China; e-mail: [yekini.shehu@zjnu.edu.cn](yekini.shehu@zjnu.edu.cn) § Center for General Education, China Medical University, Taichung 40402, Taiwan, Academy of Romanian Scientists, Bucharest, Romania; e-mail: [yaojc@mail.cmu.edu.tw](yaojc@mail.cmu.edu.tw) function. The DC programming (1) was 􀀌rst presented by Tao et al. [41], which has received attention due to its applications in image processing [46], compressed sensing [31], statistics and machine learning [19 , 27 , 28 , 35], dimensionality reduction [16] and multiple-input-multiple-output (MIMO) [53] . For example, in the application of DC programming (1) in machine learning, the function f stands for a loss function that denotes the data 􀀌delity and g − h represents a regularizer that induces some expected structures in the solution [29, 30] . The DCA (DC algorithm) is one of the most prominent methods for solving DC programming (1) . The DCA linearizes the concave part of the DC programming (1) at the current iteration and obtains the next iteration via a convex subproblem. Further studies on the DCA have been given in [20 , 32 , 52] . In [11, Algorithm 1], Chuang et al. introduced the following uni􀀌ed Douglas-Rachdford splitting algorithm to solve DC programming (1): 􀀸 􀀾 􀀾 􀀾 􀀼 􀀾 􀀾 􀀾 􀀺 yn = arinnf (v) + 2~~1~~β kv − xnk2 o, zn = arin ng (v) + 2~~1~~β kv − (2yn − xn + β∇h(yn ))k2 o, xn+1 = xn + κn (zn − yn ) , (2) with β > 0 and κn ∈ (0 , 2), and obtained, under certain conditions, that lim kyn − n→∞ xk = 0, lim xn = y, where x is a stationary point of DC programming (1) and n→∞ proxβf (y) = x. In [4], Bian and Zhang extended the three-operator splitting algorithm of Davis-Yin [15] from solving the optimization problem of the sum of three convex functions to solving nonconvex optimization problems. This three-operator in [15] could be seen as a slight extension of the Douglas-Rachford splitting algorithm [18] and the generalized forward-backward splitting algorithm in [3 , 38] . Motivated by t

FAQs

What class of problems do the proposed algorithms solve?

They solve generalized DC programming problems in real Hilbert spaces, minimizing f(x)+g(x)-h(x) where f and g are proper convex (not necessarily smooth) and h is convex and smooth.

How do the new methods improve convergence?

They use proximal properties of the nonsmooth component and introduce a fast control parameter to enhance the convergence rate.

How is the algorithm performance validated for machine learning?

Experiments on three practical machine learning examples compare accuracy and efficiency against state-of-the-art approaches such as DCA and ADMM, showing the proposed methods outperform them.

Aug 5, 2026

Upload by Noah