Efficient Optimization of a Support Vector Regression Model with Natural Logarithm of the Hyperbolic Cosine Loss Function for Broader Noise Distribution

Kocaoğlu A.

APPLIED SCIENCES, vol.14, no.9, pp.1-21, 2024 (SCI-Expanded, Scopus) identifier identifier

  • Publication Type: Article / Article
  • Volume: 14 Issue: 9
  • Publication Date: 2024
  • Doi Number: 10.3390/app14093641
  • Journal Name: APPLIED SCIENCES
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Agricultural & Environmental Science Database, Applied Science & Technology Source, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Page Numbers: pp.1-21
  • Keywords: hyperbolic secant distribution, nonlinear loss functions, nonsmooth optimization, sequential minimal optimization, support vector regression, working set selection
  • Dokuz Eylül University Affiliated: Yes

Abstract

While traditional support vector regression (SVR) models rely on loss functions tailored to specific noise distributions, this research explores an alternative approach: ε" role="presentation" >-ln SVR, which uses a loss function based on the natural logarithm of the hyperbolic cosine function (lncosh). This function exhibits optimality for a broader family of noise distributions known as power-raised hyperbolic secants (PHSs). We derive the dual formulation of the ε" role="presentation" >-ln SVR model, which reveals a nonsmooth, nonlinear convex optimization problem. To efficiently overcome these complexities, we propose a novel sequential minimal optimization (SMO)-like algorithm with an innovative working set selection (WSS) procedure. This procedure exploits second-order (SO)-like information by minimizing an upper bound on the second-order Taylor polynomial approximation of consecutive loss function values. Experimental results on benchmark datasets demonstrate the effectiveness of both the ε" role="presentation" >-ln SVR model with its lncosh loss and the proposed SMO-like algorithm with its computationally efficient WSS procedure. This study provides a promising tool for scenarios with different noise distributions, extending beyond the commonly assumed Gaussian to the broader PHS family.