Efficient Optimization of a Support Vector Regression Model with Natural Logarithm of the Hyperbolic Cosine Loss Function for Broader Noise Distribution

Kocaoğlu A.

APPLIED SCIENCES, cilt.14, sa.9, ss.1-21, 2024 (SCI-Expanded, Scopus)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 14 Sayı: 9
  • Basım Tarihi: 2024
  • Doi Numarası: 10.3390/app14093641
  • Dergi Adı: APPLIED SCIENCES
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Agricultural & Environmental Science Database, Applied Science & Technology Source, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Sayfa Sayıları: ss.1-21
  • Anahtar Kelimeler: hyperbolic secant distribution, nonlinear loss functions, nonsmooth optimization, sequential minimal optimization, support vector regression, working set selection
  • Dokuz Eylül Üniversitesi Adresli: Evet

Özet

While traditional support vector regression (SVR) models rely on loss functions tailored to specific noise distributions, this research explores an alternative approach: ε" role="presentation" >-ln SVR, which uses a loss function based on the natural logarithm of the hyperbolic cosine function (lncosh). This function exhibits optimality for a broader family of noise distributions known as power-raised hyperbolic secants (PHSs). We derive the dual formulation of the ε" role="presentation" >-ln SVR model, which reveals a nonsmooth, nonlinear convex optimization problem. To efficiently overcome these complexities, we propose a novel sequential minimal optimization (SMO)-like algorithm with an innovative working set selection (WSS) procedure. This procedure exploits second-order (SO)-like information by minimizing an upper bound on the second-order Taylor polynomial approximation of consecutive loss function values. Experimental results on benchmark datasets demonstrate the effectiveness of both the ε" role="presentation" >-ln SVR model with its lncosh loss and the proposed SMO-like algorithm with its computationally efficient WSS procedure. This study provides a promising tool for scenarios with different noise distributions, extending beyond the commonly assumed Gaussian to the broader PHS family.