Q-learning and hyper-heuristic based algorithm recommendation for changing environments


Golcuk I., ÖZSOYDAN F. B.

ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, vol.102, 2021 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 102
  • Publication Date: 2021
  • Doi Number: 10.1016/j.engappai.2021.104284
  • Journal Name: ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Aerospace Database, Applied Science & Technology Source, Communication Abstracts, Computer & Applied Sciences, INSPEC, Metadex, Civil Engineering Abstracts
  • Keywords: Dynamic optimization, Hyper-heuristics, Q-learning, Multidimensional knapsack problem, BIO-INSPIRED OPTIMIZER, DYNAMIC OPTIMIZATION, EVOLUTIONARY, INTELLIGENCE, OPERATORS
  • Dokuz Eylül University Affiliated: Yes

Abstract

A considerable amount of research has been devoted to solving static optimization problems via bio-inspired metaheuristic algorithms. However, most of the algorithms assume that all problem-related data remain unchanged during the optimization process, which is not a realistic assumption. Recently, dynamic optimization problems (DOPs) grabbed remarkable attention from the research community. However, the literature still lacks clear guidelines on selecting the most appropriate bio-inspired algorithm under changing environments. Due to the availability of many design choices, the selection of a suitable bio-inspired metaheuristic algorithm becomes an immediate challenge. This study proposes an algorithm recommendation architecture using Q-learning and hyper-heuristic approaches to help decision-makers select the most suitable bio-inspired algorithm for a given problem. To this end, Artificial Bee Colony (ABC), Manta Ray Foraging Optimization (MRFO), Salp Swarm Algorithm (SSA), and Whale Optimization Algorithm (WOA) are employed as low-level optimizers so that the Q-learning and hyper-heuristic automatically select the optimizer in each cycle of the optimization process. The proposed algorithms are implemented in dynamic multidimensional knapsack problems, a natural extension of the well-known 0-1 knapsack problem. The performances of the recommender and standalone bio-inspired algorithms are evaluated through a comprehensive experimental analysis including appropriate statistical tests. Thus, the significant differences among the algorithms are revealed. The obtained results point out the efficiencies of the Q-learning-based algorithm recommender and MRFO in solving the dynamic multidimensional knapsack problem.