Perceived usability evaluation of learning management systems via axiomatic design with a real life application


BAYKASOĞLU A., FELEKOĞLU B., Unal C.

KYBERNETES, vol.53, no.1, pp.83-101, 2024 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 53 Issue: 1
  • Publication Date: 2024
  • Doi Number: 10.1108/k-07-2022-1024
  • Journal Name: KYBERNETES
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Aerospace Database, Applied Science & Technology Source, BIOSIS, Communication Abstracts, Compendex, Computer & Applied Sciences, INSPEC, Metadex, zbMATH, Civil Engineering Abstracts
  • Page Numbers: pp.83-101
  • Keywords: Learning management systems, Usability evaluation, Axiomatic design, Distance learning, Multiple criteria decision making, SATISFACTION
  • Dokuz Eylül University Affiliated: Yes

Abstract

Purpose Usage of learning management systems (LMSs) has become widespread with the disruption of face-to-face educations after the COVID-19 pandemic. There are several software products, usually named as LMS to enable and support distance education. However, selection of a suitable LMS is a complex multiple criteria decision making (MCDM) problem that requires consideration of many criteria and inputs from different parties like students, academicians, education managers, etc. Usability evaluation of LMS is one of the critical steps in deciding which LMS system to be adapted. There are several studies related to usability evaluation of LMS in the literature, but utilization of MCDM methods and real life case studies are very rare. Based on this motivation, perceived usability evaluation of SAKAI-LMS that is in use at an academic department is performed by employing axiomatic design procedure (ADP). This paper aims to discuss the aforementioned issues. Design/methodology/approach ADP is considered as a suitable MCDM method for perceived usability evaluation as it allows an easy approach to data fusion and setting performance targets for decision makers. A questionnaire is developed to collect data from three types of system users about predetermined usability criteria and their importance. After detailed statistical analyses and weighting criteria via analytical hierarch process (AHP), ADP is carried out to evaluate usability of the LMS. Findings It is found that the proposed ADP based approach is easy to apply in practical circumstances and able to quantify perceived usability of the LMSs. Research limitations/implications The proposed approach provides an easy and practical evaluation of perceived usability of the LMSs for decision makers who are responsible for the implementation of LMSs. The developed novel and practical MCDM-based perceived usability approach for LMS in this study has been verified through a real life case study at an academic department. Perceived usability results, therefore, reflects only the views of this focus group and are not generalizable. Originality/value First time in the literature, a comprehensive ADP based MCDM approach is proposed based on the analyses of the related literature and information gathered from the system users.