IMPROVING DIGITAL LEARNING: EVALUATING THE U LEARN LMS WITH THE SYSTEM USABILITY SCALE

I Made Ardi Sudestra
Ni Wayan Eva Agustini
I Made Agus Oka Gunawan
Gede Indrawan
Musawer Hakimi


DOI: https://doi.org/10.29100/jipi.v9i4.6910

Abstract


The assessment of the usability of online learning systems is becoming more and more critical in order to ensure that users have the best possible experience. This study employed the System Usability Scale (SUS) to assess the usability of the learning management system at UNDIKNAS, using 68 students from various academic programs as participants. The employed methodology is administering a SUS questionnaire consisting of 10 items on a Likert scale from 1 to 5. The points are computed using the standard SUS methodology and multiplied by 2.5 to derive the final score. The analysis disclosed an average SUS score of 56.65, falling short of the academic usability benchmark 68. The score distribution visualization indicated that most respondents rated between 50 and 60, highlighting the need for system improvement. The SUS assessment of the learning management system at UNDIKNAS reveals that the system's usability necessitates improvement. Key recommendations include improving the user interface, optimizing navigation, and providing user guides to enhance the overall user experience

Keywords


Online Learning Systems; System Usability Scale; User Experience; User Interface

Full Text:

PDF

Article Metrics :

References


M. Furqon, P. Sinaga, L. Liliasari, and L. S. Riza, “The Impact of Learning Management System (LMS) Usage on Students,” TEM Journal, vol. 12, no. 2, pp. 1082–1089, 2023, doi: 10.18421/TEM122-54.

Y. Wang and Y. Liu, “Evaluation System of CG Art Communication Platform Based on User Experience,” IEEE Access, vol. 10, no. November, pp. 128742–128753, 2022, doi: 10.1109/ACCESS.2022.3227931.

M. A. Alterkait and M. Y. Alduaij, “Impact of Information Quality on Satisfaction with E-Learning Platforms: Moderating Role of Instructor and Learner Quality,” Sage Open, vol. 14, no. 1, pp. 1–13, 2024, doi: 10.1177/21582440241233400.

H. Cao et al., “Barriers and Enablers to the Implementation of Intelligent Guidance Systems for Patients in Chinese Tertiary Transfer Hospitals: Usability Evaluation,” IEEE Trans Eng Manag, vol. 70, no. 8, pp. 2634–2643, 2023, doi: 10.1109/TEM.2021.3066564.

N. Clark, M. Dabkowski, P. J. Driscoll, D. Kennedy, I. Kloo, and H. Shi, “Empirical Decision Rules for Improv-ing the Uncertainty Reporting of Small Sample System Usability Scale Scores,” Int J Hum Comput Interact, vol. 37, no. 13, pp. 1191–1206, 2021, doi: 10.1080/10447318.2020.1870831.

Y. W. Hsieh, M. T. Lee, C. C. Chen, F. L. Hsu, and C. Y. Wu, “Development and user experience of an innova-tive multi-mode stroke rehabilitation system for the arm and hand for patients with stroke,” Sci Rep, vol. 12, no. 1, pp. 1–9, 2022, doi: 10.1038/s41598-022-05314-8.

B. P. Knijnenburg, M. C. Willemsen, Z. Gantner, H. Soncu, and C. Newell, “Explaining the user experience of recommender systems,” User Model User-adapt Interact, vol. 22, no. 4–5, pp. 441–504, 2012, doi: 10.1007/s11257-011-9118-4.

A. Sütőová, K. Teplická, and M. Straka, “Application of the EFQM Model in the Education Institution for Driving Improvement of Processes towards Sustainability,” Sustainability (Switzerland), vol. 14, no. 13, 2022, doi: 10.3390/su14137711.

“Weir , C . J ., Heazell , A ., Whyte , S ., & Norman , J . E . ( 2020 ). Evaluating improvement interventions us-ing routine data to support a learning health system : research design , data access , analysis and reporting . BMJ Quality and Safety . https://doi.org/10.1136/bmjqs-2019-010068 University of Bristol - Explore Bristol Research,” 2020.

Kurniawan Malik Ibrahim, Ati Suci Dian Martha, and Dawam Dwi Jatmiko Suwawi, “Implementation of the Learn-er Centered Design method and the personality approach (Case Study: Redesigning The Interface mobile LMS Tel-U),” International Journal on Information and Communication Technology (IJoICT), vol. 9, no. 1, pp. 57–72, 2023, doi: 10.21108/ijoict.v9i1.723.

K. Orfanou, N. Tselios, and C. Katsanos, “Perceived usability evaluation of learning management systems: Empiri-cal evaluation of the system usability scale,” International Review of Research in Open and Distributed Learning, vol. 16, no. 2, pp. 227–246, 2015, doi: 10.19173/irrodl.v16i2.1955.

Z. Sharfina and H. B. Santoso, “An Indonesian adaptation of the System Usability Scale (SUS),” 2016 Internation-al Conference on Advanced Computer Science and Information Systems, ICACSIS 2016, pp. 145–148, 2017, doi: 10.1109/ICACSIS.2016.7872776.

M. Hammer, R. Göllner, K. Scheiter, B. Fauth, and K. Stürmer, “For whom do tablets make a difference? Examin-ing student profiles and perceptions of instruction with tablets,” Comput Educ, vol. 166, no. May 2020, 2021, doi: 10.1016/j.compedu.2021.104147.

J. Hasudungan, F. Arifianto, and H. T. Y. Achsan, “The Use of System Usability Scale as an Evaluation of Shopee PayLater,” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 9, no. 1, pp. 416–423, Mar. 2024, doi: 10.29100/jipi.v9i1.5411.

A. I. Fanada and N. Nurgiyatna, “WEBSITE-BASED IMPLEMENTATION IN INFORMATION SYSTEMS FOR SUPERVISORY MANAGEMENT PURPOSES,” JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika), vol. 9, no. 3, pp. 1395–1407, Aug. 2024, doi: 10.29100/jipi.v9i3.5341.

S. Vasanth and C. S. Sumathi, “Learning Management Systems through Moodle and Google Classroom for Educa-tion,” Adv Res, vol. 21, no. 10, pp. 32–37, 2020, doi: 10.9734/air/2020/v21i1030249.

S. Mehrolia, S. Alagarsamy, and M. Indhu Sabari, “Moderating effects of academic involvement in web-based learn-ing management system success: A multigroup analysis,” Heliyon, vol. 7, no. 5, p. e07000, 2021, doi: 10.1016/j.heliyon.2021.e07000.

J. Todd, A. Gepp, B. Richards, and B. J. Vanstone, “Improving mortality models in the ICU with high-frequency data,” Int J Med Inform, vol. 129, pp. 318–323, 2019, doi: 10.1016/j.ijmedinf.2019.07.002.

S. Lehong, J. van Biljon, and I. Sanders, “Usability Requirements for Learning Management Systems in Open Dis-tance Electronic Learning Environments: Considering Lecturers’ Views on Students’ Needs,” Int J Hum Comput In-teract, vol. 40, no. 3, pp. 567–583, 2024, doi: 10.1080/10447318.2022.2121205.

A. G. Pradnya Sidhawara, “Evaluation of UAJY Learning Management System’s Usability using USE Question-naire and Eye-tracking,” Indonesian Journal of Information Systems, vol. 4, no. 2, Feb. 2022, doi: 10.24002/ijis.v4i2.5273.