The Evaluation of Usability and Website Development using Cognitive Walkthrough, Performance Measurement, and System Usability Scale

I Gusti Putu Yada Giri; Luh Joni Erawati  Dewi; I Made Gede  Sunarya

doi:10.47709/cnahpc.v5i2.2511

Authors

I Gusti Putu Yada Giri Universitas Pendidikan Ganesha
Luh Joni Erawati Dewi Universitas Pendidikan Ganesha
I Made Gede Sunarya Universitas Pendidikan Ganesha

DOI:

10.47709/cnahpc.v5i2.2511

Keywords:

The Office of Social Affairs, usability evaluation, Cognitive Walkthrough, Incremental, Performance Measurement, System Usability Scale (SUS)

Dimension Badge Record

Abstract

The website of Social Affairs, Empowerment, and Child Protection in Bali is one of the information media used to fulfill one of the Service's requirements in implementing the Law on Public Information Disclosure. The initial observation results showed that there has never been a usability evaluation related to the website. In addition to this, according to an initial survey of 55 respondents from the general public and service information technology staff, the results obtained were (1) 69% of respondents stated that the website for the Office of Social Affairs, Women's Empowerment and Child Protection of the Province of Bali had an unattractive appearance, (2) as many as 67% of respondents stated that the website was still not good at placing and writing menus, (3) 67% of respondents stated that the website could not provide comfort when browsing, and (4) 95% of respondents stated that the display and contents of website content were still lacking and need to be developed. Based on the results of the interviews and surveys conducted, it is necessary to carry out an evaluation to find out more about the development of the website for the Bali Provincial Office of Social Affairs, Women's Empowerment and Child Protection. Therefore an evaluation was carried out on the website of Social Affairs, Women's Empowerment, and Child Protection of Bali using the Cognitive Walkthrough, Performance Measurement, and System Usability Scale (SUS) evaluation techniques. The research resulted in the following conclusions. (1) The website development for the Provincial Office of Social Affairs, Women's Empowerment, and Child Protection uses the Incremental method to produce a website with a more dynamic and modern appearance but still minimalist so that it does not interfere with the view of the website visitors and remains focused on content so that it is comfortable to view and read. (2) The performance measurement method evaluation results showed that the website is included in the effective and efficient category. The results showed that the working time for the beginner and the proficient group was short, the average for the proficient group was 17.4 seconds, and the beginner was 17.56 seconds. (3) Recommendations from the results of the usability evaluation of the Bali Provincial Office of Social Affairs, Women's Empowerment, and Child Protection using the Cognitive Walkthrough method are included in the successful category. Research showed that 100% of respondents completed the task scenario perfectly without any failures or errors. (4) Satisfaction of website users, when evaluated using the SUS method, gets an average score of 92.36 which is above the standard score of 68, so it can be concluded that respondents are very satisfied using the website for Social Affairs, Women's Empowerment and Child Protection of the Bali Province.

Downloads

Download data is not yet available.

Google Scholar Cite Analysis
Abstract viewed = 518 times

References

Arga Kusumah, M. A., Rokhmawati, R. I., & Amalia, F. (2019). Evaluasi Usability Pada Website E-commerce XYZ Dengan Menggunakan Metode Cognitive Walkthrough dan System Usability Scale (SUS). Pengembangan Teknologi Informasi Dan Ilmu Komputer, 3(5), 4340–4348.

Babich, N. (2017). Storyboarding in UX Design. UX Panet. https://uxplanet.org/storyboarding-in-ux-design-b9d2e18e5fab

Bligård, L. O., & Osvalder, A. L. (2013). Enhanced cognitive Walkthrough: Development of the cognitive walkthrough method to better predict, identify, and present usability problems. Advances in Human-Computer Interaction, 2013. https://doi.org/10.1155/2013/931698

Brooke, J. (2020a). SUS?: A Retrospective. January 2013.

Brooke, J. (2020b). SUS: A “Quick and Dirty” Usability Scale. Usability Evaluation In Industry, November 1995, 207–212. https://doi.org/10.1201/9781498710411-35

Cho, Hwayoung et al. 2019. “Eye-Tracking Retrospective Think-Aloud as a Novel Approach for a Usability Evaluation.” International Journal of Medical Informatics 129: 366–73.

Dusea, M.A ., Andriyanto, E., Ramadhan, D.W., & Saputra,M.A. (2015). Evalusi Usability Untuk Mengukur Penggunaan Website Event Organier. Seminar Informatika 2015,428-434.

Ginting, L. M., Sianturi, G., & Panjaitan, C. V. (2021). Perbandingan Metode Evaluasi Usability Antara Heuristic Evaluation dan Cognitive Walkthrough. Jurnal Manajemen Informatika (JAMIKA), 11(2), 146–157. https://doi.org/10.34010/jamika.v11i2.5480

Hwang, W., & Salvendy, G. (2010). Number of people required for usability evaluation: The 10±2 rule. Communications of the ACM, 53(5), 130–133. https://doi.org/10.1145/1735223.1735255

ISO, 9241. (1998). Iso 9241-11. 1998.

Lewis, C., Poison, P., Wharton, C., & Rieman, J. (2016). Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. Conference on Human Factors in Computing Systems - Proceedings, September 2014, 235–242. https://doi.org/10.1145/97243.97279

Naser, A. (2018). Perancangan user interface dan user experience halaman website program studi desain komunikasi visual universitas negeri padang. Dekave, 8(1)

Nielsen, J. (1993). Usability Engineering. Boston: Academic Press

Nielsen, J., Lewis, J., & Turner, C. (2006). Determining Usability Test Sample Size. International Encyclopedia of Ergonomics and Human Factors, Second Edition - 3 Volume Set, April.https://doi.org/10.1201/9780849375477.ch597

Raharjo, P., Kusuma, W. A., & Sukoco, H. (2016). Uji usability dengan metode cognitive walkthrough pada situs web perpustakaan. Jurnal Pustakawan Indonesia, 15(1), 19–27.

Sadnyana, M. A. W., ., I Gede Mahendra Darmawiguna, S.Kom, M. S., & ., I Made Ardwi Pradnyana, S.T., M. T. (2017). Evaluasi Usability Sistem Informasi Prakerin Pendidikan Teknik Informatika di Universitas Pendidikan Ganesha dengan Metode Usability Testing. Kumpulan Artikel Mahasiswa Pendidikan Teknik Informatika (KARMAPATI), 6(2), 309. https://doi.org/10.23887/karmapati.v6i2.11688

Sommerville. (2011). Software Engineering (International Computer Science Series).

Utami, N. W., Arthana, I. K. R., & Darmawiguna, I. G. M. (2020). Evaluasi Usability Pada E-Learning Universitas Pendidikan Ganesha Dengan Metode Usability Testing. Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), 9(1), 107. https://doi.org/10.23887/janapati.v9i1.23663

Wilson, C. (2014). Front-matter. User Interface Inspection Methods, i–iii. https://doi.org/10.1016/b978-0-12-410391-7.00007-5

Yuliyana, T., Arthana, I. K. R., & Agustini, K. (2019). Usability Testing pada Aplikasi POTWIS. JST (Jurnal Sains Dan Teknologi), 8(1), 12–22. https://doi.org/10.23887/jstundiksha.v8i1.12081

Zahra, S., & Santoso, H. B. (2020). An Indonesiani Adaptation of the E-Learning Usability Scale. Journal of Physics: Conference Series, 1566(1). https://doi.org/10.1088/1742-6596/1566/1/012051