A comparative study between a computer-based and a mobile-based assessment: usability and user experience

Stavros A. Nikou, Anastasios A. Economides

Research output: Contribution to journalArticle

Abstract

Purpose
The purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course.

Design/methodology/approach
The study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale.

Findings
Attractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions.

Practical implications
The usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers.

Originality/value
The study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.

LanguageEnglish
Number of pages10
JournalInteractive Technology and Smart Education
Early online date5 Aug 2019
DOIs
Publication statusE-pub ahead of print - 5 Aug 2019

Fingerprint

Mobile devices
Personal computers
Students
Education
experience
Group
student
education
educator
efficiency
questionnaire
university
methodology
evidence

Keywords

  • user experience
  • usability
  • mobile learning
  • Higher Education
  • computer-based assessments
  • mobile-based assessment

Cite this

@article{89e0b9b59b34498bb7b0d9f61d321754,
title = "A comparative study between a computer-based and a mobile-based assessment: usability and user experience",
abstract = "PurposeThe purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course.Design/methodology/approachThe study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale.FindingsAttractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions.Practical implicationsThe usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers.Originality/valueThe study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.",
keywords = "user experience, usability, mobile learning, Higher Education, computer-based assessments, mobile-based assessment",
author = "Nikou, {Stavros A.} and Economides, {Anastasios A.}",
year = "2019",
month = "8",
day = "5",
doi = "10.1108/ITSE-01-2019-0003",
language = "English",
journal = "Interactive Technology and Smart Education",
issn = "1741-5659",
publisher = "Emerald Publishing Limited",

}

TY - JOUR

T1 - A comparative study between a computer-based and a mobile-based assessment

T2 - Interactive Technology and Smart Education

AU - Nikou, Stavros A.

AU - Economides, Anastasios A.

PY - 2019/8/5

Y1 - 2019/8/5

N2 - PurposeThe purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course.Design/methodology/approachThe study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale.FindingsAttractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions.Practical implicationsThe usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers.Originality/valueThe study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.

AB - PurposeThe purpose of this study is to compare the overall usability and user experience of desktop computers and mobile-devices when used in a summative assessment in the context of a higher education course.Design/methodology/approachThe study follows a between-groups design. The participants were 110 first-year undergraduate students from a European university. Students in the experimental group participated in the assessment using mobile devices, whereas students in the control group participated using desktop computers. After the assessment, students self-reported their experiences with computer-based assessment (CBA) and mobile-based assessment (MBA), respectively. The instruments used were the user experience questionnaire and the system usability scale.FindingsAttractiveness and novelty were reported significantly higher in the experimental group (MBA), while no significant differences were found between the two groups in terms of efficiency, perspicuity, dependability and stimulation. The overall score for the system usability was not found to differ between the two conditions.Practical implicationsThe usability and user experience issues discussed in this study can inform educators and policymakers about the potential of using mobile devices in online assessment practices, as an alternative to desktop computers.Originality/valueThe study is novel, in that it provides quantitative evidence for the usability and user experience of both desktop computers and mobile devices when used in a summative assessment in the context of a higher education course. Study findings can contribute towards the interchangeable usage of desktop computers and mobile devices in assessment practices in higher education.

KW - user experience

KW - usability

KW - mobile learning

KW - Higher Education

KW - computer-based assessments

KW - mobile-based assessment

U2 - 10.1108/ITSE-01-2019-0003

DO - 10.1108/ITSE-01-2019-0003

M3 - Article

JO - Interactive Technology and Smart Education

JF - Interactive Technology and Smart Education

SN - 1741-5659

ER -