Student achievement in paper, computer/web and mobile based assessment

S.A. Nikou; A.A. Economides

Student achievement in paper, computer/web and mobile based assessment

S.A. Nikou, A.A. Economides

Education

Research output: Contribution to conference › Paper › peer-review

7 Citations (Scopus)

Abstract

Assessment can be delivered through paper and pencil (Paper Based Testing-PBT), through computers (Computer Based Testing-CBT) or mobile devices (Mobile Based Testing-MBT). Conclusions from previous research are not consistent regarding the performance of test taker related to the test delivery mode. Different test delivery modes may lead to differentiated student performance. The goal of this study is to bring new evidence regarding the impact of PBT, CBT and MBT on students' performance. The findings of the study indicate that there are statistically significant differences in test scores of first year undergraduate students of Economics in the subject of ICT among MBT, CBT and PBT in favor of MBT.

Original language	English
Pages	107-114
Number of pages	8
Publication status	Published - 2013
Event	6th Balkan Conference in Informatics, BCI-LOCAL 2013 - Thessaloniki, Greece Duration: 19 Sept 2013 → 21 Sept 2013

Conference

Conference	6th Balkan Conference in Informatics, BCI-LOCAL 2013
Country/Territory	Greece
City	Thessaloniki
Period	19/09/13 → 21/09/13

Keywords

computer-based assessment
mobile learning
mobile test
mobile-based assessment
paper-based assessment
brain computer interface
computer testing
economics
information science
mobile devices
paper
computer-based assessments
computer-based testing
statistically significant difference
evaluation
computers
student achievement
undergraduate students
students

Cite this

@conference{cfd23703914d41b2a3fb045a9ca27f4f,

title = "Student achievement in paper, computer/web and mobile based assessment",

abstract = "Assessment can be delivered through paper and pencil (Paper Based Testing-PBT), through computers (Computer Based Testing-CBT) or mobile devices (Mobile Based Testing-MBT). Conclusions from previous research are not consistent regarding the performance of test taker related to the test delivery mode. Different test delivery modes may lead to differentiated student performance. The goal of this study is to bring new evidence regarding the impact of PBT, CBT and MBT on students' performance. The findings of the study indicate that there are statistically significant differences in test scores of first year undergraduate students of Economics in the subject of ICT among MBT, CBT and PBT in favor of MBT.",

keywords = "computer-based assessment, mobile learning, mobile test, mobile-based assessment, paper-based assessment, brain computer interface, computer testing, economics, information science, mobile devices, paper, computer-based assessments, computer-based testing, statistically significant difference, evaluation, computers, student achievement, undergraduate students, students",

author = "S.A. Nikou and A.A. Economides",

note = "Akdemir, O., Aguz, A., Computer-based testing: An alternative for the assessment of turkish undergraduate students (2008) Computers & Education, 51, pp. 1198-1204; Bodmann, S.M., Robinson, D.H., Speed and performance differences among computer-based and paper-pencil tests (2004) Journal of Educational Computing Research, 31 (1), pp. 51-60; Bridgeman, B., Lennon, M.L., Jackenthal, A., Effects of screen size, screen resolution, and display rate on computer-based test performance (2003) Applied Measurement in Education, 16, pp. 191-205; Bugbee, A.C., Jr., Bernt, F.M., Testing by computer: Findings in six years of use 1982-1988 (1990) Journal of Research on Computing in Education, 23, pp. 87-101; Bunderson, C.V., Inouye, D.K., Olsen, J.B., The four generations of computerized educational measurement (1989) Educational Measurement, pp. 367-407. , R. L. Linn (Ed.) (3rd ed.) NY: American Council on Education - Macmillan; Campton, P., A comparative analysis of online and paper-based assessment methods: A university case study (2004) ACEC 2004, Conference Proceedings; Chao, K.J., Hung, I.C., Chen, N.S., On the design of online synchronous assessments in a synchronous cyber classroom (2011) Journal of Computer Assisted Learning, 28 (4), pp. 379-375; Cheung, W.S., Hew, K.F., A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings (2009) Australasian Journal of Educational Technology, 25 (2), pp. 153-183; Chin, C.H.L., Donn, J.S., Effects of computer-based tests on the achievement, anxiety, and attitudes of grade 10 science students (1991) Educational & Psychological Measurement, 51 (3), pp. 735-745; Choi, I., Kim, K., Boo, J., Comparability of a paper-based language test and a computer-based language test (2003) Language Testing, 20 (3), pp. 295-320; Choi, S.W., Tinkler, T., Evaluating comparability of paper-and-pencil and computer based assessment in a K-12 setting (2002) The Annual Meeting of the National Council on Measurement in Education, , New Orleans, LA: Paper presented at; Chu, H.C., Hwang, G.J., Tsai, C.C., Seng, T., Judy, C.R., A two-tier test approach to developing location-aware mobile learning system for natural science course (2010) Computers & Education, 55, pp. 1618-1627; Clariana, R., Wallace, P., Paper-based versus computer-based assessment: Key factors associated with the test mode effect (2002) British Journal of Educational Technology, 33 (5), pp. 593-602; Coniam, D., Evaluating computer-based and paper-based versions of an English-language listening test (2006) ReCALL, 18, pp. 193-211; Economides, A.A., Adaptive mobile learning (2006) Proceedings IEEE WMUTE - 4th International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, pp. 26-28. , Athens, Greece, November; Economides, A.A., Requirements of mobile learning applications (2008) International Journal of Innovation and Learning, 5 (5), pp. 457-479. , (23); Economides, A.A., Adaptive context-aware pervasive and ubiquitous learning (2009) International Journal of Technology Enhanced Learning, 1 (3), pp. 169-192; Economides, A.A., Grousopoulou, A., Mobiles in education: Students' usage, preferences and desires (2010) International Journal of Mobile Learning and Organisation, 4 (3), pp. 235-252; Gretes, J.A., Green, M., Improving undergraduate learning with computer-assisted assessment (2000) Journal of Research on Computing in Education, 33 (1), pp. 46-54; Huang, Y.M., Lin, Y.T., Cheng, S.C., An adaptive testing system for supporting versatile adaptive testing system for supporting versatile educational assessment (2009) Computers & Education, 52, pp. 53-56; Hwang, G.J., Chang, H.F., A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students (2011) Computers & Education, 56, pp. 1023-1031; Hwang, G.J., Wu, P.H., Zhuang, Y.Y., Huang, Y.M., Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students (2011) Interactive Learning Environments, pp. 1-17; Karadeniz, S., The impacts of paper, web and mobile based assessment on students' achievement and perception (2009) Scientific Research and Essays, 4 (10), pp. 984-991; Keng, L., McClarty, K.L., Davis, L.L., Itemlevel comparative analysis of online and paper administrations of the texas assessment of knowledge and skills (2006) Annual Meeting of the National Council on Measurement in Education, , Paper presented at the San Francisco, CA; Kim, D.H., Huynh, H., Comparability of computer and paper-and-pencil versions of algebra and biology assessments (2007) Journal of Technology, Learning, and Assessment, 6 (4); Kveton, P., Jelinek, M., Voboril, D., Klimusova, H., Computer-based tests: The impact of test design and problem of equivalency (2007) Computers in Human Behavior, 23, pp. 32-51; Leeson, H.V., The mode effect: A literature review of human and technological issues in computerized testing (2006) International Journal of Testing, 6 (1), pp. 1-24; Macedo-Rouet, M., Ney, M., Charles, S., Lallich-Boidin, G., Students' performance and satisfaction with web vs. Paper-based practice quizzes and lecture notes (2009) Computers & Education, 53, pp. 375-384; McFadden, A.C., Marsh, G.E., II, Price, B.J., (2001) Computer Testing in Education, Computers in the Schools, 18 (2-3), pp. 43-60; Mason, B.J., Patry, M., Berstein, D.J., An examination of the equivalence between non-adaptive computer-based and traditional testing (2001) Journal of Educational Computing Research, 24 (1), pp. 29-39; Masri, A.A., Using mobile phone for assessing university students in English literature in Jordan (2012) 2012 Orlando International Academic Conference, , Paper presented Orlando, FL, USA; Mayotte, S., Online assessment of problem solving skills (2010) Computers in Human Behavior, 26, pp. 1253-1258; Mazzeo, J., Harvey, A.I., The equivalence of scores from automated and conventional educational and psychological tests (1988) College Board Report No. 88-8, , NY: College Entrance Examination Board; Muhanna, W., The impact of using cell phone assessment on Jordanian university students' achievement in national education (2011) European Journal of Social Sciences, 20 (1); Noyes, J.M., Garland, J.K., Computer- vs. Paper-based tasks: Are they equivalent? (2008) Ergonomics, 51 (9), pp. 1352-1375; Paek, P., (2005) Recent Trends in Comparability Studies (PEM Research Report 05-05), , http://www.pearsonedmeasurement.com/downloads/research/RR_05_05.pdf, Available from; Pommerich, M., Developing computerized versions of paper-and-pencil tests: Mode effects for passage-based tests (2004) Journal of Technology, Learning, and Assessment, 2 (6); Pomplun, M., Frey, S., Becker, D.F., The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension (2002) Educational and Psychological Measurement, 62, pp. 337-354; Romero, C., Ventura, S., De Bra, P., Using mobile and web-based computerized tests to evaluate university students (2009) Computer Applications in Engineering Education, 17 (4), pp. 435-447; Russell, M., Haney, B., Testing writing on computers: An experiment comparing students performance on test conducted via computer and via paper-and-pencil (1997) Education Policy Analysis Archives, 5 (3), pp. 1-19; Russell, M., Testing on computers: A follow-up study comparing performance on computer and on paper (1999) Education Policy Analysis Archives, 7, p. 20. , http://epaa.asu.edu/epaa/v7n20, Available at; Schroeders, U., Wilhelm, O., Testing reasoning ability with handheld computers, notebooks, and paper and pencil (2010) European Journal of Psychological Assessment, 26 (4); Segall, N., Toni, L., Doolen, J., Porter, D., A usability comparison of PDA-based quizzes and paper-and-pencil quizzes (2005) Computers & Education, 45 (4), pp. 417-432; Shih, J.-L., Chuang, C.-W., Hwang, G.-J., An inquiry-based mobile learning approach to enhancing social science learning effectiveness (2010) Educational Technology & Society, 13 (4), pp. 50-62; Struyven, K., Dochy, F., Janssens, S., Students' perceptions about evaluation and assessment in higher education: A review (2005) Assessment & Evaluation in Higher Education, 30 (4), pp. 331-347; The usability of personal digital assistants (PDAs) for assessment of practical performance (2006) Medical Education, 40 (9), pp. 855-861; Triantafillou, E., Georgiadou, E., Economides, A.A., CAT-MD: Computerized adaptive testing on mobile devices (2008) International Journal of Web-Based Learning and Teaching Technologies, 3 (1), pp. 13-20. , (extended versions of the best papers presented at m-ICTE2006 Conference); Wang, S., Jiao, H., Young, M., Brooks, T., Olson, J., Comparability of computer-based and paper-and-pencil testing in K-12 reading assessments (2008) Educational and Psychological Measurement, 68 (1), pp. 5-24; Way, W.D., Davis, L.L., Fitzpatrick, S., Score comparability of online and paper administrations of texas assessment of knowledge and skills (2006) The Annual Meeting of the National Council on Measurement in Education, , Paper presented at San Francisco, CA; Wilson, K., Boyd, C., Chen, L., Jamal, S., Improving student performance in a first-year geography course: Examining the importance of computer-assisted formative assessment (2011) Computers & Education, 57 (2), pp. 1493-1500; Wu, J., Zhang, Y., Examining potentialities of handheld technology in students' academic attainments (2010) Educational Media International, 47 (1), pp. 57-67; 6th Balkan Conference in Informatics, BCI-LOCAL 2013 ; Conference date: 19-09-2013 Through 21-09-2013",

year = "2013",

language = "English",

pages = "107--114",

}

TY - CONF

T1 - Student achievement in paper, computer/web and mobile based assessment

AU - Nikou, S.A.

AU - Economides, A.A.

N1 - Akdemir, O., Aguz, A., Computer-based testing: An alternative for the assessment of turkish undergraduate students (2008) Computers & Education, 51, pp. 1198-1204; Bodmann, S.M., Robinson, D.H., Speed and performance differences among computer-based and paper-pencil tests (2004) Journal of Educational Computing Research, 31 (1), pp. 51-60; Bridgeman, B., Lennon, M.L., Jackenthal, A., Effects of screen size, screen resolution, and display rate on computer-based test performance (2003) Applied Measurement in Education, 16, pp. 191-205; Bugbee, A.C., Jr., Bernt, F.M., Testing by computer: Findings in six years of use 1982-1988 (1990) Journal of Research on Computing in Education, 23, pp. 87-101; Bunderson, C.V., Inouye, D.K., Olsen, J.B., The four generations of computerized educational measurement (1989) Educational Measurement, pp. 367-407. , R. L. Linn (Ed.) (3rd ed.) NY: American Council on Education - Macmillan; Campton, P., A comparative analysis of online and paper-based assessment methods: A university case study (2004) ACEC 2004, Conference Proceedings; Chao, K.J., Hung, I.C., Chen, N.S., On the design of online synchronous assessments in a synchronous cyber classroom (2011) Journal of Computer Assisted Learning, 28 (4), pp. 379-375; Cheung, W.S., Hew, K.F., A review of research methodologies used in studies on mobile handheld devices in K-12 and higher education settings (2009) Australasian Journal of Educational Technology, 25 (2), pp. 153-183; Chin, C.H.L., Donn, J.S., Effects of computer-based tests on the achievement, anxiety, and attitudes of grade 10 science students (1991) Educational & Psychological Measurement, 51 (3), pp. 735-745; Choi, I., Kim, K., Boo, J., Comparability of a paper-based language test and a computer-based language test (2003) Language Testing, 20 (3), pp. 295-320; Choi, S.W., Tinkler, T., Evaluating comparability of paper-and-pencil and computer based assessment in a K-12 setting (2002) The Annual Meeting of the National Council on Measurement in Education, , New Orleans, LA: Paper presented at; Chu, H.C., Hwang, G.J., Tsai, C.C., Seng, T., Judy, C.R., A two-tier test approach to developing location-aware mobile learning system for natural science course (2010) Computers & Education, 55, pp. 1618-1627; Clariana, R., Wallace, P., Paper-based versus computer-based assessment: Key factors associated with the test mode effect (2002) British Journal of Educational Technology, 33 (5), pp. 593-602; Coniam, D., Evaluating computer-based and paper-based versions of an English-language listening test (2006) ReCALL, 18, pp. 193-211; Economides, A.A., Adaptive mobile learning (2006) Proceedings IEEE WMUTE - 4th International Workshop on Wireless, Mobile and Ubiquitous Technologies in Education, pp. 26-28. , Athens, Greece, November; Economides, A.A., Requirements of mobile learning applications (2008) International Journal of Innovation and Learning, 5 (5), pp. 457-479. , (23); Economides, A.A., Adaptive context-aware pervasive and ubiquitous learning (2009) International Journal of Technology Enhanced Learning, 1 (3), pp. 169-192; Economides, A.A., Grousopoulou, A., Mobiles in education: Students' usage, preferences and desires (2010) International Journal of Mobile Learning and Organisation, 4 (3), pp. 235-252; Gretes, J.A., Green, M., Improving undergraduate learning with computer-assisted assessment (2000) Journal of Research on Computing in Education, 33 (1), pp. 46-54; Huang, Y.M., Lin, Y.T., Cheng, S.C., An adaptive testing system for supporting versatile adaptive testing system for supporting versatile educational assessment (2009) Computers & Education, 52, pp. 53-56; Hwang, G.J., Chang, H.F., A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students (2011) Computers & Education, 56, pp. 1023-1031; Hwang, G.J., Wu, P.H., Zhuang, Y.Y., Huang, Y.M., Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students (2011) Interactive Learning Environments, pp. 1-17; Karadeniz, S., The impacts of paper, web and mobile based assessment on students' achievement and perception (2009) Scientific Research and Essays, 4 (10), pp. 984-991; Keng, L., McClarty, K.L., Davis, L.L., Itemlevel comparative analysis of online and paper administrations of the texas assessment of knowledge and skills (2006) Annual Meeting of the National Council on Measurement in Education, , Paper presented at the San Francisco, CA; Kim, D.H., Huynh, H., Comparability of computer and paper-and-pencil versions of algebra and biology assessments (2007) Journal of Technology, Learning, and Assessment, 6 (4); Kveton, P., Jelinek, M., Voboril, D., Klimusova, H., Computer-based tests: The impact of test design and problem of equivalency (2007) Computers in Human Behavior, 23, pp. 32-51; Leeson, H.V., The mode effect: A literature review of human and technological issues in computerized testing (2006) International Journal of Testing, 6 (1), pp. 1-24; Macedo-Rouet, M., Ney, M., Charles, S., Lallich-Boidin, G., Students' performance and satisfaction with web vs. Paper-based practice quizzes and lecture notes (2009) Computers & Education, 53, pp. 375-384; McFadden, A.C., Marsh, G.E., II, Price, B.J., (2001) Computer Testing in Education, Computers in the Schools, 18 (2-3), pp. 43-60; Mason, B.J., Patry, M., Berstein, D.J., An examination of the equivalence between non-adaptive computer-based and traditional testing (2001) Journal of Educational Computing Research, 24 (1), pp. 29-39; Masri, A.A., Using mobile phone for assessing university students in English literature in Jordan (2012) 2012 Orlando International Academic Conference, , Paper presented Orlando, FL, USA; Mayotte, S., Online assessment of problem solving skills (2010) Computers in Human Behavior, 26, pp. 1253-1258; Mazzeo, J., Harvey, A.I., The equivalence of scores from automated and conventional educational and psychological tests (1988) College Board Report No. 88-8, , NY: College Entrance Examination Board; Muhanna, W., The impact of using cell phone assessment on Jordanian university students' achievement in national education (2011) European Journal of Social Sciences, 20 (1); Noyes, J.M., Garland, J.K., Computer- vs. Paper-based tasks: Are they equivalent? (2008) Ergonomics, 51 (9), pp. 1352-1375; Paek, P., (2005) Recent Trends in Comparability Studies (PEM Research Report 05-05), , http://www.pearsonedmeasurement.com/downloads/research/RR_05_05.pdf, Available from; Pommerich, M., Developing computerized versions of paper-and-pencil tests: Mode effects for passage-based tests (2004) Journal of Technology, Learning, and Assessment, 2 (6); Pomplun, M., Frey, S., Becker, D.F., The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension (2002) Educational and Psychological Measurement, 62, pp. 337-354; Romero, C., Ventura, S., De Bra, P., Using mobile and web-based computerized tests to evaluate university students (2009) Computer Applications in Engineering Education, 17 (4), pp. 435-447; Russell, M., Haney, B., Testing writing on computers: An experiment comparing students performance on test conducted via computer and via paper-and-pencil (1997) Education Policy Analysis Archives, 5 (3), pp. 1-19; Russell, M., Testing on computers: A follow-up study comparing performance on computer and on paper (1999) Education Policy Analysis Archives, 7, p. 20. , http://epaa.asu.edu/epaa/v7n20, Available at; Schroeders, U., Wilhelm, O., Testing reasoning ability with handheld computers, notebooks, and paper and pencil (2010) European Journal of Psychological Assessment, 26 (4); Segall, N., Toni, L., Doolen, J., Porter, D., A usability comparison of PDA-based quizzes and paper-and-pencil quizzes (2005) Computers & Education, 45 (4), pp. 417-432; Shih, J.-L., Chuang, C.-W., Hwang, G.-J., An inquiry-based mobile learning approach to enhancing social science learning effectiveness (2010) Educational Technology & Society, 13 (4), pp. 50-62; Struyven, K., Dochy, F., Janssens, S., Students' perceptions about evaluation and assessment in higher education: A review (2005) Assessment & Evaluation in Higher Education, 30 (4), pp. 331-347; The usability of personal digital assistants (PDAs) for assessment of practical performance (2006) Medical Education, 40 (9), pp. 855-861; Triantafillou, E., Georgiadou, E., Economides, A.A., CAT-MD: Computerized adaptive testing on mobile devices (2008) International Journal of Web-Based Learning and Teaching Technologies, 3 (1), pp. 13-20. , (extended versions of the best papers presented at m-ICTE2006 Conference); Wang, S., Jiao, H., Young, M., Brooks, T., Olson, J., Comparability of computer-based and paper-and-pencil testing in K-12 reading assessments (2008) Educational and Psychological Measurement, 68 (1), pp. 5-24; Way, W.D., Davis, L.L., Fitzpatrick, S., Score comparability of online and paper administrations of texas assessment of knowledge and skills (2006) The Annual Meeting of the National Council on Measurement in Education, , Paper presented at San Francisco, CA; Wilson, K., Boyd, C., Chen, L., Jamal, S., Improving student performance in a first-year geography course: Examining the importance of computer-assisted formative assessment (2011) Computers & Education, 57 (2), pp. 1493-1500; Wu, J., Zhang, Y., Examining potentialities of handheld technology in students' academic attainments (2010) Educational Media International, 47 (1), pp. 57-67

PY - 2013

Y1 - 2013

N2 - Assessment can be delivered through paper and pencil (Paper Based Testing-PBT), through computers (Computer Based Testing-CBT) or mobile devices (Mobile Based Testing-MBT). Conclusions from previous research are not consistent regarding the performance of test taker related to the test delivery mode. Different test delivery modes may lead to differentiated student performance. The goal of this study is to bring new evidence regarding the impact of PBT, CBT and MBT on students' performance. The findings of the study indicate that there are statistically significant differences in test scores of first year undergraduate students of Economics in the subject of ICT among MBT, CBT and PBT in favor of MBT.

AB - Assessment can be delivered through paper and pencil (Paper Based Testing-PBT), through computers (Computer Based Testing-CBT) or mobile devices (Mobile Based Testing-MBT). Conclusions from previous research are not consistent regarding the performance of test taker related to the test delivery mode. Different test delivery modes may lead to differentiated student performance. The goal of this study is to bring new evidence regarding the impact of PBT, CBT and MBT on students' performance. The findings of the study indicate that there are statistically significant differences in test scores of first year undergraduate students of Economics in the subject of ICT among MBT, CBT and PBT in favor of MBT.

KW - computer-based assessment

KW - mobile learning

KW - mobile test

KW - mobile-based assessment

KW - paper-based assessment

KW - brain computer interface

KW - computer testing

KW - economics

KW - information science

KW - mobile devices

KW - paper

KW - computer-based assessments

KW - computer-based testing

KW - statistically significant difference

KW - evaluation

KW - computers

KW - student achievement

KW - undergraduate students

KW - students

UR - https://www.scopus.com/inward/record.uri?eid=2-s2.0-84924768205&partnerID=40&md5=199010a409c5a9f260a8e492c5da5ffb

M3 - Paper

SP - 107

EP - 114

T2 - 6th Balkan Conference in Informatics, BCI-LOCAL 2013

Y2 - 19 September 2013 through 21 September 2013

ER -

Student achievement in paper, computer/web and mobile based assessment

Abstract

Conference

Keywords

Other files and links

Fingerprint

Cite this