Code or (not code): separating formal and natural language in CS education

Quintin Cutts, Richard Connor, Greg Michaelson, Peter Donaldson

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

Abstract

This paper argues that the "institutionalised understanding" of pseudo-code as a blend of formal and natural languages makes it an unsuitable choice for national assessment where the intention is to test program comprehension skills. It permits question-setters to inadvertently introduce a level of ambiguity and consequent confusion. This is not in keeping with either good assessment practice or an argument developed in the paper that CS education should be clearly fostering the skills needed for understanding formal, as distinct from natural, languages. The argument is backed up by an analysis of 49 questions drawn from the national school CS examinations of a single country, spanning a period of six years and two phases -- the first in which no formal pseudo-code was defined, the second in which a formal reference language, referred to as a "formally-defined pseudo-code", was provided for teachers and exam setters. The analysis demonstrates that in both phases, incorrect, confusing or ambiguous code was presented in questions. The paper concludes by recommending that the term reference language should be used in place of pseudo-code, and an appropriate formally-defined language specified, in national exam settings where a common language of assessment is required. This change of terms emphasises the characteristics required of a language to be used for assessment of program comprehension. The reference language used in the study is outlined. It was designed by the authors for human readability and also to make absolutely explicit the demarcation between formal and informal language, in such a way that automated checking can be carried out on programs written in the language. Formal specifications and a checker for the language are available.
LanguageEnglish
Title of host publicationWiPSCE 2014
Subtitle of host publicationProceedings of the 9th Workshop in Primary and Secondary Computing Education
Place of PublicationNew York
Pages20-28
Number of pages9
DOIs
Publication statusPublished - 5 Nov 2014
EventWorkshop in Primary and Secondary Computing Education - Berlin, Germany
Duration: 5 Nov 20147 Nov 2014
Conference number: 2014

Conference

ConferenceWorkshop in Primary and Secondary Computing Education
Abbreviated titleWiPSCE
CountryGermany
CityBerlin
Period5/11/147/11/14

Fingerprint

Education
language
education
comprehension
Formal specification
examination
teacher
school

Keywords

  • assessment
  • programming
  • pseudo-code
  • computer science education

Cite this

Cutts, Q., Connor, R., Michaelson, G., & Donaldson, P. (2014). Code or (not code): separating formal and natural language in CS education. In WiPSCE 2014: Proceedings of the 9th Workshop in Primary and Secondary Computing Education (pp. 20-28). New York. https://doi.org/10.1145/2670757.2670780
Cutts, Quintin ; Connor, Richard ; Michaelson, Greg ; Donaldson, Peter. / Code or (not code) : separating formal and natural language in CS education. WiPSCE 2014: Proceedings of the 9th Workshop in Primary and Secondary Computing Education. New York, 2014. pp. 20-28
@inproceedings{fa17757642a14c0da0663b1cf70ff09e,
title = "Code or (not code): separating formal and natural language in CS education",
abstract = "This paper argues that the {"}institutionalised understanding{"} of pseudo-code as a blend of formal and natural languages makes it an unsuitable choice for national assessment where the intention is to test program comprehension skills. It permits question-setters to inadvertently introduce a level of ambiguity and consequent confusion. This is not in keeping with either good assessment practice or an argument developed in the paper that CS education should be clearly fostering the skills needed for understanding formal, as distinct from natural, languages. The argument is backed up by an analysis of 49 questions drawn from the national school CS examinations of a single country, spanning a period of six years and two phases -- the first in which no formal pseudo-code was defined, the second in which a formal reference language, referred to as a {"}formally-defined pseudo-code{"}, was provided for teachers and exam setters. The analysis demonstrates that in both phases, incorrect, confusing or ambiguous code was presented in questions. The paper concludes by recommending that the term reference language should be used in place of pseudo-code, and an appropriate formally-defined language specified, in national exam settings where a common language of assessment is required. This change of terms emphasises the characteristics required of a language to be used for assessment of program comprehension. The reference language used in the study is outlined. It was designed by the authors for human readability and also to make absolutely explicit the demarcation between formal and informal language, in such a way that automated checking can be carried out on programs written in the language. Formal specifications and a checker for the language are available.",
keywords = "assessment, programming, pseudo-code, computer science education",
author = "Quintin Cutts and Richard Connor and Greg Michaelson and Peter Donaldson",
year = "2014",
month = "11",
day = "5",
doi = "10.1145/2670757.2670780",
language = "English",
isbn = "9781450332507",
pages = "20--28",
booktitle = "WiPSCE 2014",

}

Cutts, Q, Connor, R, Michaelson, G & Donaldson, P 2014, Code or (not code): separating formal and natural language in CS education. in WiPSCE 2014: Proceedings of the 9th Workshop in Primary and Secondary Computing Education. New York, pp. 20-28, Workshop in Primary and Secondary Computing Education, Berlin, Germany, 5/11/14. https://doi.org/10.1145/2670757.2670780

Code or (not code) : separating formal and natural language in CS education. / Cutts, Quintin; Connor, Richard; Michaelson, Greg; Donaldson, Peter.

WiPSCE 2014: Proceedings of the 9th Workshop in Primary and Secondary Computing Education. New York, 2014. p. 20-28.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - Code or (not code)

T2 - separating formal and natural language in CS education

AU - Cutts, Quintin

AU - Connor, Richard

AU - Michaelson, Greg

AU - Donaldson, Peter

PY - 2014/11/5

Y1 - 2014/11/5

N2 - This paper argues that the "institutionalised understanding" of pseudo-code as a blend of formal and natural languages makes it an unsuitable choice for national assessment where the intention is to test program comprehension skills. It permits question-setters to inadvertently introduce a level of ambiguity and consequent confusion. This is not in keeping with either good assessment practice or an argument developed in the paper that CS education should be clearly fostering the skills needed for understanding formal, as distinct from natural, languages. The argument is backed up by an analysis of 49 questions drawn from the national school CS examinations of a single country, spanning a period of six years and two phases -- the first in which no formal pseudo-code was defined, the second in which a formal reference language, referred to as a "formally-defined pseudo-code", was provided for teachers and exam setters. The analysis demonstrates that in both phases, incorrect, confusing or ambiguous code was presented in questions. The paper concludes by recommending that the term reference language should be used in place of pseudo-code, and an appropriate formally-defined language specified, in national exam settings where a common language of assessment is required. This change of terms emphasises the characteristics required of a language to be used for assessment of program comprehension. The reference language used in the study is outlined. It was designed by the authors for human readability and also to make absolutely explicit the demarcation between formal and informal language, in such a way that automated checking can be carried out on programs written in the language. Formal specifications and a checker for the language are available.

AB - This paper argues that the "institutionalised understanding" of pseudo-code as a blend of formal and natural languages makes it an unsuitable choice for national assessment where the intention is to test program comprehension skills. It permits question-setters to inadvertently introduce a level of ambiguity and consequent confusion. This is not in keeping with either good assessment practice or an argument developed in the paper that CS education should be clearly fostering the skills needed for understanding formal, as distinct from natural, languages. The argument is backed up by an analysis of 49 questions drawn from the national school CS examinations of a single country, spanning a period of six years and two phases -- the first in which no formal pseudo-code was defined, the second in which a formal reference language, referred to as a "formally-defined pseudo-code", was provided for teachers and exam setters. The analysis demonstrates that in both phases, incorrect, confusing or ambiguous code was presented in questions. The paper concludes by recommending that the term reference language should be used in place of pseudo-code, and an appropriate formally-defined language specified, in national exam settings where a common language of assessment is required. This change of terms emphasises the characteristics required of a language to be used for assessment of program comprehension. The reference language used in the study is outlined. It was designed by the authors for human readability and also to make absolutely explicit the demarcation between formal and informal language, in such a way that automated checking can be carried out on programs written in the language. Formal specifications and a checker for the language are available.

KW - assessment

KW - programming

KW - pseudo-code

KW - computer science education

UR - http://www.wipsce.org/

U2 - 10.1145/2670757.2670780

DO - 10.1145/2670757.2670780

M3 - Conference contribution book

SN - 9781450332507

SP - 20

EP - 28

BT - WiPSCE 2014

CY - New York

ER -

Cutts Q, Connor R, Michaelson G, Donaldson P. Code or (not code): separating formal and natural language in CS education. In WiPSCE 2014: Proceedings of the 9th Workshop in Primary and Secondary Computing Education. New York. 2014. p. 20-28 https://doi.org/10.1145/2670757.2670780