Code or (not code): separating formal and natural language in CS education

Quintin Cutts, Richard Connor, Greg Michaelson, Peter Donaldson

Research output: Chapter in Book/Report/Conference proceedingConference contribution book


This paper argues that the "institutionalised understanding" of pseudo-code as a blend of formal and natural languages makes it an unsuitable choice for national assessment where the intention is to test program comprehension skills. It permits question-setters to inadvertently introduce a level of ambiguity and consequent confusion. This is not in keeping with either good assessment practice or an argument developed in the paper that CS education should be clearly fostering the skills needed for understanding formal, as distinct from natural, languages. The argument is backed up by an analysis of 49 questions drawn from the national school CS examinations of a single country, spanning a period of six years and two phases -- the first in which no formal pseudo-code was defined, the second in which a formal reference language, referred to as a "formally-defined pseudo-code", was provided for teachers and exam setters. The analysis demonstrates that in both phases, incorrect, confusing or ambiguous code was presented in questions. The paper concludes by recommending that the term reference language should be used in place of pseudo-code, and an appropriate formally-defined language specified, in national exam settings where a common language of assessment is required. This change of terms emphasises the characteristics required of a language to be used for assessment of program comprehension. The reference language used in the study is outlined. It was designed by the authors for human readability and also to make absolutely explicit the demarcation between formal and informal language, in such a way that automated checking can be carried out on programs written in the language. Formal specifications and a checker for the language are available.
Original languageEnglish
Title of host publicationWiPSCE 2014
Subtitle of host publicationProceedings of the 9th Workshop in Primary and Secondary Computing Education
Place of PublicationNew York
Number of pages9
Publication statusPublished - 5 Nov 2014
EventWorkshop in Primary and Secondary Computing Education - Berlin, Germany
Duration: 5 Nov 20147 Nov 2014
Conference number: 2014


ConferenceWorkshop in Primary and Secondary Computing Education
Abbreviated titleWiPSCE


  • assessment
  • programming
  • pseudo-code
  • computer science education


Dive into the research topics of 'Code or (not code): separating formal and natural language in CS education'. Together they form a unique fingerprint.

Cite this