How do raters judge spoken vocabulary?

Research output: Contribution to journalArticle

Abstract

The aim of the study was to investigate how raters come to their decisions when judging spoken vocabulary. Segmental rating was introduced to quantify raters’ decision-making process. It is hoped that this simulated study brings fresh insight to future methodological considerations with spoken data. Twenty trainee raters assessed five Chinese students’ monologic texts on vocabulary in this study. Both segmental rating and overall rating were retrieved from the raters. Rasch analysis suggested variation between raters in their judgment of vocabulary, although consistency was found in general. Besides, there was a mismatch between candidates’ vocabulary scores and their lexical statistics. The raters’ decision-making process was generally cumulative.
LanguageEnglish
Number of pages14
JournalEnglish Language Teaching
Volume9
Issue number2
Early online date11 Jan 2016
DOIs
Publication statusPublished - 11 Jan 2016

Fingerprint

vocabulary
rating
decision-making process
mismatch
trainee
candidacy
statistics
Vocabulary
Raters
student
Rating
Decision-making Process

Keywords

  • raters
  • vocabulary
  • decision-making process
  • oral examinations

Cite this

@article{68c97f6ae3204813b7dd91872eecf3a9,
title = "How do raters judge spoken vocabulary?",
abstract = "The aim of the study was to investigate how raters come to their decisions when judging spoken vocabulary. Segmental rating was introduced to quantify raters’ decision-making process. It is hoped that this simulated study brings fresh insight to future methodological considerations with spoken data. Twenty trainee raters assessed five Chinese students’ monologic texts on vocabulary in this study. Both segmental rating and overall rating were retrieved from the raters. Rasch analysis suggested variation between raters in their judgment of vocabulary, although consistency was found in general. Besides, there was a mismatch between candidates’ vocabulary scores and their lexical statistics. The raters’ decision-making process was generally cumulative.",
keywords = "raters, vocabulary, decision-making process, oral examinations",
author = "Hui Li",
year = "2016",
month = "1",
day = "11",
doi = "10.5539/elt.v9n2p102",
language = "English",
volume = "9",
journal = "English Language Teaching",
issn = "1916-4742",
number = "2",

}

How do raters judge spoken vocabulary? / Li, Hui.

In: English Language Teaching, Vol. 9, No. 2, 11.01.2016.

Research output: Contribution to journalArticle

TY - JOUR

T1 - How do raters judge spoken vocabulary?

AU - Li, Hui

PY - 2016/1/11

Y1 - 2016/1/11

N2 - The aim of the study was to investigate how raters come to their decisions when judging spoken vocabulary. Segmental rating was introduced to quantify raters’ decision-making process. It is hoped that this simulated study brings fresh insight to future methodological considerations with spoken data. Twenty trainee raters assessed five Chinese students’ monologic texts on vocabulary in this study. Both segmental rating and overall rating were retrieved from the raters. Rasch analysis suggested variation between raters in their judgment of vocabulary, although consistency was found in general. Besides, there was a mismatch between candidates’ vocabulary scores and their lexical statistics. The raters’ decision-making process was generally cumulative.

AB - The aim of the study was to investigate how raters come to their decisions when judging spoken vocabulary. Segmental rating was introduced to quantify raters’ decision-making process. It is hoped that this simulated study brings fresh insight to future methodological considerations with spoken data. Twenty trainee raters assessed five Chinese students’ monologic texts on vocabulary in this study. Both segmental rating and overall rating were retrieved from the raters. Rasch analysis suggested variation between raters in their judgment of vocabulary, although consistency was found in general. Besides, there was a mismatch between candidates’ vocabulary scores and their lexical statistics. The raters’ decision-making process was generally cumulative.

KW - raters

KW - vocabulary

KW - decision-making process

KW - oral examinations

UR - http://www.ccsenet.org/journal/index.php/elt/article/view/56351/30161

U2 - 10.5539/elt.v9n2p102

DO - 10.5539/elt.v9n2p102

M3 - Article

VL - 9

JO - English Language Teaching

T2 - English Language Teaching

JF - English Language Teaching

SN - 1916-4742

IS - 2

ER -