Picture this: a review of research relating to narrative processing by moving image versus language

Elspeth Jajdelska, Miranda Anderson, Christopher Butler, Nigel Fabb, Elizabeth Finnigan, Ian Garwood, Stephen Kelly, Wendy Kirk, Karen Kukkonen, Sinead Mullally, Stephan Schwan

Research output: Contribution to journalArticle

Abstract

Reading fiction for pleasure is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading or hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image or text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative. We note that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery, and personal memory (Xu et al., 2005). Attempts to compare the two by creating equivalent stimuli and task demands face a number of challenges. We discuss the difficulties of such comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media and propose internal scene construction and working memory as foci for future research. Although many of the sources we draw on concentrate on English-speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew, and Japanese in their respective countries, and studies of a remote Turkish mountain community.
LanguageEnglish
Article number1161
Number of pages15
JournalFrontiers in Psychology
Volume10
DOIs
Publication statusPublished - 26 Jun 2019

Fingerprint

Reading
Language
Short-Term Memory
Research
Pleasure
Imagery (Psychotherapy)
Developed Countries
Cognition
Hearing

Keywords

  • narrative
  • media
  • reading
  • film
  • fiction
  • comprehension
  • literature
  • cognitive humanities

Cite this

Jajdelska, Elspeth ; Anderson, Miranda ; Butler, Christopher ; Fabb, Nigel ; Finnigan, Elizabeth ; Garwood, Ian ; Kelly, Stephen ; Kirk, Wendy ; Kukkonen, Karen ; Mullally, Sinead ; Schwan, Stephan. / Picture this : a review of research relating to narrative processing by moving image versus language. In: Frontiers in Psychology . 2019 ; Vol. 10.
@article{dd8cf09037d04af8afb6b941caa07b43,
title = "Picture this: a review of research relating to narrative processing by moving image versus language",
abstract = "Reading fiction for pleasure is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading or hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image or text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative. We note that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery, and personal memory (Xu et al., 2005). Attempts to compare the two by creating equivalent stimuli and task demands face a number of challenges. We discuss the difficulties of such comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media and propose internal scene construction and working memory as foci for future research. Although many of the sources we draw on concentrate on English-speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew, and Japanese in their respective countries, and studies of a remote Turkish mountain community.",
keywords = "narrative, media, reading, film, fiction, comprehension, literature, cognitive humanities",
author = "Elspeth Jajdelska and Miranda Anderson and Christopher Butler and Nigel Fabb and Elizabeth Finnigan and Ian Garwood and Stephen Kelly and Wendy Kirk and Karen Kukkonen and Sinead Mullally and Stephan Schwan",
year = "2019",
month = "6",
day = "26",
doi = "10.3389/fpsyg.2019.01161",
language = "English",
volume = "10",
journal = "Frontiers in Psychology",
issn = "1664-1078",

}

Picture this : a review of research relating to narrative processing by moving image versus language. / Jajdelska, Elspeth; Anderson, Miranda; Butler, Christopher; Fabb, Nigel; Finnigan, Elizabeth; Garwood, Ian ; Kelly, Stephen; Kirk, Wendy; Kukkonen, Karen; Mullally, Sinead; Schwan, Stephan.

In: Frontiers in Psychology , Vol. 10, 1161, 26.06.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Picture this

T2 - Frontiers in Psychology

AU - Jajdelska, Elspeth

AU - Anderson, Miranda

AU - Butler, Christopher

AU - Fabb, Nigel

AU - Finnigan, Elizabeth

AU - Garwood, Ian

AU - Kelly, Stephen

AU - Kirk, Wendy

AU - Kukkonen, Karen

AU - Mullally, Sinead

AU - Schwan, Stephan

PY - 2019/6/26

Y1 - 2019/6/26

N2 - Reading fiction for pleasure is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading or hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image or text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative. We note that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery, and personal memory (Xu et al., 2005). Attempts to compare the two by creating equivalent stimuli and task demands face a number of challenges. We discuss the difficulties of such comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media and propose internal scene construction and working memory as foci for future research. Although many of the sources we draw on concentrate on English-speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew, and Japanese in their respective countries, and studies of a remote Turkish mountain community.

AB - Reading fiction for pleasure is robustly correlated with improved cognitive attainment and other benefits. It is also in decline among young people in developed nations, in part because of competition from moving image fiction. We review existing research on the differences between reading or hearing verbal fiction and watching moving image fiction, as well as looking more broadly at research on image or text interactions and visual versus verbal processing. We conclude that verbal narrative generates more diverse responses than moving image narrative. We note that reading and viewing narrative are different tasks, with different cognitive loads. Viewing moving image narrative mostly involves visual processing with some working memory engagement, whereas reading narrative involves verbal processing, visual imagery, and personal memory (Xu et al., 2005). Attempts to compare the two by creating equivalent stimuli and task demands face a number of challenges. We discuss the difficulties of such comparative approaches. We then investigate the possibility of identifying lower level processing mechanisms that might distinguish cognition of the two media and propose internal scene construction and working memory as foci for future research. Although many of the sources we draw on concentrate on English-speaking participants in European or North American settings, we also cover material relating to speakers of Dutch, German, Hebrew, and Japanese in their respective countries, and studies of a remote Turkish mountain community.

KW - narrative

KW - media

KW - reading

KW - film

KW - fiction

KW - comprehension

KW - literature

KW - cognitive humanities

U2 - 10.3389/fpsyg.2019.01161

DO - 10.3389/fpsyg.2019.01161

M3 - Article

VL - 10

JO - Frontiers in Psychology

JF - Frontiers in Psychology

SN - 1664-1078

M1 - 1161

ER -