A new type of eye movement model based on recurrent neural networks for simulating the gaze behavior of human reading

Xiaoming Wang, Xinbo Zhao, Jinchang Ren

Research output: Contribution to journalArticle

4 Downloads (Pure)

Abstract

Traditional eye movement models are based on psychological assumptions and empirical data that are not able to simulate eye movement on previously unseen text data. To address this problem, a new type of eye movement model is presented and tested in this paper. In contrast to conventional psychology-based eye movement models, ours is based on a recurrent neural network (RNN) to generate a gaze point prediction sequence, by using the combination of convolutional neural networks (CNN), bidirectional long short-term memory networks (LSTM), and conditional random fields (CRF). The model uses the eye movement data of a reader reading some texts as training data to predict the eye movements of the same reader reading a previously unseen text. A theoretical analysis of the model is presented to show its excellent convergence performance. Experimental results are then presented to demonstrate that the proposed model can achieve similar prediction accuracy while requiring fewer features than current machine learning models.

Original languageEnglish
Article number8641074
Number of pages12
JournalComplexity
Volume2019
DOIs
Publication statusPublished - 24 Mar 2019

Fingerprint

Recurrent neural networks
Eye movements
Psychological
Learning model
Prediction
Psychology
Prediction accuracy
Theoretical analysis
Neural networks
Empirical data
Conditional random fields
Machine learning

Keywords

  • eye movement models
  • recurrent neural network (RNN)
  • gaze point prediction sequence
  • long short-term memory networks (LSTM)

Cite this

@article{9f3ee59a1e5a4c5db13d245245d9bd0a,
title = "A new type of eye movement model based on recurrent neural networks for simulating the gaze behavior of human reading",
abstract = "Traditional eye movement models are based on psychological assumptions and empirical data that are not able to simulate eye movement on previously unseen text data. To address this problem, a new type of eye movement model is presented and tested in this paper. In contrast to conventional psychology-based eye movement models, ours is based on a recurrent neural network (RNN) to generate a gaze point prediction sequence, by using the combination of convolutional neural networks (CNN), bidirectional long short-term memory networks (LSTM), and conditional random fields (CRF). The model uses the eye movement data of a reader reading some texts as training data to predict the eye movements of the same reader reading a previously unseen text. A theoretical analysis of the model is presented to show its excellent convergence performance. Experimental results are then presented to demonstrate that the proposed model can achieve similar prediction accuracy while requiring fewer features than current machine learning models.",
keywords = "eye movement models, recurrent neural network (RNN), gaze point prediction sequence, long short-term memory networks (LSTM)",
author = "Xiaoming Wang and Xinbo Zhao and Jinchang Ren",
year = "2019",
month = "3",
day = "24",
doi = "10.1155/2019/8641074",
language = "English",
volume = "2019",
journal = "Emergence",
issn = "1521-3250",

}

A new type of eye movement model based on recurrent neural networks for simulating the gaze behavior of human reading. / Wang, Xiaoming; Zhao, Xinbo; Ren, Jinchang.

In: Complexity, Vol. 2019, 8641074, 24.03.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A new type of eye movement model based on recurrent neural networks for simulating the gaze behavior of human reading

AU - Wang, Xiaoming

AU - Zhao, Xinbo

AU - Ren, Jinchang

PY - 2019/3/24

Y1 - 2019/3/24

N2 - Traditional eye movement models are based on psychological assumptions and empirical data that are not able to simulate eye movement on previously unseen text data. To address this problem, a new type of eye movement model is presented and tested in this paper. In contrast to conventional psychology-based eye movement models, ours is based on a recurrent neural network (RNN) to generate a gaze point prediction sequence, by using the combination of convolutional neural networks (CNN), bidirectional long short-term memory networks (LSTM), and conditional random fields (CRF). The model uses the eye movement data of a reader reading some texts as training data to predict the eye movements of the same reader reading a previously unseen text. A theoretical analysis of the model is presented to show its excellent convergence performance. Experimental results are then presented to demonstrate that the proposed model can achieve similar prediction accuracy while requiring fewer features than current machine learning models.

AB - Traditional eye movement models are based on psychological assumptions and empirical data that are not able to simulate eye movement on previously unseen text data. To address this problem, a new type of eye movement model is presented and tested in this paper. In contrast to conventional psychology-based eye movement models, ours is based on a recurrent neural network (RNN) to generate a gaze point prediction sequence, by using the combination of convolutional neural networks (CNN), bidirectional long short-term memory networks (LSTM), and conditional random fields (CRF). The model uses the eye movement data of a reader reading some texts as training data to predict the eye movements of the same reader reading a previously unseen text. A theoretical analysis of the model is presented to show its excellent convergence performance. Experimental results are then presented to demonstrate that the proposed model can achieve similar prediction accuracy while requiring fewer features than current machine learning models.

KW - eye movement models

KW - recurrent neural network (RNN)

KW - gaze point prediction sequence

KW - long short-term memory networks (LSTM)

UR - http://www.scopus.com/inward/record.url?scp=85064414256&partnerID=8YFLogxK

U2 - 10.1155/2019/8641074

DO - 10.1155/2019/8641074

M3 - Article

VL - 2019

JO - Emergence

JF - Emergence

SN - 1521-3250

M1 - 8641074

ER -