Investigating how conversational search agents affect user's behaviour, performance and search experience

Mateusz Dubiel, Martin Halvey, Leif Azzopardi, Sylvain Daronnat

Research output: Contribution to conference › Paper

Abstract

Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. In this paper, we explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. We conducted an interactive study comparing a simulated conversational search system against a slotbased voice search system. We hypothesise that the conversational system, with its ability to preserve and maintain conversational state, will lead to greater satisfaction. Our results indicate that participants prefer the conversational system over current voice based system, the conversational search system leads to significantly faster search task completion times, and significantly greater usability.
LanguageEnglish
Number of pages8
StatePublished - 12 Jul 2018
EventThe Second International Workshop on Conversational Approaches to Information Retrieval - Ann Arbor, United States
Duration: 12 Jul 2018 → 12 Jul 2018

Workshop

WorkshopThe Second International Workshop on Conversational Approaches to Information Retrieval
Abbreviated titleCAIR
CountryUnited States
CityAnn Arbor
Period12/07/18 → 12/07/18

Keywords

  • conversational search
  • dialogue-systems
  • voice interfaces

Cite this

Dubiel, M., Halvey, M., Azzopardi, L., & Daronnat, S. (2018). Investigating how conversational search agents affect user's behaviour, performance and search experience. Paper presented at The Second International Workshop on Conversational Approaches to Information Retrieval, Ann Arbor, United States.
Dubiel, Mateusz ; Halvey, Martin ; Azzopardi, Leif ; Daronnat, Sylvain. / Investigating how conversational search agents affect user's behaviour, performance and search experience. Paper presented at The Second International Workshop on Conversational Approaches to Information Retrieval, Ann Arbor, United States.8 p.
@conference{467ab748ad4446b78ba5537e1bee80df,
title = "Investigating how conversational search agents affect user's behaviour, performance and search experience",
abstract = "Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. In this paper, we explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. We conducted an interactive study comparing a simulated conversational search system against a slotbased voice search system. We hypothesise that the conversational system, with its ability to preserve and maintain conversational state, will lead to greater satisfaction. Our results indicate that participants prefer the conversational system over current voice based system, the conversational search system leads to significantly faster search task completion times, and significantly greater usability.",
keywords = "conversational search, dialogue-systems, voice interfaces",
author = "Mateusz Dubiel and Martin Halvey and Leif Azzopardi and Sylvain Daronnat",
year = "2018",
month = "7",
day = "12",
language = "English",
note = "The Second International Workshop on Conversational Approaches to Information Retrieval, CAIR ; Conference date: 12-07-2018 Through 12-07-2018",

}

Dubiel, M, Halvey, M, Azzopardi, L & Daronnat, S 2018, 'Investigating how conversational search agents affect user's behaviour, performance and search experience' Paper presented at The Second International Workshop on Conversational Approaches to Information Retrieval, Ann Arbor, United States, 12/07/18 - 12/07/18, .

Investigating how conversational search agents affect user's behaviour, performance and search experience. / Dubiel, Mateusz; Halvey, Martin; Azzopardi, Leif; Daronnat, Sylvain.

2018. Paper presented at The Second International Workshop on Conversational Approaches to Information Retrieval, Ann Arbor, United States.

Research output: Contribution to conference › Paper

TY - CONF

T1 - Investigating how conversational search agents affect user's behaviour, performance and search experience

AU - Dubiel,Mateusz

AU - Halvey,Martin

AU - Azzopardi,Leif

AU - Daronnat,Sylvain

PY - 2018/7/12

Y1 - 2018/7/12

N2 - Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. In this paper, we explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. We conducted an interactive study comparing a simulated conversational search system against a slotbased voice search system. We hypothesise that the conversational system, with its ability to preserve and maintain conversational state, will lead to greater satisfaction. Our results indicate that participants prefer the conversational system over current voice based system, the conversational search system leads to significantly faster search task completion times, and significantly greater usability.

AB - Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. In this paper, we explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. We conducted an interactive study comparing a simulated conversational search system against a slotbased voice search system. We hypothesise that the conversational system, with its ability to preserve and maintain conversational state, will lead to greater satisfaction. Our results indicate that participants prefer the conversational system over current voice based system, the conversational search system leads to significantly faster search task completion times, and significantly greater usability.

KW - conversational search

KW - dialogue-systems

KW - voice interfaces

UR - https://sites.google.com/view/cair-ws/cair-2018

M3 - Paper

ER -

Dubiel M, Halvey M, Azzopardi L, Daronnat S. Investigating how conversational search agents affect user's behaviour, performance and search experience. 2018. Paper presented at The Second International Workshop on Conversational Approaches to Information Retrieval, Ann Arbor, United States.

Access to Document