Investigating how conversational search agents affect user's behaviour, performance and search experience

Voice based search systems currently do not support natural conversational interaction. Consequently, people tend to limit their use of voice search to simple navigational tasks, as more complex search tasks require more sophisticated dialogue modelling. In this paper, we explore how people’s search behaviour, performance and perception of usability change when interacting with a conversational search system which supports natural language interaction, as opposed to a voice based search system which does not. Previous research has demonstrated that a voice based search system’s inability to preserve contextual information leads to user’s dissatisfaction and discourages further usage. We conducted an interactive study comparing a simulated conversational search system against a slotbased voice search system. We hypothesise that the conversational system, with its ability to preserve and maintain conversational state, will lead to greater satisfaction. Our results indicate that participants prefer the conversational system over current voice based system, the conversational search system leads to significantly faster search task completion times, and significantly greater usability.