Conversational agents trust calibration

Mateusz Dubiel; Sylvain Daronnat; Luis A. Leiva

doi:10.1145/3543829.3544518

Conversational agents trust calibration

Mateusz Dubiel, Sylvain Daronnat, Luis A. Leiva

Computer And Information Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

11 Citations (Scopus)

17 Downloads (Pure)

Abstract

Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users' reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user's trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-Agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design.

Original language	English
Title of host publication	Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022
Editors	Martin Halvey, Mary Ellen Foster, Jeff Dalton, Cosmin Munteanu, Johanne Trippas
Place of Publication	New York
Publisher	Association for Computing Machinery
Number of pages	6
ISBN (Electronic)	9781450397391
DOIs	https://doi.org/10.1145/3543829.3544518
Publication status	Published - 26 Jul 2022
Event	4th International Conference on Conversational User Interfaces, CUI 2022 - Glasgow, United Kingdom Duration: 26 Jul 2022 → 28 Jul 2022

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	4th International Conference on Conversational User Interfaces, CUI 2022
Country/Territory	United Kingdom
City	Glasgow
Period	26/07/22 → 28/07/22

Funding

This work was supported by the Horizon 2020 FET program of the European Union through the ERA-NET Cofund funding grant CHIST-ERA-20-BCI-001.

Keywords

conversational agents
design ethics
trust
user-centred design

Access to Document

10.1145/3543829.3544518Licence: CC BY-NC 4.0

Dubiel-etal-ACM-2022-Conversational-agents-trustFinal published version, 151 KBLicence: CC BY-NC 4.0

Cite this

Dubiel, M., Daronnat, S., & Leiva, L. A. (2022). Conversational agents trust calibration. In M. Halvey, M. E. Foster, J. Dalton, C. Munteanu, & J. Trippas (Eds.), Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022 Article 30 (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3543829.3544518

Dubiel, Mateusz ; Daronnat, Sylvain ; Leiva, Luis A. / Conversational agents trust calibration. Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022. editor / Martin Halvey ; Mary Ellen Foster ; Jeff Dalton ; Cosmin Munteanu ; Johanne Trippas. New York : Association for Computing Machinery, 2022. (ACM International Conference Proceeding Series).

@inproceedings{222715d2e0594027b07eb15727448c23,

title = "Conversational agents trust calibration",

abstract = "Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users' reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user's trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-Agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design.",

keywords = "conversational agents, design ethics, trust, user-centred design",

author = "Mateusz Dubiel and Sylvain Daronnat and Leiva, \{Luis A.\}",

year = "2022",

month = jul,

day = "26",

doi = "10.1145/3543829.3544518",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

editor = "Martin Halvey and Foster, \{Mary Ellen\} and Jeff Dalton and Cosmin Munteanu and Johanne Trippas",

booktitle = "Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022",

address = "United States",

note = "4th International Conference on Conversational User Interfaces, CUI 2022 ; Conference date: 26-07-2022 Through 28-07-2022",

}

Dubiel, M, Daronnat, S & Leiva, LA 2022, Conversational agents trust calibration. in M Halvey, ME Foster, J Dalton, C Munteanu & J Trippas (eds), Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022., 30, ACM International Conference Proceeding Series, Association for Computing Machinery, New York, 4th International Conference on Conversational User Interfaces, CUI 2022, Glasgow, United Kingdom, 26/07/22. https://doi.org/10.1145/3543829.3544518

Conversational agents trust calibration. / Dubiel, Mateusz; Daronnat, Sylvain; Leiva, Luis A.
Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022. ed. / Martin Halvey; Mary Ellen Foster; Jeff Dalton; Cosmin Munteanu; Johanne Trippas. New York: Association for Computing Machinery, 2022. 30 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

TY - GEN

T1 - Conversational agents trust calibration

AU - Dubiel, Mateusz

AU - Daronnat, Sylvain

AU - Leiva, Luis A.

PY - 2022/7/26

Y1 - 2022/7/26

N2 - Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users' reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user's trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-Agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design.

AB - Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users' reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user's trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-Agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design.

KW - conversational agents

KW - design ethics

KW - trust

KW - user-centred design

U2 - 10.1145/3543829.3544518

DO - 10.1145/3543829.3544518

M3 - Conference contribution book

AN - SCOPUS:85139227714

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the 4th International Conference on Conversational User Interfaces, CUI 2022

A2 - Halvey, Martin

A2 - Foster, Mary Ellen

A2 - Dalton, Jeff

A2 - Munteanu, Cosmin

A2 - Trippas, Johanne

PB - Association for Computing Machinery

CY - New York

T2 - 4th International Conference on Conversational User Interfaces, CUI 2022

Y2 - 26 July 2022 through 28 July 2022

ER -

Conversational agents trust calibration

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Fingerprint

Cite this