Suspicious Minds: the Problem of Trust and Conversational Agents
Fulltext URI
Document type
Additional Information
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In recent years, the field of natural language processing has seen substantial developments, resulting in powerful voice-based interactive services. The quality of the voice and interactivity are sometimes so good that the artificial can no longer be differentiated from real persons. Thus, discerning whether an interactional partner is a human or an artificial agent is no longer merely a theoretical question but a practical problem society faces. Consequently, the ‘Turing test’ has moved from the laboratory into the wild. The passage from the theoretical to the practical domain also accentuates understanding as a topic of continued inquiry. When interactions are successful but the artificial agent has not been identified as such, can it also be said that the interlocutors have understood each other? In what ways does understanding figure in real-world human–computer interactions? Based on empirical observations, this study shows how we need two parallel conceptions of understanding to address these questions. By departing from ethnomethodology and conversation analysis, we illustrate how parties in a conversation regularly deploy two forms of analysis (categorial and sequential) to understand their interactional partners. The interplay between these forms of analysis shapes the developing sense of interactional exchanges and is crucial for established relations. Furthermore, outside of experimental settings, any problems in identifying and categorizing an interactional partner raise concerns regarding trust and suspicion. When suspicion is roused, shared understanding is disrupted. Therefore, this study concludes that the proliferation of conversational systems, fueled by artificial intelligence, may have unintended consequences, including impacts on human–human interactions.
Description
Keywords
Citation
URI
Collections
Endorsement
Review
Supplemented By
Referenced By
Number of citations to item: 10
- Marc Relieu (2024): How Lenny the bot convinces you that he is a person: Storytelling, affiliations, and alignments in multi-unit turns, In: Discourse & Communication 6(18), doi:10.1177/17504813241271437
- Yimin Xiao, Yuewen Chen, Naomi Yamashita, Yuexi Chen, Zhicheng Liu, Ge Gao (2024): (Dis)placed Contributions: Uncovering Hidden Hurdles to Collaborative Writing Involving Non-Native Speakers, Native Speakers, and AI-Powered Editing Tools, In: Proceedings of the ACM on Human-Computer Interaction CSCW2(8), doi:10.1145/3686942
- Jakub Mlynář, Lynn de Rijk, Andreas Liesenfeld, Wyke Stommel, Saul Albert (2024): AI in situated action: a scoping review of ethnomethodological and conversation analytic studies, In: AI & SOCIETY 3(40), doi:10.1007/s00146-024-01919-x
- Adam Palmquist, Izabella Jedel, Chris Hart, Victor Manuel Perez Colado, Aedan Soellaart (2024): “Not in Kansas Anymore” Exploring Avatar-Player Dynamics Through a Wizard of Oz Approach in Virtual Reality, In: Lecture Notes in Computer Science, doi:10.1007/978-3-031-61041-7_17
- Diego Gosmar (2024): Conversational hyperconvergence: an onlife evolution model for conversational AI agency, In: AI and Ethics 2(5), doi:10.1007/s43681-024-00463-0
- Adam Palmquist, Izabella Jedel, Ole Goethe (2024): Design Implications and Processes for an Attainable Game Experience, In: Human–Computer Interaction Series, doi:10.1007/978-3-031-30595-5_3
- Jin Mao, Baiyun Chen, Juhong Christie Liu (2023): Generative Artificial Intelligence in Education and Its Implications for Assessment, In: TechTrends 1(68), doi:10.1007/s11528-023-00911-4
- Tom Ziemke (2024): Ironies of social robotics, In: Science Robotics 91(9), doi:10.1126/scirobotics.adq6387
- Marc Relieu (2024): How Lenny the bot convinces you that he is a person: Storytelling, affiliations, and alignments in multi-unit turns, In: Discourse & Communication, doi:10.1177/175048132411271437
- Soo Jung Hong (2025): What drives AI-based risk information-seeking intent? Insufficiency of risk information versus (Un)certainty of AI chatbots, In: Computers in Human Behavior, doi:10.1016/j.chb.2024.108460