Home > Research > Publications & Outputs > Holding AI to Account

Electronic data

  • Holding Ai to Account

    Accepted author manuscript, 709 KB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published
Close
Article number2
<mark>Journal publication date</mark>30/04/2023
<mark>Journal</mark>ACM Transactions on Computer-Human Interaction
Issue number2
Volume30
Number of pages34
Pages (from-to)1-34
Publication StatusPublished
Early online date13/04/23
<mark>Original language</mark>English

Abstract

The need for AI systems to provide explanations for their behaviour is now widely recognised as key to their adoption. In this article, we examine the problem of trustworthy AI and explore what delivering this means in practice, with a focus on healthcare applications. Work in this area typically treats trustworthy AI as a problem of Human–Computer Interaction involving the individual user and an AI system. However, we argue here that this overlooks the important part played by organisational accountability in how people reason about and trust AI in socio-technical settings. To illustrate the importance of organisational accountability, we present findings from ethnographic studies of breast cancer screening and cancer treatment planning in multidisciplinary team meetings to show how participants made themselves accountable both to each other and to the organisations of which they are members. We use these findings to enrich existing understandings of the requirements for trustworthy AI and to outline some candidate solutions to the problems of making AI accountable both to individual users and organisationally. We conclude by outlining the implications of this for future work on the development of trustworthy AI, including ways in which our proposed solutions may be re-used in different application settings.