AI Scribes in Primary Healthcare Settings

By the time of the conference, Digital Rights Watch will have completed research into AI scribes used in primary health care settings, specifically by GPs. It's an interesting case study in the gaps in regulation and an absence of clear pathways for complainants who have experienced harms. There are documented cases of harm that exist both here and in other countries (eg gender bias, challenges with ESL patients) and there are also concerns about poor quality practices relating to data (eg lack of consent, lack of privacy policies). There are also more generally competition concerns (concentration in the market arising from funding and size, rather than quality). Our research demonstrates some of these harms are challenging to identify at an individual level, and also there are few clear pathways for seeking to remedy them. This highlights real challenges with the government's approach to AI regulation generally, with no specific regulator, a safety institute that is under resourced and focused on other topics, and existing regulators who are stretched already. The rapid rollout of AI scribes in GP offices poses significant questions about the effectiveness of this strategy.