ÁñÁ«ÊÓƵapp

The artificial intelligence (AI) train has well and truly left the station. But are we ready? Many would say no. While ethics, data privacy, and job displacement are among the concerns people have, the hope that AI will improve our lives remains.

Healthcare is at the forefront of discussions around the transformative possibilities of AI. Proponents of the technology claim it can improve diagnostic accuracy, enhance clinical decision-making, personalise treatment plans and increase access to care. But is it really that simple?

Professor says it’s not black and white. The Director of the Australian Centre for Health Engagement, Evidence and Values (ACHEEV) believes AI will transform healthcare but guidance regarding its use and governance is urgently needed, and should reflect public expectations about the technology.

“In clinical care the use of AI could bring many benefits and serious risks,” Professor Carter said.

“Australia currently lags behind other countries in healthcare AI development, deployment and governance and there definitely needs to be more healthcare-specific strategies developed.”

Making health systems more inclusive and democratic

That’s where the unique work of ACHEEV comes in.

The team at ACHEEV conduct high quality research about what matters to people in public health and healthcare. Their mission is to make health systems more inclusive and democratic.

One of the ways they do this is through deliberative democratic methods, such as citizens’ juries. A citizens’ jury is a group of everyday people brought together in the process of policy decision making.

“Citizens’ juries bring together randomly selected people who broadly represent the diversity of the entire community,” Professor Carter says.

“The people involved learn about important issues, discuss them with one another, and make recommendations about what should happen and how things should change.

“The group’s conclusions are reported to people and organisations that can make decisions about the topic and publicised to the wider community.

“A citizens’ jury is an ideal tool to use to explore how Australians believe AI should be used and governed in health care settings, because even though it’s already being used in Australia, it’s very much in its infancy.

“We’re in a unique position to potentially change the health landscape and community input into the application of AI is crucial.”

^{Citizens’ juries bring together randomly selected people who broadly represent the diversity of the entire community. Photo: Dylan Gillis}

How should AI be used in healthcare settings in Australia?

Up until 2023 no deliberative process with national representation had considered how AI should be used in healthcare in Australia. So Professor Carter and her team at ACHEEV got to work and convened a national citizens’ jury to discuss the issue.

An independent, not-for-profit, deliberative democracy recruitment agency, , recruited thirty Australian residents for the jury. The jury were asked to discuss the question ‘Under what circumstances, if any, should artificial intelligence be used in Australian health systems to detect or diagnose disease?’.

Professor Carter said it was a robust process and one that took the jurors 18 days to complete – 15 days of online activities and three days face-to-face.

^{Professor Stacy Carter believes AI will transform healthcare but guidance regarding its use and governance is urgently needed. Photo: Michael Gray}

Going in, Professor Carter and her team were unsure what the jurors would recommend.

“A deliberative democratic process is different to other kinds of research about preferences or views. People get to learn, to talk together and potentially change their views, and to develop recommendations that they think will be best for all Australians,” Professor Carter said.

This jury decided on 10 issues that needed recommendations, and developed 15 recommendations across those categories.

There was a very strong emphasis on governance – putting systems in place to oversee and evaluate healthcare AI. The jurors called for a national Charter overseen by an independent committee, high quality research to underpin AI tools, ongoing evaluation and monitoring to make sure AI performs the way its developers and vendors claim, and close scrutiny by regulators and health professional groups to make sure introducing AI technologies does not degrade the performance of health systems.

There was also a strong focus on fairness, justice, rights and engagement: ensuring everyone can benefit from AI, preferring systems built on data from Australia to reflect our multicultural community, ensuring patient rights are upheld and engaging the community to understand AI and participate in its oversight.

The overall goal was to ensure that benefits from AI implementation outweigh harms, and that those benefits are available to everyone, whatever their background or experience.

The way forward

Using a citizens’ jury to discuss how AI should be used in Australian healthcare settings was an obvious choice for Professor Carter, and one that she hopes health care decision makers and clinicians pay attention to.

“At the core of the recommendations were concerns around evaluation, integrity and transparency, and around fairness,” Professor Carter said.

“It’s a unique opportunity for clinicians and decision makers to listen to the values and priorities of a well-informed and diverse Australian mini-public and develop appropriate guidelines and governance regarding the use of AI in health care settings.

Professor Carter said she was heartened by the wide range of decision-making and policy bodies who are seeking guidance from the recommendations. They informed the National Policy Roadmap on AI in Healthcare, and have been shared with government agencies, non-government organisations, health professional bodies and consumer groups.

“This citizens’ jury and the resulting recommendations clearly illustrate the feasibility of robust public engagement and deliberation for guiding AI development and implementation in this country.

“This is our chance to get it right.”

You can find out more about this citizens’ jury and its recommendations in the .