Doctors should avoid embracing every new artificial intelligence system uncritically or refusing to use AI on principle, argues Daniel Sokol in an opinion piece in The BMJ.
Sokol writes:
Artificial intelligence (AI) is already being introduced into workflows in Britain’s NHS, and in the coming years, it will undoubtedly become part of the day-to-day practice of most doctors.
But what happens when AI contributes to patient harm?
Unlike doctors, an AI system cannot be sued under English law. Legal responsibility may instead fall on the clinician, healthcare provider, developer, or manufacturer, depending on what exactly went wrong and why.
NHS Resolution, which handles clinical negligence claims involving NHS trusts and general practices, has already received its first cases in which “the use of AI in delivering patient care is a potential factor.”
The General Medical Council (GMC) has also revealed that it has received referrals about doctors misusing AI systems. These claims and complaints are likely to increase in number.
Doctors could face criticism for their AI use for several reasons, including:
• Using an AI system when it should not have been used
• Using an unsuitable AI system
• Using a suitable AI system but doing so incorrectly or without reasonable care and skill
• Failing to use an AI system when it ought to have been used
• Failing to provide patients with material information about the use of AI or reasonable alternatives.
Safe and responsible use
The decision to use AI should be a considered one, looking at the pros and cons of the technology. Doctors can follow these practical tips to use AI safely and responsibly.
Use the right tool and understand it
Like any medical tool, doctors should understand the purpose of the AI system. A system may be suitable for identifying diabetic retinopathy, for example, but not other retinal conditions.
Before using an AI system, check that it has been approved by your organisation and understand in broad terms what it does and how it works, including its limitations. Complete any training, or ask for some if you feel you need it, and follow local guidance.
Although there is no need to understand the source code, you should know enough about the AI system to explain to a patient what it does and why you are using it. You should know what the output means and when human review is expected.
Keep patient information safe
As a doctor, you have a duty to keep patient information confidential. If using a public or unauthorised AI system, do not enter identifiable patient information. Removing a name or NHS number might not be enough if the clinical and personal details could still identify the patient. The more unusual the medical condition, the higher the chances of identification.
Only input confidential information into AI tools approved by your organisation (ie, tools that have passed information governance and data protection checks).
Check the input before acting on the output
Even a reliable system can produce inaccurate results from inaccurate information. Check that the correct information has been entered, especially when data were copied automatically.
Using an AI system does not mean that you can abandon professional judgment. Consider whether the result fits the history, examination, and other evidence. Has AI overlooked something or produced false precision? As a rule of thumb, the higher the stakes of the AI output, the greater the level of scrutiny from the supervising clinician.
If the output conflicts with your intuition or judgment, recheck the inputs, use another method, or seek a colleague’s view. Whether you accept or reject the recommendation, have a reason that you can explain beyond “the AI said so,” and be sure to document your reason.
Tell the patient when it matters
The GMC advises doctors to discuss the use of innovative technologies with patients, including options, uncertainties, and limitations, to enable patients to make informed decisions.
Although there is no blanket requirement to tell a patient about all AI use, if AI may materially affect diagnosis, prognosis, or treatment, it would ordinarily be prudent to inform the patient.
After listening to the explanation, a patient may ask you to forgo the use of AI completely. You should explore the reasons for the request and consider whether a reasonable non-AI alternative can be offered. If not, explain why. It may be that AI use is integral to safe care.
The legal standard of care evolves. In time, a doctor might be criticised for failing to use an AI system whose benefits have become well established.
That tipping point – when no reasonably competent doctor will fail to use AI – will probably arrive at different times in different specialties. The evidence of benefit, guidance from professional bodies, the availability and cost of the technology, and the standard practice of other clinicians in your area of specialism will be relevant factors.
Leave a useful record
When AI has materially influenced a clinical decision, make this clear in the record, particularly if you departed from its recommendation. Imagine a colleague asking in the future, “Why did the doctor make this decision when the AI recommended something else?”
Act when something goes wrong
If an AI system is involved in an adverse incident or produces a potentially unsafe output, protect the patient and use the appropriate reporting process, identifying the AI product and version if known. Preserve evidence if possible, such as outputs and logs.
An unexpected result may show a wider problem with the AI system. Raise concerns through clinical governance so the system can be audited or restricted if necessary. If a patient has been harmed or distressed, the ordinary duty of candour applies.
Ultimate purpose
Doctors should avoid the extremes of embracing every new AI system uncritically or refusing to use AI on principle. When you do use AI, know your tool, maintain confidentiality, check any output that does not fit with your intuition or judgment, take responsibility for your decision, and keep a useful record of the decision and its rationale.
Finally, always remember that the ultimate purpose of using AI should be to provide better and safer care for patients.
Daniel Sokol –medical ethicist and barrister, London, United Kingdom.
The BMJ article – Six rules for doctors using AI (Open access)
See more from MedicalBrief archives:
Big Tech push for AI in healthcare prompts calls for caution
Case study insights into ethics of AI and data science in medicine
AI errors shaped doctors’ decisions despite evidence – Spanish study