With varying degrees of success, artificial intelligence (AI) has begun to play the role of research assistant, radiologist, health educator and even therapist more than most of us realise, but with rapid advancements in the technology, the implications for healthcare, especially in countries like SA, are mixed.
Jesse Copelyn, writing in Spotlight, pinpoints the most immediate implications of these new technologies for healthcare in South Africa.
AI can perform various tasks that were previously the sole purview of qualified doctors, according to a growing body of research. For instance, several recent studies show that more tumours can be found during routine breast x-rays and colonoscopies when AI detection software assists in scanning images.
Other research has looked at whether large language models (along the lines of ChatGPT) can correctly diagnose patients after being given information such as their symptoms and medical history. Researchers at Google tested this and found that an AI model that they developed could provide a list of possible diagnoses for real-world patients that were significantly more accurate than those offered by doctors who were given the same information.
This is just one study and by no means the last word, but even so, it seems likely that doctors will increasingly turn to such models to help them get diagnoses right.
And new developments aren’t just confined to diagnostics. For instance, various health services in the US now deploy chatbots that do everything from answering patients’ medical questions to booking their appointments.
In other cases, chatbots have even been developed to act as therapists, talking to patients via phone apps and providing them with techniques to overcome harmful thoughts. There are some signs that this kind of thing might be effective, but most studies in this area have been small, so whether it works is far from settled.
Meanwhile, various AI tools are being used for medical research. This includes Google DeepMind’s AlphaFold, which can predict the structure of protein molecules. An Oxford research team working to develop a more effective malaria vaccine used this software to model a crucial protein found in malaria parasites. This means researchers have more information about how to target the protein, which the parasite needs to reproduce.
The jury is still out on what exactly these various types of AI will mean for healthcare services and whether they will live up to the hype. For example, chatbots still occasionally generate false information, often with perfect confidence – such hallucinations can have devastating consequences in a healthcare context. It is unclear if, or when, this problem will be solved.
Chances are that some applications will become a day-to-day part of 21st-century healthcare, while others will fall by the wayside.
It is also still unclear what impact AI could make in health systems in developing countries. A technology that works well at a private hospital in the US may not always be appropriate for a rural Eastern Cape clinic with unreliable power supply. Some AI products will inevitably cross the divide and flourish – in fact, some already have.
We take a look at some examples.
AI is already helping to tackle TB
The area where AI appears to be making the biggest impact in South Africa is in screening TB – by some measures the country’s leading cause of death. The bacterial disease is usually tested by analysing sputum using a molecular test, typically in a lab. But because people who have TB in South Africa are often asymptomatic, many simply never get tested.
To get around this, screening initiatives have been developed in which health workers take vans containing mobile X-ray units to communities where the disease is prevalent. Residents then get a free chest X-ray, which can reveal if they have abnormal-looking lungs (even if they have no TB symptoms). If the X-ray is irregular, they are sent for a sputum test to confirm the diagnosis.
But while conducting the X-rays is simple enough, the next step is often where projects face a bottleneck. Dr Emily Wong, an infectious disease scientist at the African Health Research Institute, explains that “you have to interpret those chest X-rays, and usually that requires a doctor or even a radiologist, and (they) are very scarce in South Africa”.
In fact, the number of radiologists serving the public sector is currently a quarter of what’s required.
It’s here that AI comes in.
In 2021, a study published in The Lancet showed that a series of AI-based software applications were not only able to detect TB in X-ray images as accurately as experienced radiologists, but were outperforming them. When Wong and her colleagues tested one of these AI products in a TB screening project in KwaZulu-Natal, they found that the software was roughly as accurate as doctors.
Computer-aided detection
These AI-based tools, collectively known as computer-aided detection (CAD), typically work as follows: a digital X-ray image is captured on a computer. The CAD software analyses it and gives it an abnormality score ranging from zero to 100, where a higher number indicates an increased chance of TB.
Just as in the case of the radiologist, these scores aren’t definitive; a person with a low score might still have TB, there’s just a smaller chance that they do. Health workers decide on an appropriate threshold value (say 50) and anyone above that number is sent for a sputum test.
This threshold score is significant. If it’s set very low, say at five, then virtually everyone with TB will be sent for testing, but so will many other people with relatively normal, healthy lungs. This means expensive sputum tests and scarce laboratory capacity will get used up on people who didn’t need to be tested in the first place.
But if the threshold value is set too high – say at 95 – then more people with TB will be missed, since at more than 95, only the most extreme cases will result in testing.
In the 2021 study, it was found that at a threshold score of 60, the top-performing CAD tool, called qXR, captured 90% of TB cases, while 74% of TB-negative people were correctly categorised as negative (the remaining 26% were incorrectly identified as abnormal and sent for further testing).
By comparison, the human radiologists only captured about 89% of cases and classified 63% of the TB-negative people correctly (these values varied depending on the classifications used but they were always less accurate than the CAD).
Many of the CAD tools are based on deep learning, meaning they identify patterns in large amounts of data. For instance, CAD software is trained on thousands of chest X-ray images, each labelled as “indicating TB” or “healthy”.
As it’s fed more labelled data, the algorithm identifies various features associated with TB – for example, a more asymmetric chest X-ray image means a higher likelihood of the disease. It’s then tested on unlabelled data to see whether it can make accurate predictions.
Such models are quite different from large language models like ChatGPT – and while not perfect, they do not have the same problem with hallucinations.
How are these tools being used in South Africa?
In South Africa, CAD software is being used in various mobile chest X-ray programmes sponsored by international aid groups. Dr Jody Boffa, a scientist working at the TB Think Tank, which advises the Department of Health, said: “Global Fund and USAid fund the machines, but then various implementing agencies (typically NGOs) are taking them out into the field.” In turn, the department “sets the rules” for how these programmes should operate.
Dr Elias Ramarumo, who works in the department monitoring these projects, said that 38 CAD software products had been procured by Global Fund, while eight were bought by USAid. Additionally, provincial Health Departments were “in a process of procuring digital X-ray units”, which would “come with CAD software”.
Currently, two CAD products are being bought by funders. One is CAD4TB, owned by Dutch company Delft, while the other is qXR by the Indian venture, Qure.AI. These products were the two top performers in the 2021 study, which tested five different tools.
Ramarumo said both companies are working with South African partners: Delft with Lomaen Medical and Qure.AI with Vertice MedTech.
Boffa said the products are being used in two kinds of screening programmes. In one, vans containing mobile X-ray devices are parked next to overburdened clinics, where people can have a CAD-assisted X-ray screening. In the second case, the vans are taken to “hotspot” communities and placed in areas “where people who are less likely to visit the clinic would be found”.
Before the vans arrive, project staff or community leaders “rally the area to let them know they’ll be coming”.
AI to help miners
While international funding agencies are the driving force behind AI-assisted screening in South Africa, the Department of Health says it’s also planning on using CAD software – not only for tackling TB but also for silicosis, the lung disease caused by breathing in silica, to which miners are often exposed. Unlike TB, there’s no lab test which confirms its presence – an analysis of a chest X-ray is final.
While there isn’t a cure for the disease, miners who are confirmed to have silicosis receive financial compensation from the Medical Bureau for Occupational Disease, run by the department.
It’s the bureau’s job to determine who deserves recompense. However, it hasn’t always been able to manage claims fast enough, says Professor Rodney Ehrlich, an occupational medicine specialist at UCT. “By about a decade ago, the completion backlog (of unpaid compensation claims) was more than 100 000, and all these paper files were piled up in back offices.”
Getting through claims requires enormous staffing capacity, he says, because a panel of doctors is required to analyse each X-ray image – the opinion of one doctor isn’t considered good enough.
It’s thus no surprise that the bureau is turning to AI, which has shown promise in this field, much like in TB screening. A 2022 study, published in the International Journal of Environmental Research, co-authored by Ehrlich analysed the ability of CAD software to detect silicosis and TB in chest X-ray images of North West gold miners.
Despite concerns that the AI would not be able to distinguish between the two diseases – which can have similar presentations in X-ray images – the CAD products were able to make similar classifications to those made by doctors.
The Health Department and its partners hosted a workshop in June which was designed to “forge a way towards harnessing AI” for silicosis and TB screening in “mining, peri mining and labour-sending communities of SA”, says Ramarumo.
The event included academics, CAD companies, the WHO as well as health departments from the Southern African Development Community.
Ramarumo says it was agreed that “the adoption of CAD systems for TB and silicosis in the mining sector is essential to enhance diagnostic accuracy, improve patient outcomes, enhance the compensation process and reduce the financial burden on the (mining) industry”.
The right data?
While researchers who spoke to Spotlight are excited about the capacity of AI to make our public health system more efficient, hosts of issues must be overcome with the tech. One is that the data used to train AI are not always appropriate for our current context.
For instance, these days, screening programmes in the country often try to find TB-positive people who do not yet have symptoms – as they are often already infectious and at risk of falling ill. But for a long time such “subclinical” TB didn’t accord with our traditional understanding of the disease.
Wong says that “the original paradigm… is that someone with TB is highly symptomatic – they have fevers, they’ve been coughing for weeks, lost a lot of weight, they’re sick – they’ve now come to seek care and they’ve been diagnosed with TB”.
The result? When CAD is trained on banks of chest X-rays, the images labelled “TB” will be from people who were highly symptomatic, says Wong. As such, the CAD software only learns to associate the disease with more extreme cases.
This can have practical consequences. When Wong and her colleagues used CAD for a screening programme in KwaZulu-Natal, they began by sending anyone with a CAD abnormality score above 60 for a test – in line with what had worked elsewhere.
Yet as the research continued, it became clear that many patients with subclinical TB were being missed because many had scores below that level. This forced them to use a much lower threshold to detect these cases, which as noted comes with trade-offs since many healthy people then get sent for testing.
It is likely that, as AI is rolled out in other healthcare contexts in South Africa, more of these nuances will emerge.
New regulations needed?
A related issue is that the data on which CAD is trained are often proprietary. Companies making the software aren’t obliged to share information about where their data come from or how their algorithms change when new versions come out.
In response, Wong and her colleagues released a statement in 2023 which called for “regulation to require CAD-developing companies to communicate changes between software versions”.
And this isn’t the only area where regulation appears to be lagging.
In this country, medical devices are technically regulated by the SA Health Products Regulatory Authority (Sahpra). But a 2022 journal article in SA Biotech Law argued that existing legislation is outdated for reviewing AI-based technologies.
In particular, the paper argues, is that the safety and efficacy of a device is supposed to be reviewed according to “predefined static specifications and standards”. For instance, a defibrillator might be assessed on how well it performs a specific function, and reviewers know that a given model would work in the same way over time.
However, the function of an AI-based chatbot is broader – it provides answers to different kinds of questions depending on what it’s asked – and its responses may change over time as it is fed more data. Assessing the technology thus becomes more difficult.
Asked about this problem, Sahpra’s communications officer, Nthabi Moloi, said the body “has not commenced with the registration of medical devices” so this is presumably not yet a problem (though Sahpra does sometimes use backdoor routes to review devices).
The way forward
Nevertheless, researchers say these are exciting times, and it appears that both international funders and the government are taking significant strides to use AI to address some of the country’s most devastating diseases.
Speaking about the recent workshop on the use of CAD tools for silicosis and TB screening, Zhi Zhen Qin, a digital health specialist at the United Nations Office for Project Service, says she was “impressed by the vision” of the conference organisers. By aiming to use AI to screen for both TB and silicosis at the same time – instead of viewing them as separate problems – “the South African government has been filling a much-needed leadership gap”.
In coming years, the role of AI in our health system might, of course, stretch far beyond TB and silicosis. But judging by our messy health data landscape, we are not ready for the transition.
While the Western Cape Department of Health & Wellness and the National Health Laboratory Service have done impressive work, our health data systems, generally, remain patchy and fragmented. This will make it more difficult to train and deploy locally appropriate AI solutions.
Whether new digital infrastructure being developed under the banner of NHI will solve the problem is debatable.
As with electronic data systems, building and deploying AI capacity in the public health system will not be easy. Patient data will have to be kept secure, systems will have to be interoperable, and rather than outsourcing everything to software vendors, the state will need to build at least some internal technical capacity – at the very least the government needs people with the technical expertise to know whether we are buying the right products.
In a country where hundreds of thousands of rands have been wasted on very simple websites, this can unfortunately not be taken for granted.
Large language models (LLMs)
Further afield, meanwhile, the European Medicines Agency (EMA) and the Heads of Medicines Agencies (HMA) have recently published high-level principles and recommendations for all staff across the European medicines regulatory network (EMRN) using large language models (LLMs) in their work.
LLMs are a category of generative AI, focusing on text generation. The applications can significantly support medicine regulators in their tasks and processes.
Whether they are used to query the extensive documentation regulators routinely receive, to automate knowledge/data mining processes, or as virtual AI assistants in everyday administrative tasks, LLMs have enormous transformative potential.
However, they also present challenges, e.g, variability in results, returning of irrelevant or inaccurate responses (so-called hallucinations), and potential data security risks.
The purpose of the guiding principles is to build understanding of the capabilities and limitations of these applications among staff at regulatory agencies across the EU so that they can harness the potential of LLMs effectively and avoid pitfalls.
The guiding principles cover various aspects of using LLMs, from ensuring safe input of data, to applying critical thinking and cross-checking outputs, to knowing whom to consult when concerns arise.
Additionally, the principles encourage regulatory agencies to support their staff in using LLMs. This includes defining governance on their use, specifying permitted use cases, providing training and monitoring risks.
The guiding principles are one of the deliverables of the multi-annual AI work-plan to 2028 by EMA and the HMA, and the living document will be regularly updated.
EMA Guiding Principales (Open access)
See more from MedicalBrief archives:
Ethical dilemmas as medicine intersects with AI chatbots
OECD: How artificial intelligence could change the future of health