Data indicates that South Africa has passed the peak of the COVID-19 infections wave, says Professor Robin Wood of the University of Cape Town. Despite its limitations, pandemic modelling suggests SA should now move from a single national response to more nuanced provincial responses.
“If you've only had 34 deaths in your province, then a total shutdown and destruction of your economy may not be the right thing to do,” Wood told a webinar held on Tuesday 11 August, hosted by PPS – Health Professions Indemnity, MedicalBrief and the Desmond Tutu Health Foundation, and attended by more than 700 participants.
The weaknesses of modelling have been exposed, but out of the data has come key information – for instance on major provincial differences, which suggests there should be a move away from national to more local responses to the pandemic.
Either the data being collected in some provinces is completely wrong, “in which case we’re in real trouble”, or there are big differences between the provinces and there should be more emphasis on provincial modelling. “If we are to reintroduce some level of normality, it might be wise to do this at a local or provincial level in order to inform our way out of this.”
Wood is director of the Desmond Tutu HIV Centre in the Institute of Infectious Diseases and Molecular Medicine at UCT, where he is Emeritus Professor of Medicine, writes Karen MacGregor for MedicalBrief. He has been a visiting fellow at Harvard Medical School and honorary professor at the London School of Hygiene and Tropical Medicine, and has served on international scientific advisory boards including those of the World Health Organization, International Aids Society, TB Vaccines Initiative and Gates Foundation.
COVID-19 models – Fatally flawed?
Titled “COVID-19 models – Fatally flawed?” the webinar was the fourth in a five-part series investigating research and developments relating to COVID-19. The series is moderated by MedicalBrief Managing Editor William Saunderson-Meyer.
Around the world, models have been developed to predict COVID-19 infection and death rates, among other aspects. Models have dramatically influenced the behaviours of officials, policy-makers and people, but have also been widely criticised for inaccuracy. An example is Imperial College London’s predictions of 2.2 million COVID-19 deaths in the US and half a million in the UK.
At least two hypotheses must be tested: whether properties of the coronavirus were mis-characterised and this distorted the accuracy of models; and whether changed behaviour reduced infections. Or both. People bought into flawed models, Wood has previously said, “because we didn’t have anything else”.
How should we be using models to inform future South African policy?
“There has to be an openness. If we are going to scientifically question these models, we have to know what the models consist of; we have to know the code and the approach; we need to know when they're being updated,” said Wood.
It is very difficult and perhaps impossible to predict the next year of an epidemic from very early data, so flaws in models were to be anticipated and have happened worldwide. “It's easy in retrospect to say we overreacted. But at the time there was a lot of fear.
“But as time goes on, we need to be seeing that models are updated. Some of the assumptions should be questioned,” Wood argued.
Variability in infections and deaths between the provinces suggests that it is necessary to explore the assumption that everybody is equally susceptible to COVID-19 – perhaps not everybody is. Also, it appears the reproductive number or Ro – the number of people that one infected person will on average pass the virus on to – may not be the same everywhere.
“It looks as though we have a very interesting country which is heterogeneous, and that allows you to raise hypotheses as to what could cause that? Is it due to the solar radiation, vitamin D or whatever? Just parroting Western models in an African context is probably not right either,” said Wood.
“But the models need to be updated and checked against reality and modified. I think the most important thing is to design models to answer specific questions that are relevant.”
Politicians love it when models overestimate damage, Wood quipped, because they can take credit for ameliorating the epidemic. When the Western Cape was heavily affected early on, that was said to be because the Democratic Alliance was in charge and not the ruling party.
“So people look at models and interpret them in very strange ways.”
Wood strongly believes there is a need for open debate about models. A robust debate that would be nothing at all like what happened to Professor Glenda Gray, president of the SA Medical Research Council, when she criticised aspects of the lockdown in May.
“She was closed down straight away and publicly humiliated, and attempts were made to start investigations and get her sacked. That’s not the way to use science to try and address a very serious problem,” Wood told the webinar.
Statistics show that South Africa is well over the hump of COVID infections. Why is this not being reflected in government statements and the media?
Wood has been tracking several parameters of the COVID-19 pandemic, and uses seven-day averaging. The seven-day national rolling average for new coronavirus cases peaked in July.
Gauteng hit a sharp peak in infections in mid-July and mortality peaked about two weeks later. “A similar pattern occurred in the Western Cape, although there's a grumbling on of the epidemic.” In KwaZulu-Natal, case numbers have peaked but Wood says mortality will probably peak in the next few days. Free State has probably peaked too.
Interestingly, this has not appeared in the headlines. “It seems as though there needs to be some drama in the headlines and this is not dramatic enough for them.”
This also raises a problem of context. In the four provinces with low infection and mortality, there have been around 400 COVID deaths. There are around half a million deaths normally in South Africa each year. With a quarter of SA’s population, the four provinces would therefore normally have some 125,000 deaths a year. Against this, 400 COVID deaths looks small.
“It looks as though we have peaked,” said Wood. “But of course, if we change policies then we may unleash another small or larger second wave.” It all comes down to people’s immune status, but there are no accurate ways of measuring immunity using eaxisting tests.
But certainly, he says, the epidemic is not behaving like the conventional SIR model that is widely used in epidemic modelling. An SIR model tracks the ratio of susceptible, infected and recovered individuals within a population.
The SARSCov2 pandemic has changed the course of nations and affected all on the planet. Now that the shock is wearing off, the fallibility of models has become clear and there is scepticism about whether they are fit for purpose. What is your view?
Wood is a clinician and not a mathematician, but he has been involved in mathematic modelling of TB and HIV. His experience feeds scepticism of models but also demonstrates some of their utility. He gave the webinar three examples of epidemiological models.
The classic model for TB transmission is Wells-Riley from the mid-1900s. It is an equation which among other things enables tinkering with an important variable – the infectiousness of patients – and it is measured in theoretical quanta per unit of time. It is flawed in, for instance, assuming steady state conditions for transmission and ignoring distance between individuals.
While the model is imperfect and fundamentally does not reflect reality, it has been useful to compare, for instance, TB transmission in public transport or in Pollsmoor Prison. “It is an example of a rough and ready description of reality that has actually been quite useful over the years.”
A second example is cost effectiveness analysis. For HIV the model is well parameterised and calibrated against CD4 counts, viral loads and so on. This is plugged in to a cost effectiveness model which calculates years of life saved against per capita gross national product. However, the model does not take into consideration the scale of the problem, which affects affordability. “So you have a very good model but the results can be not so useful for policy.”
The third example is the World Health Organization’s TB model, which uses treatment as an important parameter. The model ignores the social component of TB and its transmission, says Wood, and has harmfully diverted attention from transmission and other issues.
“So you come back to the aphorism, which many people quote, from a leading British statistician, George Box, that 'all models are wrong, but some are useful'. That would be very much my view of models,” said Wood.
How has the South African government used modelling to direct policy, or not?
Several models have been available, said Wood, including one from a modelling consortium of scientists from universities such as Cape Town, Stellenbosch and the Witwatersrand. Theirs is a SIR model based on numbers of susceptible, infected and recovered people.
What would happen if you did nothing in response to the COVID-19 pandemic? Then the model is designed to see how changes can be made to the base case scenario through interventions and policies. “They varied the optimism and pessimism of their predictions.” The model was released on 6 May. Wood has not seen updates, but hopes there have been.
“They peppered their model appropriately with the caveat that the projections are subject to considerable uncertainty and variability. That's very true.”
The model predicted that far more people would be infected – though it pointed out that numbers of infections might not be properly represented because of not enough or inadequate testing, and asymptomatic individuals. The number of deaths has been between the model’s optimistic and pessimistic predictions.
“Reality is about half what they said,” said Wood. “You can interpret that in different ways. You can say, wow, intervention strategies were even more successful than we thought. Or you could say, perhaps the base case was just too pessimistic. The SIR model requires an infectious modality transmitting very very fast.”
Wood has been interested in the provincial figures. While the models based predictions on the population sizes of provinces, the aberrant regions – Mpumalanga, Limpopo, North West and Northern Cape – have incidence and fatality rates of about a quarter of the Western Cape.
“As you look at the South African epidemic, the caseload and the mortality decreases as you go north and west. Is that likely to be right or is it that reality has been misconstrued?” Wood points out that further north, Namibia has had only 16 deaths and Botswana only two deaths.
He argues that basic assumptions of the model, such as that infection rates will be the same in every province and that everybody is equally susceptible to COVID-19, are incorrect.
“Perhaps the difference between countries and the difference between provinces in our own country are not explained totally by the implementation of non-pharmaceutical interventions.”
“So I find that a model which seems as though it's missed the mark as far as predictions are concerned, and perhaps had too severe a base case, has actually given us insight into what might be happening in our population.”
Wood looked at whether sunshine could help explain the provincial differences. “Look at the photovoltaic potential of the different provinces, and it does fit very nicely. Of course, association doesn't mean causation. But it does create hypothesis generation.” Vitamin D status on the very southern tip of Africa would be different to further north.
Another possible explanation might be age distribution in different provinces. “Why haven't we got that data? Why haven't we got mortality data by age? Then we could explore those hypotheses. We should be able to scientifically look at them,” said Wood.
How have models in other countries informed policy?
Worldwide, there are models that have helped in the response to SARS-CoV-2.
One early model, in late February from the London School of Hygiene & Tropical Medicine and titled “Feasibility and Controlling COVID-19 Outbreaks by Isolation of Cases and Contacts”, influenced some national policies, said Wood.
Essentially, it was an elimination model that showed that the Ro number – the number of other people that one infected person will on average pass the virus to – was up in the air. They modelled Ro numbers of 1.5, 2.5 and 3.5. The study found that the higher the infection rate, the higher the proportion of transmitters who needed to be identified and isolated.
They also found some unknowns, such as the time from initially being infected to presenting with symptoms, the time from presenting with symptoms to being isolated, and the numbers of people who were asymptomatic.
The London School model informed some national policies. China isolated not only infected patients but also people they had been in contact with. New Zealand was relatively successful in containing infection, which has been helped by being an island. South Korea used the strategy and its number of cases has remained very low.
“The strategy can work for elimination when the numbers are small at the beginning of an epidemic. Once there is an established epidemic, the resources needed to test and isolate high proportions of cases would be very difficult indeed.”
The Imperial College London model
The other model that has determined global policy came from Imperial College London, led by famous epidemiological modeller Neil Ferguson. It had seemed as though the United Kingdom was heading towards a laissez faire approach similar to Sweden, relying on mitigation. But then Ferguson wrote a paper that addressed the concepts of mitigation and suppression of the epidemic.
Wood was interested that Ferguson’s was the only early paper that made a direct comparison with the 1918 ‘Spanish flu’ pandemic. He described SARS-CoV-2 as the worst thing to hit humanity since 1918. In a previous paper in 2007, Ferguson mathematically modelled the 1918 flu epidemic and described his study as “far from historical. This is really important for policy-makers when they mitigate future pandemics”, Ferguson wrote.
There were four important findings from the analysis of 1918, Wood told the webinar. In it Ferguson focused in on 16 cities in North America that had slightly different trajectories and different respiratory and influenza mortality during 1918-19,
It has been assumed that with an Ro number of about two, once 50% of a population has been exposed, an epidemic can no longer maintain itself. The first, surprising, lesson Wood learned from the Ferguson study is that if nothing is done, you get an overshoot – instead of an epidemic becoming unsustainable at 50% it can go up to 80% of the population infected.
A second, interesting finding from Ferguson – which is relevant to COVID-19 – is that if interventions in a city were too strong, once they were stopped those cities saw a second wave of infections. “The key was that the different timing and intensities of interventions changed the shape of subsequent epidemics.”
One aspect that stuck in Wood’s mind was that the differences between cities and their non-pharmaceutical interventions did not explain the totality of the difference between epidemic experiences. “It comes back to what we were saying about the provinces.”
The Ferguson model, first released in March and published in Nature in June, “scared the UK government to death”. Its base case predicted that 81% of people in the UK and United States would be infected, and that there would be half a million deaths in the UK and two million in America. In this scenario, health care facilities would be overwhelmed by a factor of 30-fold.
Ferguson investigated a so-called mitigation strategy, which tried to decrease infections using identification and isolation, and social distancing for the elderly and vulnerable, but not closing of education and other institutions. “He showed that these horrendously high mortality figures could be reduced by about 50%,” said Wood, but the health system would collapse.
So then Ferguson looked at supressing or controlling the epidemic, via lockdown. At the end of the paper Ferguson said that mitigation strategies and suppression strategies would require multiple interventions, but mitigation would still result in disaster and therefore the epidemic must be suppressed as the only viable intervention.
That model influenced many national policies that have some sort of suppression. The study of 1918 showed that if secondary epidemics were monitored very closely, they could be closed down. “You could have pushing the brake pedal and releasing it as another strategy.”
“The problems with the two strategies are that mitigation needs to allow half the population to get infected, with the subsequent mortality, whereas suppression needs to be maintained or on-off until a vaccine is available.” Of course, it is almost impossible to check whether models are correct or not because a variety of interventions have been made.
Wood cited the intriguing case of Sweden. Some epidemiologists were so alarmed by the country’s mitigation approach that they produced a model predicting far higher deaths than occurred. “One of the problems with models is that they are often coloured by the beliefs of the people writing them,” Wood pointed out. Models can also be very inaccurate.
“You can see why scepticism comes in.”
Moving forward, there needs to be more access to data and scientific analysis
At the beginning of the epidemic, said Wood, South Africa had to do something to assess likely developments and how to react to them. “But subsequently I have become sceptical that the whole population is equally susceptible. None of the curves shown by the models look anything like the models for South Africa or even the provinces.
“So there are approximations and you have to ask yourself, what's the useful thing coming out of the approximations?” Not very much, said Wood.
While projections are carefully thought out with the best motives, they have resulted in misallocation of resources. For example, the concept under all scenarios that health facilities would be overwhelmed, has not happened. Specially built facilities have hardly been used.
“We don’t want to apportion blame, but we have made mistakes. And I don't see anything now moving us forward with a new vision”.
When models go wrong, it is usually because there is a factor that has not been included in the understanding of the natural history of the disease. “There are some unusual things about this disease, such as why it is so age related, why it's so obesity and diabetes associated.”
South Africa is trying to distil messages from inadequate data. It is not alone in this, said Wood. For instance, it turned out – shockingly – that the UK was not counting COVID-19 deaths that occurred out of hospitals, in homes and old age homes.
“Standardisation of data, comparison of data, quality of data are all very important. When you can't do them at a large scale, you should set up sentinel sites where you research well in order to try and get a clue as to what's going on. We need decent data if we're going to seriously address this epidemic.”
Should models be used for scenario planning rather than predicting epidemics?
Wood agreed that scenario comparisons could be more useful. He started the webinar by describing the Wells-Riley model being retrospectively fitted to data in order to answer questions. The London School’s elimination model showed how models could inform strategies rather than trying to predict the future.
“Models need to be designed to answer specific questions. They are always problematic when trying to predict long-term future trajectories of the epidemic. Modellers were asked to do that and they got it wrong; that's natural. It is highly risky to try and predict the future.”
The bottom line, said Wood, is that modelling is a really interesting field “but we’re getting data fed to us at a very early stage and we don't have definitive answers”.
[link url="https://www.youtube.com/watch?v=yhOaIrcc5uo&feature=youtu.be"]Click here to view the webinar: COVID-19 models – Fatally flawed?[/link]
About webinar sponsor PPS
PPS Health Professions Indemnity was developed in response to the challenges South African health professionals face, from the aggressive litigation landscape to rising professional indemnity costs. This indemnity protection is delivered through the financial strength and security of the PPS Group. The focus is on sustainability, providing quality protection for the duration of your career as a health professional. More information on the solution is available in the Product Brochure.