Saliva tests for COVID-19, which are being introduced for NHS workers as part of the government’s mass testing programme, pick up only 13% of people with low levels of the virus and not 91%, as the official assessment has claimed, according to experts.
Two members of the Royal Statistical Society’s working group looking at the accuracy of Covid tests have questioned the results and the way they have been evaluated.
Prof Jon Deeks from Birmingham University and Prof Sheila Bird, formerly of the MRC Biostatistics Unit at Cambridge University, say the tests perform poorly where people have low levels of the virus, which is often the case in people without symptoms.
They say the discrepancy in the figures is because the evaluation used “spiked” samples – saliva to which the virus has been added in the lab. Those manufactured samples were picked up efficiently by the test, but “real world” samples from people with asymptomatic Covid were not.
The OptiGene RT-Lamp test, which can be used with saliva samples or nose swabs, was intended to be part of the Liverpool mass testing programme, routinely offered to asymptomatic NHS workers. People in the community have been given lateral flow tests, which involve nose and throat swabs. Both tests have the advantage of speed, with results within half an hour.
The accuracy of the Lamp tests came into question last month, however, when a letter emerged from scientists with Greater Manchester’s mass testing expert group (MTEG). They said they found only 47% accuracy and warned that the tests should not be widely used in care homes and hospitals.
But the government says that the Lamp tests passed an evaluation with 79% overall accuracy, which included high and low virus levels using both swabs and saliva samples. Most of the testing took place in a pilot in Southampton, where 55,000 people in the NHS and at the university took part.
But Deeks said the data showed that spiked samples had been used to increase the number of samples where the virus levels were low. Using laboratory-made “spiked” samples did not reflect performance in the real world and meant that some people would wrongly think they were in the clear when they had the virus, he said.
“Before the government decides that these tests are to be used in the population, it is really important that we find out how well they work in people like us. This study has mixed together real clinical data with data from samples that have been made in the laboratory. This has led to its performance on saliva looking better than it is, and wrongly suggests that it can detect Sars-CoV-2 in samples with less virus in them,” he said.
Bird said tests should be tried out in the context in which they are going to be used. Early on in the development of a test, she said, you would use spiked samples to assess how well it was doing. “Then you gradually move closer and closer to the real world and it is the real world that matters. If you are using the test very widely and helping people to make decisions about their own lives and the lives of their relatives, you want to know about its performance in the context of use.”
A Department of Health and Social Care spokesperson said: “With up to a third of individuals with Covid-19 not displaying symptoms, broadening testing to identify those showing no symptoms and who can infect people unknowingly will mean finding positive cases more quickly and break chains of transmission.
“The use of spiked samples is a perfectly valid and standard practice for a study of this sort; it is in accordance with the guidelines laid down by the Technical Validation Group. The inclusion of spiked samples does not change the conclusions made”.
OptiGene was approached for comment but made no response.
[link url="https://www.theguardian.com/world/2020/dec/04/experts-question-claimed-accuracy-of-covid-19-saliva-tests"]Full The Guardian report[/link]