An evaluation of eight rapid diagnostic tests widely used in a variety of African countries by Médecins Sans Frontières (MSF) shows that the tests vary in their performance, with false positive results being a concern. Samples from some geographical locations were more likely to have false positive results than others, suggesting that tests need to be locally validated, researchers report.
The findings confirm that the diagnosis of HIV should not be based on results from a single HIV rapid diagnostic test. A combination of HIV tests, and more specifically an algorithm (sequence) of two or three different HIV rapid tests, is required to make an HIV-positive diagnosis. This is already recommended by the World Health Organisation (WHO).
Misdiagnosing someone as having HIV can cause psychological trauma and profound social effects. Marriages may break down and people may take HIV treatment which they do not need.
Rapid diagnostic tests (RDTs) are the main diagnostic tool for HIV screening and diagnosis in resource-constrained settings. Despite their widespread use, there has been no systematic, head-to-head evaluation of their accuracy with specimens from diverse settings across sub-Saharan Africa. Experience from the field has suggested that, in some settings, some RDTs have sensitivities and specificities that are inferior to those recorded in WHO evaluations.
Venous blood samples were collected from 2,785 people attending six clinics in Guinea, Cameroon, Democratic Republic of Congo, Uganda (two clinics) and Kenya. These samples were frozen and sent to a laboratory in Belgium where they were tested with eight different RDTs. Because all tests were done in one location, it is unlikely that any differences in performance could be attributed to user variation, storage conditions or other methodological factors. Tests were performed and interpreted according to the manufacturer’s instructions, with each test being read by two laboratory technicians separately.
A reference algorithm, using laboratory tests, showed that of the 2,785 samples, 1,474 were HIV negative and 1,306 were HIV positive.
The sensitivity of a test is the percentage of results that will correctly provide a positive result when HIV is actually present. In the evaluation, sensitivities were extremely high: 98.8% for the First Response HIV Card Test 1–2.0: 99.5% or above for the Uni-Gold HIV, HIV 1/2 STAT-PAK and Vikia HIV 1/2: and 100% for the Determine HIV-1/2, Genie Fast HIV 1/2, INSTI HIV-1/HIV-2 Antibody Test and SD Bioline HIV 1/2 3.0.
The high sensitivities mean that it is rare for people to be given false negative results (people being told they are HIV-negative when they, in fact, have HIV) in these settings. Nonetheless, this may still occur when people have very recent (acute) HIV infection.
The other key measure of test accuracy is specificity, in other words, the percentage of results that will correctly provide a negative result when HIV is not present. Lower rates of specificity will produce more false positive results.
Here the results were mixed and sometimes sub-optimal: between 90% and 95% for the First Response HIV Card Test 1–2.0, INSTI HIV-1/HIV-2 Antibody Test, Determine HIV-1/2 and Genie Fast HIV 1/2: between 97% and 98% for the Vikia HIV 1/2, SD Bioline HIV 1/2 3.0 and Uni-Gold HIV – and 99.7% for the HIV 1/2 STAT-PAK.
As a result of the lower specificity, a total of 438 specimens had false-positive results on at least one test. False-positive results were associated with different factors for each of the tests – including being male (INSTI, Vikia and Genie Fast tests), being referred for testing by a clinician (probably as a result of having co-morbidities, relevant to the Determine test) and the geographical location (INSTI, SD Bioline and First Response tests). For some tests in some locations, the odds of a false positive result were ten times greater than elsewhere, but it remains unclear what factors are driving this effect.
The researchers note that these widely used rapid diagnostic tests performed more poorly than in the evaluations conducted for WHO – only one test met the recommended thresholds for RDTs of ≥99% sensitivity and ≥98% specificity. There appear to be geographical and population differences in test performance.
“This publication reports on the performance of individual HIV rapid tests, but not the accuracy of entire HIV testing algorithms,” commented lead author Cara Kosack. “The results underscore the challenges in designing accurate testing algorithms, and the need for local validation to be part of the design process.”
A separate analysis highlights the dangers of HIV treatment programmes not routinely re-testing individuals before they begin treatment. Although re-testing is recommended by WHO, only 2 of 48 national testing policies that were surveyed recommended re-testing before initiating antiretroviral therapy.
The financial costs could be considerable. Researchers created a simple mathematical model to compare the cost of re-testing all HIV-positive people versus the expected cost of providing HIV treatment to misdiagnosed HIV-negative individuals.
In a setting with an HIV prevalence of 1%, initially testing 10,000 people with a series of three RDTs cost $83,000 and the testing algorithm had a specificity of 99.9%. Without re-testing, 9 HIV-negative people would be misdiagnosed as positive and initiated on antiretroviral therapy (ART) for life, costing $58,000 in unnecessary ART costs. However re-testing all those initially diagnosed HIV-positive would only cost $2000.
In a setting with an HIV prevalence of 10%, a two-test strategy is recommended. This would have a specificity of 99.6%. The cost to initially test 10,000 people would be $87,000. Without re-testing, 39 HIV-negative people would be misdiagnosed as positive and initiated on ART, costing $243,000 in unnecessary ART costs. Because of the larger numbers, re-testing people with HIV-positive results would cost more than in the other example, $14,000. The net savings would be $225,000.
In both cases, re-testing would be cost-saving within a few months. But the researchers note that their narrow focus on financial and human resources does not capture a number of other important factors: “the potential ethical, personal, and social consequences of incorrect diagnosis and treatment for an HIV-negative person, the quality-of-life implications of unneeded regular treatment and potential associated toxicities, and the potential undermining effects of misdiagnosis for confidence in the health system more widely.”
Abstract 1
Introduction: Although individual HIV rapid diagnostic tests (RDTs) show good performance in evaluations conducted by WHO, reports from several African countries highlight potentially significant performance issues. Despite widespread use of RDTs for HIV diagnosis in resource-constrained settings, there has been no systematic, head-to-head evaluation of their accuracy with specimens from diverse settings across sub-Saharan Africa. We conducted a standardized, centralized evaluation of eight HIV RDTs and two simple confirmatory assays at a WHO collaborating centre for evaluation of HIV diagnostics using specimens from six sites in five sub-Saharan African countries.
Methods: Specimens were transported to the Institute of Tropical Medicine (ITM), Antwerp, Belgium for testing. The tests were evaluated by comparing their results to a state-of-the-art reference algorithm to estimate sensitivity, specificity and predictive values.
Results: 2785 samples collected from August 2011 to January 2015 were tested at ITM. All RDTs showed very high sensitivity, from 98.8% for First Response HIV Card Test 1–2.0 to 100% for Determine HIV 1/2, Genie Fast, SD Bioline HIV 1/2 3.0 and INSTI HIV-1/HIV-2 Antibody Test kit. Specificity ranged from 90.4% for First Response to 99.7% for HIV 1/2 STAT-PAK with wide variation based on the geographical origin of specimens. Multivariate analysis showed several factors were associated with false-positive results, including gender, provider-initiated testing and the geographical origin of specimens. For simple confirmatory assays, the total sensitivity and specificity was 100% and 98.8% for ImmunoComb II HIV 12 CombFirm (ImmunoComb) and 99.7% and 98.4% for Geenius HIV 1/2 with indeterminate rates of 8.9% and 9.4%.
Conclusions: In this first systematic head-to-head evaluation of the most widely used RDTs, individual RDTs performed more poorly than in the WHO evaluations: only one test met the recommended thresholds for RDTs of ≥99% sensitivity and ≥98% specificity. By performing all tests in a centralized setting, we show that these differences in performance cannot be attributed to study procedure, end-user variation, storage conditions, or other methodological factors. These results highlight the existence of geographical and population differences in individual HIV RDT performance and underscore the challenges of designing locally validated algorithms that meet the latest WHO-recommended thresholds.
Authors
Cara S Kosack, Anne-Laure Page, Greet Beelaert, Tumwesigye Benson, Aboubacar Savane, Anne Ng’ang’a, Bita Andre, Jean-Paul BN Zahinda, Leslie Shanks, Katrien Fransen
Abstract 2
We compared estimated costs of re-testing HIV-positive persons before ART initiation to the costs of ART provision to misdiagnosed HIV-negative persons. Savings from averted unnecessary ART costs were greater than re-testing costs within one year using assumptions representative of HIV testing performance in programmatic settings. Countries should implement re-testing before ART initiation.
Authors
Jeffrey W Eaton, Cheryl C Johnson, Simon Gregson
[link url="http://www.aidsmap.com/Avoiding-false-positives-rapid-HIV-tests-vary-in-their-accuracy-so-need-to-be-used-in-combination/page/3134286/"]Aidsmap material[/link]
[link url="http://www.jiasociety.org/index.php/jias/article/view/21345/html"]Journal of the International Aids Society abstract[/link]
[link url="https://academic.oup.com/cid/article/3751368/The-cost-of-not-re-testing-HIV-misdiagnosis-in-the"]Clinical Infectious Diseases abstract[/link]