The efficacy of Ivermectin, the drug being unofficially promoted worldwide for treating COVID-19, is again under challenge after a much cited study suggesting the treatment is effective against the virus was withdrawn over “ethical concerns”, writes MedicalBrief.
Meanwhile, also in MedicalBrief this week, the medical risk insurer Medical Protection has warned that any medical practitioner who prescribes Ivermectin “takes on a considerable legal liability … unless the letter of the law has been followed”, while a University of KwaZulu-Natal medical scientist writes that Ivermectin has become a “battle of ideologies on safety and efficacy [that] pits a group of doctors who deal with dying patients every day against bureaucrat academic clinicians”.
Researchers found that the much-cited study from Egypt is riddled with fabrications, inconsistencies and plagiarism. Melissa Davey writes in The Guardian that the preprint study on the efficacy and safety of the anti-parasitic Ivermectin in treating COVID-19, was published on the Research Square website in November. It was led by Dr Ahmed Elgazzar from Benha University in Egypt.
The study claimed to be a randomised control trial, a type of study crucial in medicine because it is considered to provide the most reliable evidence on the effectiveness of interventions due to the minimal risk of confounding factors influencing the results. Elgazzar is listed as chief editor of the Benha Medical Journal, and is an editorial board member, according to The Guardian.
It says the study found that patients with COVID-19 treated in hospital who “received Ivermectin early reported substantial recovery” and that there was “a substantial improvement and reduction in mortality rate in Ivermectin treated groups” by 90%.
But the drug’s promise as a treatment for the virus is in serious doubt after the Elgazzar study was pulled from the Research Square website on Thursday 15 July “due to ethical concerns”. Research Square [a platform that publishes pre-print articles so that research is made public quickly], did not outline what those concerns were.
According to The Guardian, a medical student in London, Jack Lawrence, was among the first to identify serious concerns about the paper, leading to the retraction. He became aware of the Elgazzar preprint when it was passed to him by one of his lecturers for an assignment that formed part of his master’s degree.
He found that the introduction section of the paper appeared to have been almost entirely plagiarised. It appeared that the authors had run entire paragraphs from press releases and websites about Ivermectin and COVID-19 through a thesaurus to change key words.
The data also looked suspicious to Lawrence, with the raw data apparently contradicting the study protocol on several occasions.
“The authors claimed to have done the study only on 18-80 year olds, but at least three patients in the dataset were under 18,” Lawrence said.
“The authors claimed they conducted the study between the 8 June and 20 September 2020, however most of the patients who died were admitted into hospital and died before 8 June according to the raw data. The data was also terribly formatted, and includes one patient who left hospital on the non-existent date of 31/06/2020.”
There were other concerns, according to The Guardian.
“In their paper, the authors claim that four out of 100 patients died in their standard treatment group for mild and moderate COVID-19,” Lawrence said. “According to the original data, the number was 0, the same as the Ivermectin treatment group. In their Ivermectin treatment group for severe COVID-19, the authors claim two patients died, but the number in their raw data is four.”
Lawrence and The Guardian sent Elgazzar a comprehensive list of questions about the data, but did not receive a reply. The university’s press office also did not respond.
Lawrence contacted an Australian chronic disease epidemiologist from the University of Wollongong, Gideon Meyerowitz-Katz, and a data analyst affiliated with Linnaeus University in Sweden who reviews scientific papers for errors, Nick Brown, for help analysing the data and study results more thoroughly.
Brown created a comprehensive document uncovering numerous data errors, discrepancies and concerns, which he provided to The Guardian. According to his findings the authors had clearly repeated data between patients.
“The main error is that at least 79 of the patient records are obvious clones of other records,” Brown told The Guardian. “It’s certainly the hardest to explain away as innocent error, especially since the clones aren’t even pure copies. There are signs that they have tried to change one or two fields to make them look more natural.”
The Elgazzar study was one of the largest and most promising showing the drug may help COVID patients, and has often been cited by proponents of the drug as evidence of its effectiveness.
Meyerowitz-Katz told The Guardian that “this is one of the biggest Ivermectin studies out there”, and it appeared to him the data was “just totally faked”. This was concerning because two meta-analyses or Ivermectin for treating COVID-19 included the Elgazzar study.
Because the Elgazzar study is so large, and so massively positive – showing a 90% reduction in mortality – it hugely skews the evidence in favour of Ivermectin,” Meyerowitz-Katz said.
“If you remove this one study from the scientific literature, suddenly there are very few positive randomised control trials of Ivermectin for COVID-19. Indeed, if you get rid of just this research, most meta-analyses that have found positive results would have their conclusions entirely reversed.”
Kyle Sheldrick, a Sydney doctor and researcher, also independently raised concerns about the paper. He found numbers the authors provided for several standard deviations mentioned in tables in the paper were “mathematically impossible”, that completeness of data suggested possible fabrication, and that there was duplication of patient deaths and data.
Ivermectin has gained momentum throughout Latin America and India, largely based on evidence from preprint studies. In March, the World Health Organization warned against the use of Ivermectin outside well designed clinical trials.
Why was a major study on Ivermectin for COVID-19 just retracted?
Questions about potential data fabrication, plagiarism, and ethical breaches led to the withdrawal of a study that formed a critical component of the pro-Ivermectin case, wrote Jack Lawrence for GRFTR News – a website and discussion forum that counters disinformation and misleading stories, and exposes online grifters. The article was published 15 July 2021.
This is a shortened version of the GRIFTR article.
Grftr News has discovered that the authors of an oft-cited study purporting to show a strong benefit when using Ivermectin in the treatment of COVID-19 may have fabricated their data and plagiarised large portions of their paper.
The preprint paper had provided the most positive evidence for using Ivermectin to treat COVID-19 and had been cited in several other high-profile studies. After Grftr News contacted Research Square (the company hosting the preprint) alerting them of our findings, they swiftly removed the paper citing “ethical concerns.”
The study had formed a crucial piece of evidence in the pro-Ivermectin case and its removal largely destroys the current scientific case for using the drug in COVID-19 care.
Grftr News reached out to the paper’s authors for comment but had not received a reply by the time of publication.
Ivermectin has long been used to treat parasitic worms in humans and sees frequent use for similar purposes in livestock and is very effective for both purposes. In contrast, little good evidence exists to justify its use in treating COVID-19.
Both the US Food and Drug Administration and the European Medicines Agency advise against its use. Meanwhile, the World Health Organization only recommends its use within clinical trials.
To date, much of the evidence supporting Ivermectin’s use in treating COVID-19 has been weak and inconsistent. A few studies have found a positive effect for Ivermectin, with many others finding no effect.
A question of evidence
On the 13 November 2020, six researchers from two Egyptian universities uploaded a paper to the Research Square preprint server titled “Efficacy and Safety of Ivermectin for Treatment and prophylaxis of COVID-19 Pandemic”.
The Elgazzar paper was met with interest by parts of the scientific community thanks to its astonishing results and because it was the largest trial on Ivermectin to date. Despite never passing peer-review or being published in any scientific journal, the Elgazzar study went on to get cited in approximately 30 studies, including several peer-reviewed ones in prominent journals.
According to Gideon Meyerowitz-Katz, an epidemiologist at the University of Wollongong, doctors also used the results to justify treating “hundreds of thousands, if not millions of patients”.
From the start, the Elgazzar study also drew scepticism. A January 2021 briefing from the South African National Department of Health complained about a lack of specific details. It concluded that the paper suffered from a “high risk of bias due to reporting issues”.
Just a series of knife slips?
After its initial publication, the Elgazzar preprint went through two further revisions. The purported authors published their third draft on 28 December 2020. It featured minor corrections and further statistical information but notably also included a link to what the authors stated was a copy of the original data, which Grftr News later obtained.
When opening what the authors claim is their original data, the first thing that any reader notices is that it’s remarkably complete. In many columns data for all patients are fully listed.
The second thing the reader will likely notice is that the original data do not match the author’s public results. In three of the four study arms measuring patient death as an outcome, the numbers between the paper and original data differ (See The Guardian story for details).
The original data provided by the authors suggest that efforts to randomise patients between different groups either failed or was not attempted – despite claims to the contrary by the authors.
Every patient in the severe COVID-19 group receiving standard care was an ICU patient, while the patients with severe disease in the Ivermectin group were mixed between wards and ICU. The experts Grftr News spoke to confirmed this is extremely unlikely to happen by chance.
Following these initial detections, Grftr News provided a copy of the data to science fraud expert Nick Brown and asked him to analyse it. Mere hours later, Brown had already conducted an extensive preliminary analysis and agreed to take a more in-depth look. Brown’s complete findings are posted to his blog: find the link below.
Wherever he looked, Brown found problems, from numbers containing non-numeric characters, confusion about date formats, and – most damningly of all – multiple incidences of data being copied between patients.
Brown also discovered that many numbers between the original data and the paper do not match. In some cases, numbers were off by only a couple per cent, while in others, the numbers were off by over 10%.
Grftr News also reached out to Kyle Sheldrick, a Sydney-based doctor and researcher who had been independently looking at the paper. Sheldrick explained that even when just looking at the results in the paper, problems emerged. For example, numbers that the authors provide for several standard deviations mentioned in tables are mathematically impossible.
Many of the patients who died appear to be duplicates. For example, according to the original data, there were ‘four’ patients with the initials NME, NEM and NES (twice), who were all males aged 51 years old, all suffered from diarrhoea, had the same blood haemoglobin levels, were all diagnosed on the 22 May, and all died on the 29 May 2020. They also all share identical values in at least four other data columns.
At least a further 10 deceased patients also display evidence of being duplicated. As such, duplicates make up around half of the recorded deaths. Although much of the patient data is identical, minor changes exist, further proving that a simple copy and paste error cannot be the cause of the duplicates.
Sheldrick also argued that the completeness of data is further evidence of fabrication, noting that this is incredibly unlikely to happen in real-world conditions. In a Skype call with Grftr News, Sheldrick pointed out further statistical impossibilities.
Brown, too, is convinced that the evidence points in the direction of fabrication. “It seems impossible to me that this data file, with its obvious cloning of substantial numbers of patient records and numerous other inconsistencies, contains a true record of the study of 600 patients that Elgazzar et al claim to have conducted.”
Plagiarism
Grftr News also detected what appears to be significant levels of plagiarism in the Elgazzar paper. With one or two minor exceptions, the entirety of the paper’s introduction appears to be copied from various sources, including several other studies, press releases, and letters to the editor from other journals.
Where the copying is not verbatim, the author’s appear to have employed techniques more commonly used by students to disguise plagiarism, for example, by using synonyms or changing one or two words.
This is how “severe acute respiratory syndrome” becomes “extreme intense respiratory syndrome” in one sentence in the paper, despite the fact that “Severe Acute Respiratory Syndrome” is part of the exact full name of COVID-19, SARS-CoV-2.
Unethical Behaviour?
Seemingly not content with apparently potentially fabricating data and plagiarising their introduction, the authors decided to go for a hat-trick and engaged in some other ethically dubious behaviour too.
One sign of poor research design – though not fraud – was the author’s decision to only register their trial on a clinical trials registry after completing their study and publishing their first draft. Meyerowitz-Katz explained that while this is not optimal it’s still common.
The purpose of registering a trial in advance is to avoid the authors changing the questions or analysis they perform once the trial is complete. Such behaviours would be considered bad practice, but not registering a trial isn’t proof that they occurred.
More problematic is the authors’ decision to provide conflicting information about their trial start date. In their trial registry information, within the paper, and in replies to comments on the paper, the authors claim to have received ethical approval and commenced the trial on 8 of June 2020.
However, according to their original data, the authors recruited and treated several patients before this date. Moreover, almost half of the patients who died during the trial died before this date. Meyerowitz-Katz and Sheldrick confirmed that this is a problem. If the authors started their study before they had ethical approval, this would be a major ethics violation.
Additionally, the authors claim to have conducted their trial on 18-80-year-olds, but the original data contains records for four patients younger than 18. Update: Grftr News has learned that in earlier versions of the paper the authors correctly stated patients were aged 14-80. It is unclear why this was changed in their most recent revision.
Grftr News approached the Benha University Research Ethics Committee and the paper’s authors with several questions regarding these aspects of the study but has received no response.
Grftr News approached Research Square, the company which controls the preprint server the Elgazzar paper had been published on for comment. A day later, on 14 July, a notice appeared on the study stating in full, “Research Square has withdrawn this preprint due to ethical concerns.”
Michele Avissar-Whiting, Research Square’s Editor in Chief, later explained to Grftr News that although previous versions of the preprint currently remain online, the paper’s permanent digital object identifier (DOI) would link to the withdrawal notice.
Now what?
“That almost entirely reverses any argument for using Ivermectin,” Meyerowitz-Katz explained in response to Brown and Sheldrick’s findings because “the Elgazzar study is the single biggest piece of research” conducted on Ivermectin.
He was also keen to rerun the results of two recent meta-analyses that found an overall positive effect for Ivermectin, this time without the Elgazzar data… After excluding the data from the Elgazzar study, he found that the effect for Ivermectin drops significantly with no discernible effect on severe disease.
Meyerowitz-Katz later also reran the analysis while excluding an additional poor-quality study and found that after this Ivermectin showed no effect in treating COVID-19.
Asked whether he thought Ivermectin may still turn out to have some benefits, Meyerowitz-Katz granted that better-designed trials might find some benefit. “But it’s extraordinarily unlikely that Ivermectin will have the sort of benefits being proposed by the groups who want everyone to use it.”
Sheldrick also had strong words to say about the study: “Ivermectin’s not a drug with no side effects.”
The experts Grftr News spoke to agree that the findings made concerning the paper and its data cannot easily be waved away. The Elgazzar paper, its data and its authors are fundamentally suspect.
As Sheldrick explained, “there really isn’t any process now by which this can be corrected. Once a group or at least members of a group have been willing to just wholesale fabricate data, if they now come out with a second data set that’s different, how can we trust that?”
Something rotten in the state of science?
Though the fabrication and plagiarism are egregious and a severe breach of scientific integrity, the paper’s damage would have been more confined were it not for the efforts of a cadre of Ivermectin hypers.
Contrary to the claims of conspiracy-minded thinkers, Ivermectin has seen support from a range of influential political figures.
Ivermectin has also been promoted several times in both Houses of the Parliament of the United Kingdom and at least three times in the Australian House of Representatives by MP Craig Kelly. A cottage industry of pro-Ivermectin advocacy groups has also sprung up, each boasting of their founders medical and scientific credentials.
Even if the paper’s authors end up providing an innocent explanation for all this it would be puzzling why it took them so long to notice their error. Whether the final story is one of purposeful fabrication or a series of escalating mistakes involving training or test datasets, this research group has still screwed up in a big way.
Although science trends towards self-correction, something is clearly broken in a system that can allow a study as full of problems as the Elgazzar paper to run unchallenged for seven months.
Thousands of highly educated scientists, doctors, pharmacists, and at least four major medicines regulators missed a fraud so apparent that it might as well have come with a flashing neon sign. That this all happened amid an ongoing global health crisis of epic proportions is all the more terrifying.
For those reading this article, its findings may serve as a wake-up call. For those who died after taking a medication now shown to be even more lacking in positive evidence, it’s too late. Science has corrected, but at what cost?
GRFTR editor’s note: Since publication Egyptian news outlets have reported that the Egyptian Minister of Higher Education has launched a review into the study…We still accept the authors may provide a satisfactory explanation for the questions raised and look forward to any response they might have.
See also from the MedicalBrief archives
Ivermectin is not a ‘miracle drug’ against COVID. Vaccines are
Ivermectin: Studies come thick and fast; Regulators remain unmoved
Ivermectin significantly reduces COVID-19 mortality risk — FLCC meta-analysis
Ivermectin added to world’s largest clinical trial of possible COVID-19 treatments
Ivermectin climbdown comes with multimillion price tag for SAHPRA and DoH