Researchers from the University of Sydney's multidisciplinary Charles Perkins Centre and faculty of pharmacy reviewed 35 published academic studies of so-called “spin” in biomedical scientific papers – also known as “science hype”. Their findings found more than 26% of papers identified as systematic reviews or meta-analyses contained spin. This figure rose to up to 84% in papers reporting on non-randomised trials.
While spin was variably defined across the 35 studies, a wide variety of strategies to spin results were identified including: making inappropriate claims about statistically non-significant results; making inappropriate recommendations for clinical practice that were not supported by study results; attributing causality when that was not possible selective reporting, such as emphasising only statistically significant or subsets of data in the conclusions; and presenting data in a more favourable light than was warranted, for example writing overly optimistic abstracts, misleadingly describing the study design and under-reporting adverse events.
Of the 35 studies reviewed, 19 examined whether particular factors were associated with the presence of spin – however the factors were considered too wide-ranging and unrelated to draw conclusions.
Most of the factors also focused on the characteristics of the individual scientists, journals or studies rather than broader issues in the sector, said co-author Professor Lisa Bero from the faculty of pharmacy and research group leader of the Charles Perkins Centre's Evidence, Policy and Influence Collaborative. "The contribution of research incentives and reward structures – for example financial and reputational – that rely on 'positive' conclusions in order to publish and garner media attention is yet to be addressed," she said.
"We see an urgent need for further research to determine the institutional or cultural factors that could contribute to such a high prevalence of spin in scientific literature – and to better understand the potential impact of spin on research, clinical practice and policy."
Lead author, Honours graduate Kellia Chiu, said researchers, peer reviewers and editors had a responsibility to remain constantly alert for spin. "The scientific academic community would benefit from the development of tools that help us effectively identify spin and ensure accurate and impartial portrayal and interpretation of results," she said.
"Publishing data alongside multiple interpretations of the data from multiple researchers is one way to be transparent about the occurrence of spin."
Abstract
In the scientific literature, spin refers to reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favourable light. The presence of spin in biomedical research can negatively impact the development of further studies, clinical practice, and health policies. This systematic review aims to explore the nature and prevalence of spin in the biomedical literature. We searched MEDLINE, PreMEDLINE, Embase, Scopus, and hand searched reference lists for all reports that included the measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of reports and their included studies and all spin-related outcomes. Results were grouped inductively into themes by spin-related outcome and are presented as a narrative synthesis. We used meta-analyses to analyse the association of spin with industry sponsorship of research. We included 35 reports, which investigated spin in clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. The nature of spin varied according to study design. The highest (but also greatest) variability in the prevalence of spin was present in trials. Some of the common practices used to spin results included detracting from statistically nonsignificant results and inappropriately using causal language. Source of funding was hypothesised by a few authors to be a factor associated with spin; however, results were inconclusive, possibly due to the heterogeneity of the included papers. Further research is needed to assess the impact of spin on readers’ decision-making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research in order to ensure accurate interpretation and dissemination of research.
Authors
Kellia Chiu, Quinn Grundy, Lisa Bero
[link url="http://sydney.edu.au/news-opinion/news/2017/09/12/don_t-believe-the-hype–science-spin-prevalent–researchers-warn.html"]University of Sydney material[/link]
[link url="http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2002173"]PLOS Biology abstract[/link]