Read the COMPare papers:

  • A prospective cohort study correcting and monitoring 58 misreported trials in real time
  • Qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials

COMPare

Tracking switched outcomes in clinical trials

  • Methods
  • Results
  • Team
    • SPEAKING DATES
  • FAQ
  • Blog

Archives for February 2016

How did NEJM respond when we tried to correct 20 misreported trials?

February 25, 2016 - by Ben Goldacre 14 Comments

It is well established that academic journals routinely permit outcome switching in the trial reports that they publish, despite public commitments to address this problem. For six weeks from October 2015, the COMPare team analysed every RCT published in the top 5 medical journals to check if they had correctly reported their pre-specified outcomes. Where we found discrepancies between the outcomes that were pre-specified and those reported, we wrote a letter to that journal to correct the record. The responses we have received from journals have been extremely varied: from full public engagement and transparent corrections in the BMJ, to inaccurate and concerning responses in the Annals of Internal Medicine.

New England Journal of Medicine (NEJM) is the highest impact factor journal in medicine. So far we have analysed 23 of their trials. To NEJM’s credit, of 32 primary outcomes pre-specified across these papers, 31 were correctly reported. However, 16 publications added in a total of 65 additional outcomes that were not pre-specified in the protocol or trial registry, and only 2 of these were declared as novel. Furthermore, a total of 160 pre-specified secondary outcomes spread across these publications were either not reported, or were reported incorrectly. Our full results and raw data are available here. Only 3 trials were perfectly reported.

[Read more…]

Tagged With: nejm

Resistance and misunderstanding from the journal editors on outcome switching – What does the FDA say?

February 12, 2016 - by Carl Heneghan 1 Comment

There seems to be some resistance and misunderstanding from journal editors and the academic community on outcome switching, so we thought it would be useful to give an overview of what various authoritative bodies have to say on the topic. In this series, we will cover ICMJE, CONSORT, the FDA, and the Word Of God. Today, we look at the FDA.

[Read more…]

How often are outcomes switched in clinical trials? And why does it matter?

February 5, 2016 - by Kamal Mahtani 2 Comments

We have been monitoring outcome switching in five top journals, and writing letters to correct the record wherever we have found misreporting. You can read more about our project here, here and here. One peculiar response has been: “you’ve found so much misreporting, in so many trials! Your findings cannot be credible!”. This argument seems to have been used, for example, by Annals in their responses here and here (comment #2) saying they will not engage with our letters pointing out their misreporting.

So are our findings exceptional? Are we the only people to have found a problem? No. The phenomenon of academic journals permitting outcome switching has been studied at length, and it is now extremely well documented. The only thing we have done, which seems to have solicited some odd responses, is correct the record on individual trials, rather than simply publish overall prevalence figures.

For the avoidance of any doubt, here is a walk through the some of the recent literature on the prevalence of outcome switching. [Read more…]

Resistance and misunderstanding from the journal editors on outcome switching – What does the ICMJE say?

February 1, 2016 - by Carl Heneghan 1 Comment

There seems to be some resistance and misunderstanding from the journal editor and academic community on outcome switching, so we thought it would be useful to give an overview of what various authoritative bodies have to say on the topic. In this series, we will cover CONSORT, the FDA, ICMJE and the Word Of God. Today, we look at ICMJE.

[Read more…]

How to cite us

Prior to publication of the first paper on COMPare, please cite this project as:

The COMPare Trials Project. Goldacre B, Drysdale H, Powell-Smith A, et al. www.COMPare-trials.org, 2016.

Or alternatively:

The COMPare Trials Project. Ben Goldacre, Henry Drysdale, Anna Powell-Smith, Aaron Dale, Ioan Milosevic, Eirion Slade, Philip Hartley, Cicely Marston, Kamal Mahtani, Carl Heneghan. www.COMPare-trials.org, 2016.

University of Oxford logo Nuffield Department of Primary Care Health Sciences logo Centre for Evidence-Based Medicine logo

Copyright © 2019 · Bennett Institute for Applied Data Science