Read the COMPare papers:

  • A prospective cohort study correcting and monitoring 58 misreported trials in real time
  • Qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials

COMPare

Tracking switched outcomes in clinical trials

  • Methods
  • Results
  • Team
    • SPEAKING DATES
  • FAQ
  • Blog

Post-hoc “pre-specification” and undeclared deferral of results: a broken record in the making

January 29, 2016 - by Henry Drysdale 1 Comment

COMPare aims to fix the prevalent and ongoing problem of outcome switching in clinical trials. In short, outcome switching is when trialists report something different to what they originally said they were going to: it increases the risk of false positive results and exaggerated findings, and is therefore a bad thing.

For 6 weeks we assessed every trial published in the top 5 medical journals for discrepancies between pre-specified and reported outcomes. Of 67 trials, we found that 58 (87%) contain misreported outcomes [1].

That was phase 1 of COMPare. We are now in phase 2: blogging on our findings and engaging the journals and authors in public discussion on our specific assessments and their attitudes and policies on outcome reporting. The range of responses this has generated has been fascinating and enlightening; from best practice demonstrated by the BMJ [2], to extremely concerning views expressed and actions taken by Annals of Internal Medicine [3]. We are beginning to see a set of recurring themes in these responses that shed light on the underlying reasons for this prevalent problem; largely based on a fundamental misunderstanding of the importance of pre-specification and correct reporting of outcomes.

[Read more…]

Tagged With: annals

REEACT in the BMJ: Good science in action

January 20, 2016 - by Henry Drysdale 1 Comment

COMPare aims to fix the ongoing problem of outcome switching in clinical trials by assessing individual trials for misreported outcomes, openly sharing our results, and then submitting correction letters to the journals concerned. From the outset, the question on our minds was: how will the journals respond? We’ve now had a range of responses, from a range of journals, and one striking feature is the diversity of approaches to these reporting errors being pointed out. Here is an example of best practice, from the BMJ.

[Read more…]

Tagged With: bmj

Where does Annals of Internal Medicine stand on outcome switching? A detailed response.

January 20, 2016 - by Ben Goldacre 1 Comment

This is a long post on a response we have received from the editors of Annals of Internal Medicine, one of the world’s leading medical journals. We hope you will read it, as their approach to this issue raises some extremely important issues.

The COMPare project sets out to address the problem of outcome switching in clinical trials. We wanted to move on from simple studies reporting the prevalence of this issue, and instead we are exploring what happens when you try to correct the record on individual trials. From the outset, the key aspect of this work was: how will journals respond? In advance of our first academic publication on the project, consistent with our open data approach, we are now beginning to share the responses we have had from journals. We think they provide important qualitative information on the reasons why this problem has persisted for so long. They are also highly varied: the BMJ, for example, demonstrated best practice by immediately issuing a correction when alerted to misreporting of pre-specified outcomes in their journal. Other journals have not responded in the same way.

This blog post is about a long response to COMPare written by the editors of Annals [1], after we sent a series of correction letters on trials misreported in that journal. The editors have published their own response as a letter in the online edition of the journal, and it will soon appear in the paper edition, alongside some of our letters. We believe their response reveals a set of important misunderstandings about the issue of outcome switching, from the editors of the 4th highest impact factor journal in medicine.

[Read more…]

  • « Previous Page
  • 1
  • 2
  • 3

Subscribe here for news and updates on our latest posts!

* indicates required

Archives

  • [+] 2016 (13)
    • [+] May (1)
    • [+] April (3)
    • [+] March (2)
    • [+] February (4)
    • [+] January (3)

Latest tweets

Tweets by @compare_trials

How to cite us

Prior to publication of the first paper on COMPare, please cite this project as:

The COMPare Trials Project. Goldacre B, Drysdale H, Powell-Smith A, et al. www.COMPare-trials.org, 2016.

Or alternatively:

The COMPare Trials Project. Ben Goldacre, Henry Drysdale, Anna Powell-Smith, Aaron Dale, Ioan Milosevic, Eirion Slade, Philip Hartley, Cicely Marston, Kamal Mahtani, Carl Heneghan. www.COMPare-trials.org, 2016.

University of Oxford logo Nuffield Department of Primary Care Health Sciences logo Centre for Evidence-Based Medicine logo

Copyright © 2019 · Bennett Institute for Applied Data Science