We have been monitoring outcome switching in five top journals, and writing letters to correct the record wherever we have found misreporting. You can read more about our project here, here and here. One peculiar response has been: “you’ve found so much misreporting, in so many trials! Your findings cannot be credible!”. This argument seems to have been used, for example, by Annals in their responses here and here (comment #2) saying they will not engage with our letters pointing out their misreporting.

So are our findings exceptional? Are we the only people to have found a problem? No. The phenomenon of academic journals permitting outcome switching has been studied at length, and it is now extremely well documented. The only thing we have done, which seems to have solicited some odd responses, is correct the record on individual trials, rather than simply publish overall prevalence figures.

For the avoidance of any doubt, here is a walk through the some of the recent literature on the prevalence of outcome switching. [Read more…]

This is a long post on a response we have received from the editors of Annals of Internal Medicine, one of the world’s leading medical journals. We hope you will read it, as their approach to this issue raises some extremely important issues.

The COMPare project sets out to address the problem of outcome switching in clinical trials. We wanted to move on from simple studies reporting the prevalence of this issue, and instead we are exploring what happens when you try to correct the record on individual trials. From the outset, the key aspect of this work was: how will journals respond? In advance of our first academic publication on the project, consistent with our open data approach, we are now beginning to share the responses we have had from journals. We think they provide important qualitative information on the reasons why this problem has persisted for so long. They are also highly varied: the BMJ, for example, demonstrated best practice by immediately issuing a correction when alerted to misreporting of pre-specified outcomes in their journal. Other journals have not responded in the same way.

This blog post is about a long response to COMPare written by the editors of Annals [1], after we sent a series of correction letters on trials misreported in that journal. The editors have published their own response as a letter in the online edition of the journal, and it will soon appear in the paper edition, alongside some of our letters. We believe their response reveals a set of important misunderstandings about the issue of outcome switching, from the editors of the 4th highest impact factor journal in medicine.

[Read more…]