We have been monitoring outcome switching in five top journals, and writing letters to correct the record wherever we have found misreporting. You can read more about our project here, here and here. One peculiar response has been: “you’ve found so much misreporting, in so many trials! Your findings cannot be credible!”. This argument seems to have been used, for example, by Annals in their responses here and here (comment #2) saying they will not engage with our letters pointing out their misreporting.
So are our findings exceptional? Are we the only people to have found a problem? No. The phenomenon of academic journals permitting outcome switching has been studied at length, and it is now extremely well documented. The only thing we have done, which seems to have solicited some odd responses, is correct the record on individual trials, rather than simply publish overall prevalence figures.
For the avoidance of any doubt, here is a walk through the some of the recent literature on the prevalence of outcome switching. [Read more…]