The COMPare Trials research project assessed 5 trials published in Annals of Internal Medicine for misreported outcomes – a well-documented and common source of bias. We found that all 5 trials published in Annals over the 6 week period misreported their prespecified outcomes, and so we sent a correction letter on each trial.

We also identified a systemic problem. None of these trials had a publicly accessible protocol dated before trial commencement, so we took the prespecified outcomes from the trial registry entries, and compared these against those published in the report. This is not necessarily a problem: trial registers were set up explicitly for the purpose of preventing selective reporting of outcomes, with support from august bodies such as ICMJE, WHO, and legislation across various territories.

In response, Annals’ editors claimed that the registry entries are often “outdated, vague or erroneous”, and that the discrepancies we found are accounted for by trial protocols that cannot be accessed publicly, but are reviewed by editors behind closed doors. We view this as a very outdated approach, lacking in transparency, and a problematic view of trial registers. [Read more…]