Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials—ESORT)BMJ 2017; 356 doi: https://doi.org/10.1136/bmj.j917 (Published 14 March 2017) Cite this as: BMJ 2017;356:j917
All rapid responses
We thank Spiegel et al. for their interest in our article . We agree that the inability to account for unpublished trials is a limitation. However, it is difficult to predict whether and how the inclusion of unpublished trials would affect our results. While it is a reasonable assertion that unpublished trials are more likely to have negative study findings, it is less clear what percentage of unpublished trials are registered versus not registered in a trial registry.
Therefore, the inclusion of unpublished studies should be the focus of future research on trial registration and would respectfully disagree with the suggestion by Spiegel et al. that this research question is impossible to answer. Following an initial period of reluctance , research ethics committees have increasingly provided researchers with access to clinical trial protocols. Most recently, the United Kingdom Heath Research Authority has provided access to clinical trial protocols as well as “end of study reports” (equivalent to synopses of clinical study reports) for the Adherence to SPIRIT Recommendations Study (ASPIRE) . This allows for the assessment of trial registration status as well as a determination of whether study findings were “positive”, irrespective of whether the trial results are published in a scientific journal. With these study details, one can determine whether there is indeed an inverse association between trial registration and positive study findings. These findings will be important for informing future efforts to expand or refine trial registration policies.
1 Odutayo A, Emdin CA, Hsiao AJ, et al. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials-ESORT). BMJ 2017;356:j917.
2 Chan A-W, Upshur R, Singh JA, et al. Research protocols: waiving confidentiality for the greater good. BMJ 2006;332:1086–9. doi:10.1136/bmj.332.7549.1086
3 Odutayo A, Copsey B, Dutton S, et al. Characteristics and Dissemination of Phase 1 Trials Approved by a UK Regional Office in 2012. JAMA 2017;317:1799–801. doi:10.1001/jama.2017.1471
Competing interests: No competing interests
We read with great interest the cross sectional study of Odutayo et al. analysing whether trial registration is associated with positive study findings (1). They included all PubMed-indexed randomised controlled trials published in December 2012, providing indexation by November 2013. Among all eligible studies, they found 593 registered and 529 non-registered trials. Although registered trials had a slight trend towards positive study findings, no clear effect was identified, nor was there conclusive evidence with regard to the effect of funding sources. Due to the inclusion of a large sample and careful analyses, this study marks an important contribution.
However, a relevant risk of bias remains unaccounted for. As the authors themselves point out, unpublished studies were not included in the data analysis, because PubMed obviously does not index unpublished findings. Since the most important driver for non-publication of negative research results – i.e. publication bias – was not addressed, the finding of no difference between registered and non-registered trials is hardly surprising. Due to publication bias, positive results will more likely make it into indexed journals no matter whether they are registered or not. The finding of a (small) association between registration and positive study findings may be related to more thorough study planning and preparation of registered trials, especially multicentre trials, typically involving a team of statisticians (and substantial funding). Moreover, multicentre trials often analyse several outcomes, and Odutayo et al. defined a positive study finding if at least one primary outcome was significant (1).
In order to address the very relevant question if a real difference in publication of positive study findings between registered and non-registered trials exists, one would certainly need access to all unpublished data, which is impossible, thus the study aim cannot be addressed. To clarify, we absolutely agree with the authors that trials should be registered and we support the attitude that journals should make trial registration mandatory for article submission. In addition, we underline that all results should be made accessible after trial registration, e.g. if the results do not make it into a journal within a certain period after registration (e.g. 5 years for a single-centre, randomised controlled trial), they should be published in an online database of the trial registry. If trials are registered prior to data collection and their results are published after a certain time interval, this would ensure that the data are accessible to the scientific community. More importantly, editors and reviewers should encourage publication of negative study findings, providing satisfactory quality of trials.
Odutayo A, Emdin CA, Hsiao AJ, et al. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials—ESORT). BMJ 2017;356:j917.
Competing interests: No competing interests