Readers of Epidemiology (the journal) will be aware of the request to pre-register not only randomized trials but also observational studies. This request comes from the industry sponsored European Center for Ecotoxicology and Toxicology of Chemicals (1) and some journals have already adopted the rule. We are pleased to see Epidemiology will not be among them (2) and a number of invited authors add important points to the discussion (3-6). The BMJ had asked Sørensen and Rothman to write an editorial on the topic and in spite of their warnings BMJ will actively support the registration of observational studies (BMJ 2010; 340: 374-376). We accept that such a pre-registration may make sense for studies designed to provide specific documentation or to be used to make a specific decisions. Registration of randomized trials and animal studies used to examine toxicity of specific exposures is probably a good idea..
In research designed to identify new causes of diseases or ill health one should encourage an open mind approach and creative thinking with as few rules as possible. We see no scientific reasons to put more constraints on research than necessary and we find it difficult to understand why these constraints come from epidemiologists themselves. We do advocate openness and we do recommend that collected data are made available for other researchers.
Pre-registration of protocols is meant to reduce the risk of data dredging by specifying prior hypotheses, to make it more transparent what were prior hypotheses and what emerged from “fishing expeditions”, to separate “hypothesis testing” and “hypothesis generation”, a terminology related to the unfortunate P-value obsession that has done much harm and little good over the years. Rather than basing our influence on modifications of prior beliefs in the light of data we should then accept that data somehow depend on how long and how intense we have thought about the hypothesis. It makes little sense and will provide no guarantee against a dishonest or uncritical reporting of data. Open external access to the data source is a much better safeguard against such a misuse of science.
The fear is that epidemiologists start studies collecting data on a bunch of more or less justified “exposures” of potential interest, run statistical analyses on these all by all analyses and only present associations that reach statistical significance (P<0.05), or that only significant results are publishable. Except for the GWAS studies this scenario is far from reality. Identified associations are subject to critical analyses and interpreted in the light of prior evidence. The aim is to see if we can make associations go away, to see how robust they are to sensitivity analyses.
Epidemiologic research is heavily regulated by ethic committees, data protection agencies, funding organization and more. There is no need to add to that list and we are also skeptical towards standardizing reporting of results (The STROBE Statements, another “rapid response’ comment). We need more variation in what we study and how we do it. In that process we will make mistakes. These mistakes will, if they are important, be corrected with time. Making research more difficult may not prevent many of those mistakes but may prevent the detection of them.
Jørn Olsen, Shah Ebrahim, Cesar Victora, Neil Pearce
1. Workshop: Enhancement of the scientific process and transparency of observational epidemiology studies. 24-25 September 2009, London. http://links.lww.com/ede/a415
2. Editors. The registration of observational studies – when metaphors go bad. Epidemiology 2010;21(5):607-609.
3. Samet JM. To register or not to register. Epidemiology 2010;21(5):610-611.
4. Lash TL. Preregistration of study protocols is unlikely to improve the yield from our science, but other strategies might. Epidemiology 2010;21(5):612-613.
5. Takkouche B, Norman G. Meta-analysis protocol registration. Sed quis custodiet ipsos custodes? [But who will guard the guardians?] Epidemiology 2010;21(5):614-615.
6. Poole C. A vision of accessible epidemiology. Epidemiology 2010;21(5):616.-617.