Only 40% of Psychology Studies and 11% of Preclinical Cancer Studies Successfully Replicate, Yet 88% of Non-Replicable Citations Ignore the Failure
educationeducation0 views
The Open Science Collaboration's landmark 2015 replication of 100 psychology studies found only about 40% produced significant results on replication. In preclinical biomedical research, Amgen scientists could replicate only 6 out of 53 (11%) landmark cancer studies. Yet only 12% of post-replication citations of non-replicable findings acknowledge the replication failure. Why it matters: non-replicable findings continue to be cited as fact and built upon by subsequent researchers, so entire research programs are constructed on foundations that have been shown to be unreliable, so pharmaceutical companies invest billions in drug development pipelines targeting mechanisms identified in unreplicable preclinical studies, so clinical trials fail at enormous cost because the preclinical basis was never solid, so patients are enrolled in trials testing hypotheses that were effectively already disproven. The structural root cause is that the academic incentive structure rewards novelty over verification: replication studies are difficult to publish in top journals, provide no career advancement for the researchers who conduct them, and threaten the reputation of original authors who may serve as reviewers or editors, creating a system where the scientific self-correction mechanism is systematically suppressed.
Evidence
The Open Science Collaboration (Science, 2015) replicated 100 psychology studies and found only ~40% reproduced significant results. Begley & Ellis (Nature, 2012) reported Amgen scientists replicated only 6 of 53 (11%) landmark preclinical cancer studies. A Nature survey (2016) found 70% of researchers failed to reproduce others' experiments and 50% failed to reproduce their own. A 2021 UCSD study found only 12% of post-replication citations of non-replicable findings acknowledge the failure. Ioannidis (PLOS Medicine, 2005) estimated that 'most published research findings are false,' a paper that has been cited over 10,000 times.