Main Page

From ReplicationWiki
Revision as of 16:52, 19 April 2016 by Jan H. Höffler (Talk | contribs)

Jump to: navigation, search
12,504 articles

About the project

For scientific progress, it is pivotal to review research findings by independently replicating results, thus making the findings more reliable. However, in econometric research, it is not yet common practice to publish replication findings. This wiki serves as a database of empirical studies, the availability of replication material for them and of replication studies. It can help teaching replication to students. Seminars at several faculties internationally were already taught for which the information of this database was used.
So far the focus has been on some leading journals in economics. Replication results can be published as replication working papers of the University of Göttingen's Center for Statistics.
Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable. The starting phase of this project was funded by the Institute for New Economic Thinking (INET).

Join the project

You can join the project by entering a new replication or empirical studies that could be replicated. For the latter you can vote which ones you find most relevant to be replicated. Furthermore, you can improve the articles in the wiki and make comments. Under current events you can announce news and upcoming events. The community portal lists open tasks and suggestions for the wiki.

Latest replication


Article How to Make Psychology Studies More Reliable, Ed Yong, The Atlantic, March 31, 2016

Letter "A research symbiont", Benedikt Fecher, Gert G. Wagner, Science, 25 Mar 2016.

Article Evaluating replicability of laboratory experiments in economics, Science, 25 March 2016

Article The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline, Journal of Experimental Social Psychology, available online since 24 March 2016, in press

Article Are results in top journals to be trusted?, S.K., The Economist, Jan 21st 2016

Blog post Secret Data Encore by John Cochrane, January 5, 2016

Secret Data by John Cochrane, December 28, 2015

Replication and Transparency in Economic Research, INET blog post by Jan H. Höffler, December 3, 2015

Workshop on Replication and Transparency in Economic Research at Mozilla San Francisco, January 6-7, 2016, with the Young Scholars Initiative of the Institute for New Economic Thinking, Richard Ball of the project Teaching Integrity in Empirical Reseach, Johannes Pfeifer, Edward Miguel of the Berkeley Institute for Transparency in the Social Sciences, Jan H. Höffler of the ReplicationWiki, and Thomas Herndon of the Public Debt and Growth replication debate

New working paper Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say ”Usually Not” by Andrew C. Chang and Philipp Li

Critical Finance Review announces Special Replication Issues, September 2015

New working paper On the Need for a Replication Journal by Christian Zimmermann

A session on Replication in Economics will be held during the American Economic Association Annual Meeting, January 3-5, 2016, in San Francisco

The American Journal of Political Science publishes Guidelines for Preparing Replication Files and announces that "submitted replication materials will be verified" by the University of North Carolina’s Odum Institute for Research in Social Science "to guarantee that they do, in fact, properly reproduce the analysis results". March 26, 2015.

The 3ie Replication Programme accepts expressions of interest for its call on replications on HIV prevention until May 31, 2015.

Working paper: "The Meaning of Failed Replications: A Review and Proposal" by Michael Clemens, April 9, 2015 - response: "Replication Talk Costs Lives: Why are economists so concerned about the reputational effects of replications?" by Richard Palmer-Jones, April 24, 2015

Working paper: "Pre Analysis Plans Help (only) when Replications are Infeasible" by Lucas C. Coffman and Muriel Niederle, February 14, 2015

Peer Reviewers' Openness Initiative calls for reviewers to, "beginning January 1, 2017, [...] make open practices a pre-condition for more comprehensive review."

Since during the 2014 Research Transparency Forum of the Berkeley Initiative for Transparency in the Social Sciences we received the valuable feedback that our traffic lights could be misinterpreted as a judgment on quality of the studies we now changed the red one to a warning triangle. Feel free to discuss!

RePEc bibliography on replication started

Article "The Need for Replications" in Public Finance Review by James Alm and W. Robert Reed in special issue with replications

ideas/RePEc now links back to studies of ReplicationWiki

Quality evidence for policymaking: I’ll believe it when I see the replication - 3ie Replication Paper (3ie Replication Programme Candidate Studies List for impact evaluations now integrated in the ReplicationWiki)

Article about unreliable research - The Economist

Paper about Teaching Replication in Quantitative Empirical Economics - updated for Econometric Society European Meeting - 28th August 2014

INET blog: Economics needs replication

Working paper online: Replication in the narrow sense of Banzhaf/Walsh (2008)

Working paper online: Narrow Replication of Ashcraft (2005): Are Banks Really Special?

Personal tools