Main Page

From ReplicationWiki
(Difference between revisions)
Jump to: navigation, search
(link fix)
([https://www.cascad.tech/ cascad] is launched, the first certification agency for scientific code & data, July 10, 2019)
(47 intermediate revisions by one user not shown)
Line 5: Line 5:
 
{| style="width:280px; border:none; background:none;"
 
{| style="width:280px; border:none; background:none;"
 
| style="width:280px; text-align:center; white-space:nowrap; color:#000;" |
 
| style="width:280px; text-align:center; white-space:nowrap; color:#000;" |
<div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">Welcome to <br />[[Replication in Economics]]!
+
<div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">Welcome to <br />[[Replication in economics|Replication in the social sciences, particularly economics]]!
 
</div>
 
</div>
 
<div id="articlecount" style="font-size:85%;">[[Special:Statistics|{{NUMBEROFARTICLES}}]] articles</div>
 
<div id="articlecount" style="font-size:85%;">[[Special:Statistics|{{NUMBEROFARTICLES}}]] articles</div>
Line 34: Line 34:
 
     <td width="19%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Browse in {{PAGESINCAT:Journal}} [[:Category:Journal|journal]]s,<br/ >{{PAGESINCAT:Working paper series & blogs}} [[:Category:Working paper series & blogs|Working paper series & blogs]] and {{PAGESINCAT:Book}} [[:Category:Book|book]]s'''</td>
 
     <td width="19%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Browse in {{PAGESINCAT:Journal}} [[:Category:Journal|journal]]s,<br/ >{{PAGESINCAT:Working paper series & blogs}} [[:Category:Working paper series & blogs|Working paper series & blogs]] and {{PAGESINCAT:Book}} [[:Category:Book|book]]s'''</td>
 
     <td width="17%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Browse by [[:Category:Article methods and estimation|methods]],  [[:Category:Article software used|software]] used, [[:Category:Article JEL codes|JEL codes]], [[:Category:Article keywords|keywords]], [[:Category:Article authors|authors]], [[:Category:Article data type|data type]],<br/ >[[:Category:Geographical origin of data|geographical origin of data]], and [[:Category:Article data source|data source]]'''</td>
 
     <td width="17%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Browse by [[:Category:Article methods and estimation|methods]],  [[:Category:Article software used|software]] used, [[:Category:Article JEL codes|JEL codes]], [[:Category:Article keywords|keywords]], [[:Category:Article authors|authors]], [[:Category:Article data type|data type]],<br/ >[[:Category:Geographical origin of data|geographical origin of data]], and [[:Category:Article data source|data source]]'''</td>
     <td width="4%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Find one of {{#expr:{{PAGESINCAT:Replication}}+{{PAGESINCAT:Replication of more than one study}}+{{PAGESINCAT:Replication that was also replicated}}-2}} [[:Category:Replication |replication]]s'''</td>
+
     <td width="10%" height="50" align="center" valign="top" bgcolor="#cedef7">'''Find one of {{#expr:{{PAGESINCAT:Replication}}+{{PAGESINCAT:Replication of more than one study}}+{{PAGESINCAT:Replication that was also replicated}}-2}} [[:Category:Replication |replication]]s, {{PAGESINCAT:Corrigendum}} [[:Category:Corrigendum|Corrections]]''' & '''{{PAGESINCAT:Retracted study}} [[:Category:Retracted study|Retractions]]'''</td>
 
     <td width="10%" height="50" align="center" valign="top" bgcolor="#cedef7">'''{{#formlink:form=New_Replication|link text=replication}}''' </td>
 
     <td width="10%" height="50" align="center" valign="top" bgcolor="#cedef7">'''{{#formlink:form=New_Replication|link text=replication}}''' </td>
 
     <td width="13%" height="50" align="center" valign="top" bgcolor="#cedef7">'''{{#formlink:form=New_Replicated_study|link text=replicated study}}''' </td>
 
     <td width="13%" height="50" align="center" valign="top" bgcolor="#cedef7">'''{{#formlink:form=New_Replicated_study|link text=replicated study}}''' </td>
Line 73: Line 73:
 
| style="color:#000; padding:2px 5px 5px" |  
 
| style="color:#000; padding:2px 5px 5px" |  
 
<div id="mp-tfa" style="width:55%; >For scientific progress, it is pivotal to review research findings by independently replicating results, thus making the findings more reliable. However, in econometric research, it is not yet common practice to publish replication findings. This wiki serves as a database of empirical studies, the availability of replication material for them and of replication studies. It can help teaching replication to students. Seminars at several faculties internationally were already taught for which the information of this database was used.<br>
 
<div id="mp-tfa" style="width:55%; >For scientific progress, it is pivotal to review research findings by independently replicating results, thus making the findings more reliable. However, in econometric research, it is not yet common practice to publish replication findings. This wiki serves as a database of empirical studies, the availability of replication material for them and of replication studies. It can help teaching replication to students. Seminars at several faculties internationally were already taught for which the information of this database was used.<br>
So far the focus has been on some leading journals in economics. Replication results can be published as [http://www.uni-goettingen.de/en/425319.html replication working papers] of the University of Göttingen's [http://www.uni-goettingen.de/en/420023.html Center for Statistics].<br/>Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable. The starting phase of this project was [http://ineteconomics.org/grants-research-programs/grants/replication-in-empirical-economics funded by the Institute for New Economic Thinking (INET)].<br/></div>
+
So far the focus has been on some leading journals in economics. Replication results can be published as [https://ideas.repec.org/s/stg/wpaper.html replication working papers] of the University of Göttingen's [http://www.uni-goettingen.de/en/420023.html Center for Statistics].<br/>Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable. The starting phase of this project was [https://www.ineteconomics.org/research/grants/replication-in-empirical-economics funded by the Institute for New Economic Thinking (INET)].<br/><br/>
 +
<big>'''Please cite as:'''</big><br/>
 +
If you mention the ReplicationWiki in your research, especially when you use data from it, do not forget to cite the study describing it: [http://www.dlib.org/dlib/march17/hoeffler/03hoeffler.html ReplicationWiki - Improving Transparency in the Social Sciences], by [[User:Jan H. Höffler|Jan H. Höffler]], ''[http://www.dlib.org/ D-Lib Magazine]'', March/April 2017, Volume 23, Number 3/4, doi: [https://doi.org/10.1045/march2017-hoeffler 10.1045/march2017-hoeffler].
 +
<br/></div>
 
|-
 
|-
 
! style="padding:2px;" |  
 
! style="padding:2px;" |  
 
|}
 
|}
 
| style="border:1px solid transparent;" |
 
| style="border:1px solid transparent;" |
<!--        Latest replication studies      -->
+
<!--        Most recently added replication studies      -->
 
| class="MainPageBG" style="width:50%; border:1px solid #cedff2; background:#f5faff; vertical-align:top;"|
 
| class="MainPageBG" style="width:50%; border:1px solid #cedff2; background:#f5faff; vertical-align:top;"|
 
{| id="mp-right" style="width:100%; vertical-align:top; background:#f5faff;"
 
{| id="mp-right" style="width:100%; vertical-align:top; background:#f5faff;"
Line 85: Line 88:
 
| style="color:#000; padding:2px 5px 5px;" | <div id="mp-dyk">You can join the project by [[Help:New Replication|entering a new replication]] or [[:Help:New Article|empirical studies that could be replicated]]. For the latter you can vote which ones you find [[Top rated studies for replication|most relevant to be replicated]]. Furthermore, you can improve the articles in the wiki and make comments. Under [[ReplicationWiki:Current events | current events]] you can announce news and upcoming events. The [[ReplicationWiki:Community portal|community portal]] lists open tasks and suggestions for the wiki.</div>
 
| style="color:#000; padding:2px 5px 5px;" | <div id="mp-dyk">You can join the project by [[Help:New Replication|entering a new replication]] or [[:Help:New Article|empirical studies that could be replicated]]. For the latter you can vote which ones you find [[Top rated studies for replication|most relevant to be replicated]]. Furthermore, you can improve the articles in the wiki and make comments. Under [[ReplicationWiki:Current events | current events]] you can announce news and upcoming events. The [[ReplicationWiki:Community portal|community portal]] lists open tasks and suggestions for the wiki.</div>
  
<h2 id="mp-itn-h2" style="margin:3px; background:#cedff2; font-size:120%; font-weight:bold; border:1px solid #a3b0bf; text-align:left; color:#000; padding:0.2em 0.4em 0.0em 0.2em;">Latest replication</h2>
+
<h2 id="mp-itn-h2" style="margin:3px; background:#cedff2; font-size:120%; font-weight:bold; border:1px solid #a3b0bf; text-align:left; color:#000; padding:0.2em 0.4em 0.0em 0.2em;">Most recently added replication studies</h2>
 
|-
 
|-
 
| style="color:#000; padding:2px 5px;" | <div id="mp-itn"><DynamicPageList>
 
| style="color:#000; padding:2px 5px;" | <div id="mp-itn"><DynamicPageList>
Line 99: Line 102:
 
|-  
 
|-  
 
| style="color:#000; padding:2px 5px 5px;" | <div id="mp-otd">
 
| style="color:#000; padding:2px 5px 5px;" | <div id="mp-otd">
Article [http://www.theatlantic.com/science/archive/2016/03/save-psychology-by-replicating-studies-before-theyre-published/475983/ How to Make Psychology Studies More Reliable], Ed Yong, The Atlantic, March 31, 2016
+
[https://www.cascad.tech/ cascad] is launched, the first certification agency for scientific code & data, July 10, 2019
  
Letter "[http://dx.doi.org/10.1126/science.351.6280.1405-b A research symbiont]", Benedikt Fecher, Gert G. Wagner, Science, 25 Mar 2016.
+
The ReplicationWiki will be presented at [https://digitalsocieties2019.net/program-events/ digitalsocieties2019, 2nd Annual Conference of the Academy of Sociology (AS), Session on Replication and Publication Bias], Konstanz, Germany, September 25-27, 2019
  
Article [http://dx.doi.org/10.1126/science.aaf0918 Evaluating replicability of laboratory experiments in economics, Science,  25 March 2016]
+
A [https://www.fun-mooc.fr/courses/course-v1:inria+41016+session01bis/about MOOC on Reproducible Research] is offered by the [https://www.inria.fr/en/ French Institute for Research in Computer Science and Automation (INRIA)] in French and English from October 22  to December 19, 2018.
  
Article [http://dx.doi.org/10.1016/j.jesp.2015.10.001 The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline, Journal of Experimental Social Psychology, available online since 24 March 2016, in press]
+
A session ''[http://tinyurl.com/y8kt66ak Openness and Integrity in Methods Teaching and Research] '', including a [http://tinyurl.com/yb5l559b talk that suggests to use the ReplicationWiki], was held at the [https://connect.apsanet.org/apsa2018/ American Political Sciences Association Annual Meeting ], August 30 – September 2, 2018, in Boston, USA.
  
Article [http://www.economist.com/blogs/freeexchange/2016/01/fudging-hell Are results in top journals to be trusted?, S.K., The Economist, Jan 21st 2016]
+
[[3ie Replication Programme|3ie]] and the [https://www.ifad.org/ International Fund for Agricultural Development (IFAD)] are hosting an event on replication and research transparency on [http://www.3ieimpact.org/en/events/external-events/verifying-evidence-policymaking-assessing-financial-services-poo/ Programmes on financial services for the poor: what does the evidence say?] on April 23, 2018 in Rome, Italy.
  
Blog post [http://johnhcochrane.blogspot.com/2016/01/secret-data-encore.html Secret Data Encore by John Cochrane], January 5, 2016
+
The journal [[:Category:Review of Development Finance|Review of Development Finance]] posted a [https://www.journals.elsevier.com/review-of-development-finance/call-for-papers/call-for-papers-reproducibility-studz call for Reproducibility Studies in Development Finance]. They plan to publish them regularly as well as in special issues.
  
[http://johnhcochrane.blogspot.com/2015/12/secret-data.html Secret Data by John Cochrane, December 28, 2015]
+
[http://www.airleap.org/ AIRLEAP] Conference “[http://www.airleap.org/SL_Conference_Flyer.pdf An Urgency for Evidence and Transparency in Economic Analysis and Policy]”, St Louis, USA, October 13th-14th had a [[AIRLEAP 2017 conference session on replication|session on replication in empirical economics]]
  
[http://ineteconomics.org/ideas-papers/blog/replication-and-transparency Replication and Transparency in Economic Research, INET blog post by Jan H. Höffler, December 3, 2015]
+
The journal [[:Category:Energy Economics|Energy Economics]] posted a [http://www.journals.elsevier.com/energy-economics/call-for-papers/special-issue-on-replication-in-energy-economics call for contributions to a Special Issue on Replication in Energy Economics] (in which it mentions the ReplicationWiki). Submission deadline was July 2017.
  
[http://ineteconomics.org/community/events/replication-and-transparency-in-economic-research Workshop on Replication and Transparency in Economic Research] at Mozilla San Francisco, January 6-7, 2016, with the [http://ineteconomics.org/community/young-scholars Young Scholars Initiative] of the [http://ineteconomics.org/ Institute for New Economic Thinking], [https://www.haverford.edu/users/rball Richard Ball] of the project [https://www.haverford.edu/tier Teaching Integrity in Empirical Reseach], [https://sites.google.com/site/pfeiferecon/ Johannes Pfeifer], Edward Miguel of the [http://bitss.org/ Berkeley Institute for Transparency in the Social Sciences], [http://replication.uni-goettingen.de/wiki/index.php/User:J.Hoeffler Jan H. Höffler] of the ReplicationWiki, and [https://ideas.repec.org/f/phe474.html Thomas Herndon] of the [[Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff (CambJE 2014)|Public Debt and Growth replication debate]]
+
[https://www.aeaweb.org/conference/2017/preliminary/1530?sessionType%5Bsession%5D=1&search_terms=replication Replication Session at the American Economic Association Annual Meeting], Chicago, USA, January 6-8, 2017
  
New working paper [http://www.federalreserve.gov/econresdata/feds/2015/files/2015083pap.pdf Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say ”Usually Not”] by Andrew C. Chang and Philipp Li
+
:::::::::::::::::::::'''''[[ReplicationWiki:Current_events|Archive]]'''''
 
+
[[:Category:Critical Finance Review|Critical Finance Review]] announces [http://cfr.ivo-welch.info/RFP.html Special Replication Issues], September 2015
+
 
+
New working paper ''[https://ideas.repec.org/p/fip/fedlwp/2015-016.html On the Need for a Replication Journal]'' by Christian Zimmermann
+
 
+
A [https://www.aeaweb.org/conference/2016/preliminary.php?search_string=replication&search_type=session&association=&jel_class=&search=Search#search_box session on Replication in Economics] will be held during the [https://www.aeaweb.org/conference/2016/ American Economic Association Annual Meeting, January 3-5, 2016, in San Francisco]
+
 
+
The [http://ajps.org/ American Journal of Political Science] publishes [http://ajpsblogging.files.wordpress.com/2015/03/ajps-guide-for-replic-materials-1-0.pdf Guidelines for Preparing Replication Files] and announces that "submitted replication materials will be verified" by the [http://www.odum.unc.edu/odum/home2.jsp University of North Carolina’s Odum Institute for Research in Social Science] "to guarantee that they do, in fact, properly reproduce the analysis results". March 26, 2015.
+
 
+
The [[3ie Replication Programme]] accepts expressions of interest for its call on replications on HIV prevention until May 31, 2015.
+
 
+
[http://www.cgdev.org/publication/meaning-failed-replications-review-and-proposal-working-paper-399 Working paper: "The Meaning of Failed Replications: A Review and Proposal"] by [[Michael A. Clemens|Michael Clemens]], April 9, 2015 - response: "[http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2598464 Replication Talk Costs Lives: Why are economists so concerned about the reputational effects of replications?]" by [[Richard Palmer-Jones]], April 24, 2015
+
 
+
[https://editorialexpress.com/cgi-bin/conference/download.cgi?db_name=ESWC2015&paper_id=2280 Working paper: "Pre Analysis Plans Help (only) when Replications are Infeasible"] by Lucas C. Coffman and Muriel Niederle, February 14, 2015
+
 
+
[https://opennessinitiative.org/ Peer Reviewers' Openness Initiative] calls for reviewers to, "beginning January 1, 2017, [...] make open practices a pre-condition for more comprehensive review."
+
 
+
Since during the [http://www.bitss.org/events/annual-meeting-2014/ 2014 Research Transparency Forum] of the [http://bitss.org Berkeley Initiative for Transparency in the Social Sciences] we received the valuable feedback that our [[ReplicationWiki:Help/Traffic_lights|traffic lights]] could be misinterpreted as a judgment on quality of the studies we now changed the red one to a warning triangle. Feel free to [[ReplicationWiki_talk:Help/Traffic_lights|discuss]]!
+
 
+
[https://biblio.repec.org/entry/ada.html RePEc bibliography on replication] started
+
 
+
Article "[http://pfr.sagepub.com/content/early/2014/09/22/1091142114549229.full.pdf+html The Need for Replications]" in [[:Category:PFR|Public Finance Review]] by [[James Alm]] and [[W. Robert Reed]] in [http://pfr.sagepub.com/content/43/2.toc special issue with replications]
+
<br />
+
 
+
[http://blog.repec.org/2014/07/16/economics-replication-wiki-now-on-ideas/ ideas/RePEc now links back to studies of ReplicationWiki]
+
<br />
+
 
+
[http://www.3ieimpact.org/media/filer/2014/04/11/rps-1-replication-top-final.pdf Quality evidence for policymaking: I’ll believe it when I see the replication - 3ie Replication Paper] ([[3ie Replication Programme Candidate Studies List]] for impact evaluations now integrated in the ReplicationWiki)
+
<br />
+
 
+
[http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble Article about unreliable research - The Economist]<br />
+
 
+
[http://curriculumconference2013.weaconferences.net/papers/teaching-replication-in-quantitative-empirical-economics/ Paper about Teaching Replication in Quantitative Empirical Economics] - [http://www.eea-esem.com/eea-esem/2014/prog/viewpaper.asp?pid=3108 updated for Econometric Society European Meeting - 28th August 2014]<br />
+
 
+
[http://ineteconomics.org/ideas-papers/blog/economics-needs-replication INET blog: Economics needs replication]<br />
+
 
+
[http://www.uni-goettingen.de/en/449472.html Working paper online: Replication in the narrow sense of Banzhaf/Walsh (2008)]<br />
+
 
+
[http://www.uni-goettingen.de/en/449471.html Working paper online: Narrow Replication of Ashcraft (2005): Are Banks Really Special?]<br />
+
 
</div>
 
</div>
 
|}
 
|}

Revision as of 10:04, 10 July 2019

12,504 articles
Slide1.png
Slide2.png
Slide3.png
Slide4.png
Slide5.png
Slide6.png
Slide7.png
Slide8.png
Slide9.png
Slide10.png
Slide11.png

About the project

For scientific progress, it is pivotal to review research findings by independently replicating results, thus making the findings more reliable. However, in econometric research, it is not yet common practice to publish replication findings. This wiki serves as a database of empirical studies, the availability of replication material for them and of replication studies. It can help teaching replication to students. Seminars at several faculties internationally were already taught for which the information of this database was used.

So far the focus has been on some leading journals in economics. Replication results can be published as replication working papers of the University of Göttingen's Center for Statistics.
Teaching and providing access to information will raise awareness for the need for replications, provide a basis for research about the reasons why replications so often fail and how this can be changed, and educate future generations of economists about how to make research replicable. The starting phase of this project was funded by the Institute for New Economic Thinking (INET).

Please cite as:
If you mention the ReplicationWiki in your research, especially when you use data from it, do not forget to cite the study describing it: ReplicationWiki - Improving Transparency in the Social Sciences, by Jan H. Höffler, D-Lib Magazine, March/April 2017, Volume 23, Number 3/4, doi: 10.1045/march2017-hoeffler.


Join the project

You can join the project by entering a new replication or empirical studies that could be replicated. For the latter you can vote which ones you find most relevant to be replicated. Furthermore, you can improve the articles in the wiki and make comments. Under current events you can announce news and upcoming events. The community portal lists open tasks and suggestions for the wiki.

Most recently added replication studies

News

cascad is launched, the first certification agency for scientific code & data, July 10, 2019

The ReplicationWiki will be presented at digitalsocieties2019, 2nd Annual Conference of the Academy of Sociology (AS), Session on Replication and Publication Bias, Konstanz, Germany, September 25-27, 2019

A MOOC on Reproducible Research is offered by the French Institute for Research in Computer Science and Automation (INRIA) in French and English from October 22 to December 19, 2018.

A session Openness and Integrity in Methods Teaching and Research , including a talk that suggests to use the ReplicationWiki, was held at the American Political Sciences Association Annual Meeting , August 30 – September 2, 2018, in Boston, USA.

3ie and the International Fund for Agricultural Development (IFAD) are hosting an event on replication and research transparency on Programmes on financial services for the poor: what does the evidence say? on April 23, 2018 in Rome, Italy.

The journal Review of Development Finance posted a call for Reproducibility Studies in Development Finance. They plan to publish them regularly as well as in special issues.

AIRLEAP Conference “An Urgency for Evidence and Transparency in Economic Analysis and Policy”, St Louis, USA, October 13th-14th had a session on replication in empirical economics

The journal Energy Economics posted a call for contributions to a Special Issue on Replication in Energy Economics (in which it mentions the ReplicationWiki). Submission deadline was July 2017.

Replication Session at the American Economic Association Annual Meeting, Chicago, USA, January 6-8, 2017

Archive
Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox