3ie Replication Programme
3ie's Replication Programme was established as a global public good to help improve the quality and reliability of impact evaluation evidence used for policymaking. The programme is designed to highlight the benefits of internal replication of impact evaluations of development studies to the sector and to incentivise the conduct of replication studies of influential, innovative, and controversial impact evaluations of development interventions.
What is replication?
Replication is the most established method of research validation in science. The replications funded by 3ie are internal replications— those that use data from the original study, and possibly existing secondary datasets from the same location, in order to check the validity and robustness of the estimations and recommendations.
This selected bibliography attests to the fact that there are a growing number of replications in economics. For a multidisciplinary list of previous research on the practice of replication, see our replication methodologies bibliography.
Benefits of replication
Where the replications confirm that the impact evaluation findings are valid and robust, the Replication Programme lends additional credibility to these findings for use in policymaking. For impact evaluations revealed to have invalid or non-robust findings, the Replication Programme cautions policy makers in the use of the findings. The more general benefit of the programme, though, is improving the incentives for all impact evaluators to conduct careful analysis leading to credible findings in the first place
Watch Benjamin D K Wood, 3ie Evaluation Specialist for Replication, talk about the importance of replication in international development.
3ie is committed to encouraging open and healthy dialogue between replication researchers and original authors. To support this goal, we have created the 3ie Replication Programme Notification and Communication Policy.
3ie's Replication Publications
3ie’s Replication Paper Series (RPS) is a publication and dissemination outlet intended to highlight and create a defined space for replication studies of development impact evaluations. The replications studies are published regardless of whether the findings support or question the results of the original paper. By encouraging researchers to undertake the replication of research in general, RPS contributes to the public good of improved impact evaluation research practices.
The Replication Studies Status Page allows the development community, and anyone else interested in the pursuit of improving the quality and reliability of evidence for policymakers, to see where 3ie's replication researchers are in the process of their replication studies.
To see a list of replication studies please visit our site for 3ie Completed Replication Studies.
Over the last four decades, financial services for the poor have become a priority in low- and middle-income countries. This has led to the development of multiple financial instruments, including microlending, microsavings, microinsurance, alternative collateralisation, prize-linked savings accounts, and others with different sets of risk profiles and targeted at poor populations in developing countries. Although financial services for the poor have seen significant uptake, questions remain if impacts vary based on population targeted, product design and delivery, risk management practices of the service provider, country of operation, associated skills transfer, and other aspects of the programme design.
3ie has therefore launched a fourth Replication Window: Financial Services for the Poor and is requesting expressions of interest from researchers interested in conducting replication studies.
Interested researchers must select an original paper that is included in the RW4 FSP Candidate Studies List. 3ie will only fund one replication study of each paper listed. This list includes seven highly influential impact evaluations of interventions of financial services for the poor related to mobile money, cash transfers, bank deposits, and other financial service interventions targeted towards underserved and unbanked populations in developing countries.
The deadline for submitting expressions of interest is 23.59 GMT, 2 August 2016. For more information about the application, please read the Request for Expressions of Interest. Additional information about eligibility can be found here.
A list of frequently asked questions about Window 4 is also available here.
Three decades of work on HIV prevention have yielded a series of effective biomedical, behavioural, and structural interventions among specific population groups in small-scale settings and/or in controlled trials. To date, few of these have been adopted at a population level to prevent HIV. Although most of them show clear potential for a major impact in reducing HIV and AIDS and the associated human and economic costs, the importance of the effort to prevent HIV and the magnitude of effort required to scale up any evidence-based intervention require the study results to be carefully reviewed, understood, and confirmed.
3ie received over 10 expressions of interest from researchers to conduct internal replications of influential, innovative, or controversial impact evaluations of biomedical, behavioural, social and structural HIV prevention interventions. This is 3ie’s first replication funding window with a specific thematic focus.
The RW3 team will review expressions of interest on a rolling basis and invite selected researchers to submit full proposals. Any researcher invited to submit a proposal must do so within 45 days of receiving the invitation.
3ie provides funding for impact evaluators to prepare their raw datasets and accompanying codebooks for public use and online release.
3ie’s new Data Preparation and Release Window will provide up to $10,000 to authors of impact evaluations for preparing their raw datasets and accompanying codebooks for public use and online release.
Only authors of impact evaluations which are on the 3ie Replication Programme’s Candidate Studies List, and authors of articles selected for an in-house 3ie replication study, are eligible for these funds.
The window is only available for datasets used in papers published before January 1, 2011. You can direct general enquiries about the programme to email@example.com.
For more information, read the Request for Proposals
Replication studies should be prepared according to the instructions for authors document and submitted to firstname.lastname@example.org. Please refer to the instructions for replication paper series reports for an overview of the requisite style and structure of the final report. All submissions should include the signed cover form listed below.
Upon receipt, the managing editor will review the submission to ensure the replication study: --Is of an impact evaluation included in the Impact Evaluation Repository --Includes measurement and estimation analysis and/or theory of change analysis in addition to the pure replication --Is of acceptable quality and tone, and --Reflects transparent communication between the replication researchers and the original authors.
Assuming the above requirements are satisfied, the managing editor will send the replication study to at least one external referee. The managing editor and editor will also review the submission and may request additional inputs from 3ie’s technical staff, particularly for sectoral expertise.
The managing editor will send compiled comments and revision requests to the replication researchers. The reviewer process iterates until the replication study meets 3ie quality and tone standards. The editor conducts a final approval review before publication. Upon acceptance, each study undergoes professional copyediting and proofreading.
Read our blog
Follow us on Twitter: 3ieimpact
Connect with us on LinkedIn
Follow our RSS Feeds