Discussion Paper
No. 2017-71 | September 28, 2017
Benjamin Douglas Kuflick Wood and Maria Vasquez
Microplots and food security: encouraging replication studies of policy relevant research
(Published in The practice of replication)

Abstract

Replication research is a valuable, yet often misunderstood, tool for increasing our understanding of promising research findings. In their short paper below the authors discuss their principles for conducting replication research, explain how they picked a candidate study for replication, describe the robustness checks they would conduct in their replication plan, and give an overview of how they would interpret their replication results. The authors also describe some of their lessons learned after working in replication research for over five years.

JEL Classification:

Q18, O13

Links

Cite As

[Please cite the corresponding journal article] Benjamin Douglas Kuflick Wood and Maria Vasquez (2017). Microplots and food security: encouraging replication studies of policy relevant research. Economics Discussion Papers, No 2017-71, Kiel Institute for the World Economy. http://www.economics-ejournal.org/economics/discussionpapers/2017-71


Comments and Questions



Garret Christensen, UC Berkeley - Referee report 1
October 11, 2017 - 20:49
This paper describes a plan to replicate Santos et al. (2014)—“Can Government-Allocated Land Contribute to Food Security? Intrahousehold Analysis of West Bengal’s Microplot Allocation Program.” The replication plan is fairly well thought out, and the general strategy seems fine, but I have suggested a few places below where more detail could be useful.Pg 2: I assume “We would then assess the reproducibility of the paper by recoding the original results, the pure replication stage” means recoding from scratch, or from the earliest point possible? Authors should clarify exactly what that means—going back to primary data sources, or just re-writing code, etc.Pg 3: A more complete justification of why this particular paper was selected instead of others on a similar intervention would help. The program evaluated is obviously policy relevant, but as the authors indicate, similar programs have been implemented elsewhere. Were they evaluated, and are there papers on them? If no other papers exist, state that.A brief summary of the analysis conducted in the original paper would be very useful for the readerPg 4: Is "iv) Household used seedlings, seeds, or grafted stems in last year" a good measure of use of improved inputs? Please define these terms, as at least to this non-farmer, it's curious how seeds are considered an improved input since they seem like the required input in essentially all farming.Also, some justification of why the four chosen indicators are the ones chosen from the roughly twenty in the original paper’s Table 3 might be helpful. The authors refer to the theory of change, but a more specific discussion of why the other sixteen are less important could help.Pg 5: I agree the propensity score is a potentially important avenue of researcher degrees of freedom, both for the original researcher and the replicator. Please provide more detail on how this would be investigated.Pg 6: Investigating the adult/child definition sounds good. Please provide a citation or definition of adult equivalency unit.Pg 6: “We plan to clarify with the original authors…” This is the first mention of contacting the original authors. I might be useful to provide general information in the intro section about a plan for interaction with the original authors. To get the original data and code would one have to contact the original authors, or is the data publicly available? (I couldn’t find it in a quick search.) These are important general questions in the area of replication research.Pg 7: The discussion of land deciles seems unclear. I assume decile is referring to some percentile of some distribution, but it is unclear what distribution that is. Or is decile a unit of land measurement? I have never heard of that term, but that seems to be the way you describe it: “households that received 5 or less deciles of land and 10 or less deciles of land.”

Benjamin D. Wood and Maria Vasquez - Reply to Referee Report 1
October 23, 2017 - 10:49
see attached file

Anonymous - Referee report 2
November 08, 2017 - 15:31
see attached file

Anonymous - Reply to Referee Report 2
January 19, 2018 - 06:25
Please see response attached

W. Robert Reed - Decision letter
January 20, 2018 - 19:38
see attached file