Discussion Paper
No. 2019-65 | November 18, 2019
Norbert Hirschauer, Sven Grüner, Oliver Mußhoff and Claudia Becker
Inference in economic experiments

Abstract

Replication crisis and debates about p-values have raised doubts about what we can statistically infer from research findings, both in experimental and observational studies. With a view to the present debate on inferential errors, this paper systematizes and discusses experimental designs with regard to the inferences that can and – perhaps more important – that cannot be made from particular designs.

JEL Classification:

B41, C18, C90

Links

Cite As

[Please cite the corresponding journal article] Norbert Hirschauer, Sven Grüner, Oliver Mußhoff, and Claudia Becker (2019). Inference in economic experiments. Economics Discussion Papers, No 2019-65, Kiel Institute for the World Economy. http://www.economics-ejournal.org/economics/discussionpapers/2019-65


Comments and Questions



Anonymous - Referee Report 1
January 06, 2020 - 08:21
This is a very useful paper, reminding the reader what the requirements are for establishing causation in experiments. The paper also contains a good discussion of external validity. This paper is timely, given the awarding of the 2019 Nobel Prize in Economic Sciences to three pioneers of using randomised controlled trials (RCTs) in development economics. I have only a couple of minor comments on the paper. The discussion of randomised controlled trials versus non-randomised controlled trials, on pages 2 and 3, could give the impression that RCTs do not involve before and after comparisons, when typically they do. The paper correctly makes the point that a low p-value tells us nothing about the external validity of the results. This is presumably also true of studies based on observational data. It would have been useful to include a discussion of why external validity is more problematic (assuming it is) for experimental studies than for studies using observational data.

Sven Grüner - Response
January 09, 2020 - 14:01 | Author's Homepage
We very much appreciate the reviewer’s thorough reading of our manuscript as well as his very thoughtful comments. In the following, we have listed the reviewer’s comments along with our replies to the changes implemented in the manuscript.
[ Download attached file ]

Anonymous - Referee Report 2
January 06, 2020 - 08:31
see attached file
[ Download attached file ]

Sv en Grüner - Reply
January 09, 2020 - 14:02 | Author's Homepage
We very much appreciate the reviewer’s thorough reading of our manuscript as well as his very thoughtful comments. In the following, we have listed the reviewer’s comments along with our replies to the changes implemented in the manuscript.
[ Download attached file ]

Bob Reed - Decision Letter
January 27, 2020 - 08:32
see attached file
[ Download attached file ]