Comments and Questions

Paldam provides a nice, sharp and informative introduction to meta-analysis. Overall, the introduction is very well written. I found some statements in the paper that could be reassessed or qualified. On page 2, Paldam states that regressions are cheap in economics. They are certainly cheaper than running clinical trials but more and more economists are conducting field and other experiments, some of which are rather expensive. And even though medical trials are more expensive, the analysis of the trials is also vulnerable to various research biases. Paldam’s overall point, however, is valid. Also on page 2 Paldam refers to simulation studies that have been conducted. Some of these are actually problematic in the way they have been constructed. There remains a need for a comprehensive assessment of these studies. This is an area that warrants further attention by meta-analysts. Page 4, the second point in table 1 was a bit unclear: what is meant by the expression “it is actually true”? Further on this page the term cp-controls is unclear (I assume it refers to ceteris paribus controls). There are other such terms used in the paper, and they may not all be helpful to readers new to meta-analysis: though this comment is probably more of an issue of style. Paldam devotes the paper to the FAT-PET MRA. This is actually only one strand/framework/methodology. There are others. While I share Paldam’s enthusiasm for the FAT-PET, readers should be aware that there are other approaches to conducting MRA. On page 8 there is an important discussion on ‘top journals’. My own suspicion is that ‘top’ journals will tend to publish larger effects – they not only innovate, they inflate!