Discussion Paper
No. 2017-23 | May 09, 2017
Vincent Vandenberghe
Treatment-effect identification without parallel paths: an illustration in the case of Objective 1–Hainaut/Belgium, 1994–2006

Abstract

Imagine an impoverished region that becomes eligible for a generous transfer programme (the treatment). Imagine difference-in-differences analysis (DiD)—a before-and-after comparison of the income-level handicap—shows that the handicap has risen. Most observers would conclude to the policy's inefficiency. The point made in this paper is that second thoughts are needed, because DiD rests heavily on the validity of a key assumption: parallel paths in the absence of treatment. What is more, when several pre-treatment periods are available in the data, it can easily be assessed and, if necessary, abandoned in favour of more relevant ones.

Data Set

JEL Classification:

C21, R11, R15, O52

Links

Cite As

[Please cite the corresponding journal article] Vincent Vandenberghe (2017). Treatment-effect identification without parallel paths: an illustration in the case of Objective 1–Hainaut/Belgium, 1994–2006. Economics Discussion Papers, No 2017-23, Kiel Institute for the World Economy. http://www.economics-ejournal.org/economics/discussionpapers/2017-23


Comments and Questions



Anonymous - Referee Report 1
September 21, 2017 - 08:00
This was an interesting paper to read. I strongly agree that the “common trends” assumption is often not tenable; and that it can be tested when there is sufficient pretreatment data. I also agree that with sufficient pretreatment data alternative assumptions, such as “common acceleration” can identify a treatment effect. The problem is that this is all, I believe, pretty well known. So I do not see a methodological contribution here (although the issues are laid out nicely). These issues are discussed in standard textbooks. For example, Angrist and Pischke’s well-known “Mostly Harmless Econometrics” from 2008 discusses the common trends assumption and the inclusion of group specific trends to relax that assumption (and cites earlier papers that do this). The Mora & Reggio paper that the author does cite does not really offer any methodological innovation, but does offer a nice taxonomy and nomenclature for alternatives to the common trends assumption in standing difference-in-difference designs (the Mora & Reggio paper has never been published although they did publish a STATA Journal article introducing a software package that implements the various alternatives). So the contribution of this paper would need to lie in the empirical application, rather than the methodical discussion. I am less qualified to judge the contribution here. However, I do have a further concern. It is now widely appreciated that standard inference methods are not appropriate in difference-in-difference (and related) settings where the number of groups are small. There are particular challenges when, as in this case, there is a single treated unit. Methods to deal with these situations have been developed by, eg., Conley and Taber (ReStat,2011) and by Abadie et al. (JASA, 2010). Appropriate methods do not seem to have been applied here.

VIncent Vandenberghe - reply to referees
January 25, 2018 - 18:19 | Author's Homepage
Many thanks for drawing our attention to the fact that economists have for long tried to cope with the issue of non-parallel trends. In that sense, our WP does not innovate. Still, the way the WP copes with non-parallelism is original and deviates from standard practice. What most authors do is to include polynomial (linear, quadratic…) trends among the regressors, and estimate the treatment effect as a once-in-a-time trend shift. In practice that strategy does not work very well, because inter alia the estimation of the trend uses post-treatment data. An extreme case is when sample covers only one period before treatment and many after. Then the trend's estimate relies almost completely on post-treatment developments, and absorbs most of the treatment effect. What is needed is a method that i) uses pre-treatment observations to capture linear or non-linear trend differences, and ii) extrapolates these to compute the treatment effect.What the working paper shows how this can be achieved using a fully-flexible version of the canonical DD equation. See appended file for the full response.
[ Download attached file ]

Anonymous - Referee Report 2
October 23, 2017 - 08:14
see attached file
[ Download attached file ]

Anonymous - reply to referees
January 25, 2018 - 18:33 | Author's Homepage
Like referee #1, you draw our attention to the fact that economists have for long tried to cope with the issue of non-parallel trends in DiD models. What they do, most of the time, is to include polynomial (linear, quadratic…) trends [interacted with treatment dummies] among their regressors, and estimate the treatment effect as a once-in-a-time trend shift. As suggested by Wolfers (2006), the problem with this strategy is that it uses post-treatment observations, and that the treatment outcome takes the form of a once-in-a-time trend shift. An extreme case is when sample covers only one period before treatment and many after. Then the trend's estimate relies almost completely on post-treatment developments, and absorbs most of the treatment effect. What is needed is a method -- like the one we expose and use in the WP -- that i) only uses pre-treatment observations to capture linear or non-linear trend differences, and ii) extrapolates these to compute the treatment. Wolfers, J. (2006), Did Unilateral Divorce Laws Raise Divorce Rates? A Reconciliation and New Results, American Economic Review, 96(5), pp. 1802-1820.