Discussion Paper
No. 2015-33 | May 05, 2015
Peter A.G. van Bergeijk and Muhammad Shahadat Hossain Siddiquee
Bias and Methodological Change in Economic Sanction Reconsidered

Abstract

The authors investigate the influence of case selection and (re)coding for two vintages of a key resource for research on economic sanctions: the Peterson Institute data base reported in Hufbauer et al. (Economic Sanctions Reconsidered, 2nd edition in 1990 and 3rd edition in 2007). The Peterson Institute has not reported transparently on these changes. At the level of individual case studies the authors uncover a tendency to inflate success scores, reclassifying failures into successes even when the evidence for doing so was not convincing. At the level of the aggregated case studies and general methodology they uncover positive bias (that is: methodological changes that make it more likely to find sanction success as indicated by a higher success score, either on average or in individual cases): splitting of episodes into cases and the changed definition of sanction contribution increase the success ratio in general and ultimately the share of sanctions that are judged to be a success. The authors also show the importance of the reclassification of ‘destabilization cases’ into ‘regime changes’. Their probit analysis shows that the 3rd edition’s methodology underestimates the contribution of certain sanction characteristics, including the positive impact of the costs of sanctions to the sender, duration of the sanctions and the sender’s companion policies.

Data Set

JEL Classification:

F51, C82

Cite As

Peter A.G. van Bergeijk and Muhammad Shahadat Hossain Siddiquee (2015). Bias and Methodological Change in Economic Sanction Reconsidered. Economics Discussion Papers, No 2015-33, Kiel Institute for the World Economy. http://www.economics-ejournal.org/economics/discussionpapers/2015-33


Comments and Questions



Anonymous - Referee Report 1
May 21, 2015 - 08:20
see attached file

Peter A.G. van Bergeijk and Muhammad Shahadat Hossain Siddiquee - Response to Referee Report 1
May 27, 2015 - 08:23
Thank you for your comments. 1- We will take care of this in a new version. 2- We will follow this useful comments. 3- We will correct this. 4- Thank you for this suggestion to improve on the structure of our paper.

Anonymous - Referee Report 2
June 11, 2015 - 08:58
see attached file

Peter A.G. van Bergeijk and Muhammad Shahadat Hossain Siddiquee - Response to Referee Report 2
June 19, 2015 - 09:05
Thank you for your valuable and constructive comments.Your point about the need to discuss the fact whether the new definitions are more meaningful than the previous ones is an excellent addition and we will incorporate this in the next version. We disagree on the assessment that the change constitutes an improvement, but your exposition is a good starting point and we will provide a balanced text so that readers can make up their own mind.Your point about the percentages is well taken. We will adjust the text accordingly.We believe that the case studies are important and will expand and provide clear case descriptions. Readers indeed need more help.With respect to your comments on Table 5 we agree that it is possible to check for ‘double counting’ and will include a dummy to test the impact of this factor.

Anonymous - Referee Report 3
July 31, 2015 - 10:10
see attached file