Concealing agreements over climate–conflict results

Do short-term climatic changes increase civil war risk? This question has been subject to much scientific debate, prompted by a pair of PNAS studies that reached different conclusions (1, 2). Now, another paper (3), authored by colleagues of the first study, seeks to reconcile this debate by claiming that the results from the second study are consistent with the first.

Hsiang and Meng are gifted scholars whose advice is worth considering. Unfortunately, their latest contribution (3) misses the opportunity to reconcile disagreements—and acknowledge existing agreements—by not addressing the fundamental issue of the debate: the robustness of empirical findings on the climate–conflict relationship. A recap of Hsiang and Meng and the aforementioned studies by Burke et al. (1) and Buhaug (2) is instructive:

• i)Hsiang and Meng’s replication of Buhaug with their preferred specification (table 2) produced statistically insignificant temperature effects across all five models. Three of the models indicate that, if anything, a negative relationship is more probable. This is in exact agreement with what Buhaug reported.
• ii)Burke et al.’s claim of a robust temperature effect was based on a single, unconventional definition of conflict. Using an identical research design with 10 complementary definitions of conflict, Buhaug (2, 4) found that the temperature effect varies in direction and magnitude and is sensitive to small changes in sample coverage.
• iii)In their response, Burke et al. (5) conceded on these matters and agreed that the temperature effect dissipates with alternative conflict indicators or an update of the original data.

Hsiang and Meng did not find space to deliberate on these facts. Instead, they concentrate on alleged “errors” in Buhaug’s study. First, Hsiang and Meng make the case for joint inclusion of fixed effects and time trends. Although this specification sometimes is appropriate, the challenge is not to obtain the highest possible R² but to arrive at unbiased estimates. Hsiang and Meng’s F test provides little insight into how alternative specifications fare in that regard. Personally, I find it more unsatisfactory to accept Hsiang and Meng’s assumption of a uniform climate effect (i.e., a 1 °C increase has the same influence on civil war risk in all countries at all times) than leaving more of the variance unexplained. Second, Hsiang and Meng advocate standardizing conflict variables and converting logit coefficients into risk ratios. This is a sensible approach if the purpose is to directly compare effect sizes across models. The conflicting findings in the Buhaug study rendered such a calculation of little added value. Last, Hsiang and Meng criticize Buhaug of using the null of no association instead of Burke et al.’s reported estimate as baseline. That is a puzzling allegation that not only defies standard hypothesis testing but also reflects a misinterpretation of Buhaug’s objective: to evaluate whether climate variability robustly affects conflict risk.

To conclude, Burke et al. (1) claim a robust temperature–conflict relationship. Buhaug (2) claims that the temperature–conflict relationship is not robust. Hsiang and Meng (3) conclude that their nonsignificant and inconsistent results “invalidate” Buhaug’s claim while they can “neither confirm nor reject” Burke et al.’s claim. Sorry, guys, but you’ve lost me.