Modeling and data choices sway conclusions about climate-conflict links

In an era of growing concern about the human impacts of climate change, the academic and policy communities are paying increasing attention to the possible link between weather anomalies and violent conflict. Early research papers on the topic by Burke et al. (1) and the reanalysis and extension of their work by Buhaug (2) claim contradictory findings, the first showing a link between increased temperatures and war, and the second—using an expanded dataset and different models—calling these results into question. Hsiang and Meng (3) reexamine the data and argue that the original Burke et al. (1) conclusions are robust and remain a “benchmark” for future studies of the climate–conflict relationship.

Hsiang and Meng (3) compare the coefficient estimates and SEs of Buhaug’s model (2) to the Burke et al. (1) findings (using Seemingly Unrelated Regression after standardizing the dependent variable), arguing that this was the comparison that Buhaug ought to have made rather than comparison with the hypothesis that higher temperature has no effect upon war. For Hsiang and Meng, the Burke et al. findings were published first and therefore represent a standard against which all subsequent research must be compared. However, we believe that the Burke et al. finding is not a “benchmark” in the sense that it is the scientific truth or an objective reality because disciplinary-related modeling decisions, data availability and choices, and coding rules are critical in deriving robust conclusions about temperature and conflict.

The sub-Saharan Africa data used by Burke et al. (1) and the replications are countrywide data and averaged by year. Such a coarse spatial and temporal resolution limits our ability to uncover explanations for any relationships that emerge in statistical analyses. Increasingly in the field of conflict studies, researchers use subnational geographic data for single-country studies and cross-national inquiries. Relying on a fine spatial resolution for the analysis of political violence allows intergroup social dynamics within a country to emerge. In the original Burke et al. data, countries as large as Sudan or Democratic Republic of Congo are single units, with political, economic, and climate characteristics that are uniform across enormous territories. This is a bold and naïve assumption, a view increasingly rejected by most conflict researchers because civil war tends to be concentrated in certain regions (e.g., southern Sudan and eastern Democratic Republic of Congo) (4).

Recently released conflict and climate datasets allow much more nuanced analysis of conflict dynamics than the Burke et al. (1) study, more accurately portraying the process of violence across space and time. Ecological zones (e.g., semiarid, tropical) and land use types (e.g., agricultural, pastoral) can vary dramatically within large countries. Pastoralists, agriculturalists, and government office workers are not affected by temperature anomalies in the same way. We cannot pinpoint these regional and typological differences using coarse country–year data. Although there are some national-level characteristics that undoubtedly influence the outbreak and spread of violent conflict in Africa—the institutional capacity (and will) of states to provide public goods is important—privileging only these country-wide influences is likely to simplify social complexities.

Conclusions about the climate–conflict relationship are also contingent on the assumptions behind the respective statistical analyses. Although this simple fact is generally understood, we stress the disciplinary preferences in modeling decisions. Hsiang and Meng (and their various coauthors) prefer the well-established statistical procedure of fixed effects (FE) for identifying causal relationships, using binary variables for the units of analysis to capture all unobserved influences upon an outcome of interest (here, war). This perspective allows them to isolate the climatic effects and make the case for a consistently significant higher risk of conflict with increased temperature (5). Different estimation strategies have led to some inconsistencies in the broader arena of civil war research (6). Regarding model selection, Hodges (7) has bluntly suggested “this is a messy business, which seems to be determined in practice less by facts than by the department in which one was trained.”

Criticism of the FE approach to understanding cross-national and temporal social dynamics is frequently leveled even if the data units exhibit variation in trends (model intercepts) for the response. Beck and Katz (8) criticize the use of FE dummy variables in the study of international economic and political relations and, similarly, we believe that by excluding all independent variables, Hsiang and Meng (3) are effectively eliminating other known explanations for conflict. Conflict researchers generally prefer a modeling approach that mixes climatic and social predictors. Our published (9) and ongoing work has identified a significant temperature effect in raising conflict risk, but this variable is less effective in predicting conflict than political factors. The conclusions that can be drawn from FE results are of no substantial value for broadly explaining conflict; in controlling for almost everything, a model in such a reduced form actually tells us little.

A random-effects multilevel model (MLM-RE) offers a different mode of thinking about climatic and other nonclimatic conditions that may be related to conflict risk. A properly specified MLM-RE removes the assumption that differences within countries and differences between countries are identical (10). In contrast, the FE approach only models within-effects, as each country is individually examined across its temporal range of data. Climate change may be associated with conflict within a country because large temperature increases can shock social institutions and produce conflict over time, but such temporal trends do not capture the conflict differences between countries of different social and environmental character.

In the traditional MLM-RE specification for these kinds of time-series cross-sectional data, the covariates and residuals are typically correlated because of heterogeneity bias. This approach then results in an omitted variable bias because the within-country and between-country effects are combined in a weighted average with a single coefficient. (FE models do not suffer from heterogeneity bias because they only estimate within effects.) We adopt the formulation suggested by Bell and Jones (10) by adding a between-effect variable for each time-varying covariate, calculated as the country-level mean.

Despite our strong reservations in using coarse resolution country–year data to address the climate–conflict question, we demonstrate the advantages of this alternate multilevel model specification by applying it to two models from Hsiang and Meng (3), selected for their centrality in this debate and for their alternative conflict thresholds. We first modify the published models by replacing country time trends with a single time variable, year (Table 1). This change does not affect the original Burke et al. (1) model, but does slightly alter the precipitation effect in the Buhaug model (2).

Table 1.

Country fixed effects models

	1. Threshold 1000		2. Threshold 25
	Estimate	SE	Estimate	SE
(Intercept)	7.520	7.973	−11.278	9.316
Temp	0.045*	0.024	0.050	0.042
Tempt−1	0.013	0.025	−0.009	0.034
Precip	0.013	0.074	0.117	0.094
Precipt−1	0.027	0.075	−0.019	0.097
Year	−0.005	0.004	0.005	0.005
Conflicts	98		226
σresiduals	0.229		0.305
σ reduction	26.9%		30.0%

Model 1 similar to Burke et al. (1); model 2 similar to Buhaug (2) model 7. 889 country-year observations. Significance code ***P < 0.01, **P < 0.05, *P < 0.1. σ is SD. σ reduction (%) is percentage decrease in the dependent variable (DV) σ compared to the residuals σ: ( σ_DV - σ_res)/σ_DV. SEs are robust country-clustered.

We then apply our estimation to the models (Table 2), noting that the within effects match the original coefficients (models 1 and 3, and models 2 and 5). After adding additional covariates (models 4 and 6), the significant temperature effect in the Burke et al. (1) model disappears, with socio-political variables predicting conflict more effectively than the climate variables. Furthermore, this specification provides additional insights into the between- and within-effects that vary for factors such as political exclusion and prior conflict.

Table 2.

Random effects multilevel models (MLM-RE)

	3. Threshold 1000		4. Threshold 1000 with covariates		5. Threshold 25		6. Threshold 25 with covariates
Fixed Part	Estimate	SE	Estimate	SE	Estimate	SE	Estimate	SE
(Intercept)	9.668	8.248	5.916	6.058	−9.598	10.159	4.144	6.187
Temp B	0.384	1.557	−0.068	1.171	1.129	2.691	0.342	0.233
Temp W	0.045*	0.023	0.026	0.022	0.051	0.041	0.030	0.032
Tempt−1, B	−0.392	1.561	0.061	1.173	−1.131	2.697	−0.342	0.233
Tempt−1 W	0.013	0.024	−0.008	0.023	−0.008	0.033	−0.011	0.033
Precip B	1.420	3.386	0.134	2.560	3.349	7.045	−0.166	0.448
Precip W	0.014	0.072	−0.026	0.060	0.118	0.092	0.050	0.065
Precipt−1 B	−1.456	3.394	−0.154	2.560	−3.395	7.060	0.167	0.451
Precipt−1 W	0.029	0.073	0.018	0.073	−0.017	0.095	−0.049	0.081
Year W	−0.005	0.004	−0.003	0.003	0.005	0.005	−0.002	0.003
Pol. exclusiont−1 B			0.128	0.087			0.020	0.014
Pol. exclusiont−1 W			0.195**	0.088			0.014	0.067
Ln GDP capitat−1 B			0.014	0.014			−0.005	0.005
Ln GDP capitat−1 W			−0.119	0.076			−0.111***	0.041
Post-cold war B			−0.120	0.228			0.090	0.068
Post-cold war W			−0.030	0.030			0.072*	0.039
Conflictt−1 B			0.531***	0.099			0.997***	0.011
Conflictt−1 W			0.271***	0.061			0.443***	0.077
Random Part
Level 2: σ2	0.042		0.010		0.091		0.000
Level 1: σ2	0.055		0.045		0.097		0.073
Country VPC	43.6%		18.4%		48.3%		0.0%
Conflicts	98		98		226		226
σresiduals	0.229		0.208		0.305		0.270
σ reduction	26.9%		33.6%		30.0%		38.0%

Models 3–4 extend Burke et al. (1) model 2, models 5–6 extend Buhaug (2) model 7. Significance codes: ***P < 0.01, **P < 0.05, *P < 0.1. B, between countries; W, within country; σ is SD. VPC is the variance partition coefficient = σ²_country/(σ²_country + σ²_year). σ reduction (%) is the percentage decrease in the dependent variable (DV) σ compared to the residuals σ: (σ_DV - σ_res)/σ_DV. Linear MLM-RE models fit using maximum likelihood, 889 country–year observations. SEs are robust country-clustered.

For those who believe that FE models are the best functional form and that country-year data adequately capture climate effects, the Burke et al. (1) results hold; but for those interested in a nuanced model that offers more substantial explanations for the presence of conflict, alternate specifications that support additional covariates are warranted. It would be a mistake to take away from the Hsiang and Meng piece (3) that it is unnecessary to make further careful inquiry of whether, and more importantly how, climate variability may be linked to conflict in Africa and elsewhere. Data and modeling choices are critical to making reliable and insightful conclusions.