A Method to Detect Residual Confounding in Spatial and Other Observational Studies

Abstract

Background

Residual confounding is challenging to detect. We recently described a method for detecting confounding and justified it primarily for time-series studies. The method depends on an indicator with two key characteristics: (1) it is conditionally independent (given measured exposures and covariates) of the outcome, in the absence of confounding, misspecification and measurement errors; and (2) like the exposure, it is associated with confounders, possibly unmeasured. We proposed using future exposure levels as the indicator to detect residual confounding. This choice seems natural for time-series studies because future exposure cannot have caused the event, yet they could be spuriously related to it. A related question – addressed here – is whether an analogous indicator can be used to identify residual confounding in a study based on spatial, rather than temporal, contrasts.

Methods

Using directed acyclic graphs, we show that future air pollution levels may have the characteristics appropriate for an indicator of residual confounding in spatial studies of environmental exposures. We empirically evaluate performance for spatial studies using simulations.

Results

In simulations based on a spatial study of ambient air pollution levels and birth weight in Atlanta, and using ambient air pollution one year after conception as the indicator, we were able to detect residual confounding. The discriminatory ability approached 100% for some factors intentionally omitted from the model, but was very weak for others.

Conclusion

The simulations illustrate that an indicator based on future exposures can have excellent ability to detect residual confounding in spatial studies, although performance varied by situation.

Residual confounding is difficult to detect, in part because assessment must be based on causal considerations. It is not enough to model statistical associations accurately, as associations may not mirror causal patterns.¹

We have previously proposed a method for detecting residual confounding or other forms of model misspecification on the tenet that a cause must precede its effect.^2,3 The method utilizes a variable with two key characteristics. The variable should be independent of disease, absent confounding or other misspecification, and it should be associated with the exposure and confounding covariates. Emphasizing time-series studies, we previously argued that future levels of many environmental exposures can often have the needed characteristics: future exposures cannot have caused past disease, and so an association with prior disease can be spurious.

Using future exposure variables in time-series studies to detect temporal confounding is intuitively attractive – future exposures cannot have caused prior health events and yet could be spuriously related to the event. A related question is whether an analogous variable can be used in spatial studies to identify residual confounding, such as in studies where region-specific pollutant concentrations are correlated with region-specific disease rates. Risk factors such as smoking could co-vary with pollutant concentrations across regions and thus lead to spatial confounding. Although the nature of confounding differs between spatial and time-series studies,⁴ the previously proposed method for detecting residual confounding can be similarly applied in spatial studies. We present simulation results that evaluate the method’s performance and discuss its strengths and weaknesses.

Methods

To illustrate the approach for spatial studies, we consider a study whose objective is to assess the association of area-specific air pollution levels (e.g., ambient ozone, AP₀ in the directed acyclic graph^5,6 in Figure 1) with the birth weight of newborn infants in the same area (D₁ in Figure 1). An important presumption is that birth weight (D₁) does not affect future pollution levels (AP₂), reflected by the absence of an arrow from D₁ to AP₂. Furthermore, spatial factors that affect health and are associated with future pollution levels (e.g., poverty) should often also be associated with earlier pollutant levels, illustrated by U₀ (Figure 1). If these relationships hold, we argue in the eAppendix 1 (http://links.lww.com) – and previously for time-series studies³ – that, conditional on AP₀, future air pollutant levels should not be associated with disease in the absence of confounding (Figure 2), but associated in its presence (Figure 1). In this case, future pollutant levels have key characteristics needed for an indicator of unmeasured confounding^2,3 and can be used to detect it or other model mis-specification.

Figure 1.

Directed acyclic graph showing basic causal relationships among exposure (air pollution, AP₀), the health outcome (birth weight, D₁), an unmeasured covariate (U₀) and future air pollution (AP₂). Confounding by U₁ is present, indicated by the backdoor path from AP₀ to D₁.

Figure 2.

Directed acyclic graph showing basic causal relationships among exposure (air pollution, AP₀), the health outcome (birth weight, D₁), an unmeasured covariate (U₀) and future air pollution (AP₂). Confounding by U₁ is absent, as there is no open backdoor path from AP₀ to D₁.

The model used to analyze the study should include the exposure of interest (air pollutant level prior to outcome occurrence, AP_a,i) and relevant covariates. Equation (1) illustrates a linear form:

$$\mathrm{E}({\mathrm{Y}}_{\mathrm{a},\mathrm{i}})=\mathrm{\alpha }+{\mathrm{\beta }}^{*}\mathrm{A}{\mathrm{P}}_{\mathrm{a},\mathrm{i}}+{\mathrm{\gamma }}^{*}{\text{covariates}}_{\mathrm{a},\mathrm{I}}$$ (1)

For infant i in area a, E(Y_a,i) is the expected birth weight and AP_a,i is a relevant, prenatal air pollutant level.

To use future area-specific air pollutant levels as an indicator, we also fit the same model that now also includes the indicator variable (area-specific, pollutant level measured after infant i's birth, say AP_a,i^f):

$$\mathrm{E}({\mathrm{Y}}_{\mathrm{a},\mathrm{i}})=\mathrm{\alpha }\text{+}{\mathrm{\beta }}^{*}\mathrm{A}{\mathrm{P}}_{\mathrm{a},\mathrm{i}}+{\mathrm{\gamma }}^{*}{\text{covariates}}_{\mathrm{a}}+{\mathrm{\delta }}^{*}\mathrm{A}{{\mathrm{P}}_{\mathrm{a},\mathrm{i}}}^{\mathrm{f}}$$ (2)

If residual confounding and model mis-specification are absent and the assumed causal relationships are adequate approximations, AP_a,i^f should be unassociated with Y_a,i after adjustment for covariates. An association between AP_a^f and the outcome (δ ≠ 0) suggests residual confounding or other model misspecification. Thus, we use the statistic I to test for residual confounding:

$$I=\ddot{\delta }/{\widehat{\sigma }}_f,$$ (3)

where δ̂ is the estimated slope and σ̂_f its standard error. Under the null hypothesis of no model misspecification, I is approximately normally distributed.

Simulations

We assess the indicator’s ability to detect residual confounding using data from a spatial study of ambient air pollution and birth weight in Atlanta. Use of simulations allows us to specify the “true” causal relationships; using estimated parameters to calculate the “true,” expected birth weights makes the simulations realistic. We consider two ambient pollutants commonly assessed in the birth outcomes literature, PM_2.5 (micrograms per meter³) and NO₂ (parts per billion, ppb). We assess relationships between pollutant levels during the first month of pregnancy and birth weight of full-term infants. Zip code-specific levels for each pollutant were a weighted average estimate.⁷

Each full-term infant (37 to 43 weeks’ gestation) was assigned to the zip code of the mother’s residence. We calculated the ambient air pollution level for each infant’s zip code, averaged over the four weeks following the estimated conception date. Modeled covariates included: gestational age (weekly indicators); maternal education, age (linear spline with three knots), tobacco use (yes/no), Medicaid (yes/no), and race/ethnicity (non-Hispanic white, African American and Hispanic); and the zip code’s percent of population below poverty. We included indicators for date-of-conception in two-week intervals, so comparisons were spatial.

We also calculated the air pollution level for each zip code averaged over the four-week period beginning one year after the conception date. Because these exposures occurred after birth (8+ weeks), they cannot have affected birth weight. Future levels (AP_i,a^f) are included only in models that also include pollutant levels prior to birth and a subset of the covariates.

We used the model in Equation (1), with zip code as the area. We fit this model (once) to the observed birth weights to obtain model-predicted weights for each infant, and then treated the model-predicted weights as the true expected value in the simulations. To assess the indicator’s ability to detect confounding, we first generated a birth weight for each infant, using the “true” predicted values and adding a random Gaussian error. We fit the correct model including the indicator, and then, to simulate confounding, we fit the misspecified model (incorrectly omitting a factor) along with the indicator. In most scenarios, we omitted an actual, measured factor (e.g., smoking). In a few situations, we created a hypothetical factor and included it when fitting the model to obtain alternative “true” predicted weights. We subsequently omitted the variable to simulate additional patterns of confounding. We calculated the proportion of simulations in which I exceeded 1.96 in absolute value, rejecting the null hypothesis of no confounding. We also calculated the area under the receiver operating curve as a measure of discriminatory ability. We include a program for simulating the power to detect model misspecification due to omission of a confounder (eAppendix 2, http://links.lww.com). The user can either specify parameters to generate data hypothetically, or use actual observations to fit a model and base simulations on the fitted parameters.

Results

PM_2.5 was negatively associated with birth weight in these spatial analyses (Table 1, Scenario 1; β̂ = 20.6/grams/10 µg/m³). Compared with the “true” model that generated the data, improperly omitting various variables led to varying degrees of simulated confounding. β̂ changed by about 70% when age was omitted and by 700% when race was omitted (Table 1, column 3). The indicator’s ability to detect simulated confounding also varied substantially. For the situations considered, the indicator’s ability was weak when confounding was weak to modest, as when age or tobacco was omitted (AUC = 0.51 – 0.56, Table 1, column 5). However, this was sample-size dependent: with quadrupling of the sample size, the ability to detect confounding (created when the poverty variable is incorrectly omitted increased from 19% (Table 1) to over 50% (data not shown). With stronger degrees of simulated confounding, the indicator consistently signaled that confounding might be a problem (e.g., scenarios 5 and 6; AUC = 0.93 –1.00, Table 1).

Table 1.

Simulation Results. True (connect) Model^a stipulates An Effect of Exposure (PM_2.5) on Birth Weight

Scenario	Median β̂b (SE (β̂))	Percent Bias	Median ( I )	Proportion of Simulations in which Null is Rejected (Ho: δf= 0)	AUC
1 (correct model)a	−20.6 (17.2)	−	−0.01	0.050	0.50
2 (drop tobacco)	0.0(17.3)	100	0.64	0.086	0.56
3 (drop age)	− 5.8 (17.3)	72	0.19	0.042	0.51
4 (drop percent below poverty)	−50.6 (17.1)	146	−1.06	0.184	0.65
5 (drop Medicaid, tobacco, age, education)	61.4 (17.4)	398	1.85	1.00	0.93
6 (Drop Race)	−168. (17.3)	716	−5.48	1.00	1.00

^aCorrect model includes indicator for estimated date of conception (2-week intervals), gestational age (weeks), maternal tobacco use, education, age, and Medicaid status, and zip-code-specific percent below poverty line and PM_2.5 (see text).

^bβ̂ is the estimated change in birth weight (gms), per 10 ug/m³ change in PM_2.5.

AUC indicates the area under the receiver operating curve; it is 0.5 in the absence of discriminatory ability and 1.0 for perfect discriminatory ability

Simulation results were similar for NO₂ (Table 2), and also when we considered confounding by the hypothetical factors (Table 3). While the ability to detect confounding again varied, the indicator consistently signaled possible residual confounding with these sample sizes when the degree of simulated confounding was moderate to strong (e.g., when race or several variables were omitted (Table 2, scenarios 5 and 6).

Table 2.

Simulation Results. True Model^a (correct) stipulates An Effect of Exposure (NO₂) on Birth Weight

Scenario	Median β̂b (SE (β̂))	Percent Bias	Median (| I |)	Proportion of Simulations in which Null is Rejected (Ho: δf= 0)	AUC
1 (correct model)a	−12.1 (8.0)	−	−0.01	0.052	0.50
2 (drop tobacco)	−6.90 (8.0)	44	0.28	0.054	0.51
3 (drop age)	− 8.9 (8.0)	26	0.07	0.048	0.50
4 (drop percent below poverty)	−18.4 (8.0)	52	−0.55	0.106	0.55
5 (drop Medicaid, tobacco, age, education)	7.9 (8.2)	165	1.27	0.224	0.68
6 (Drop Race)	−48.2 (8.0)	298	2.00	0.521	0.86

^aCorrect model includes indicator for estimated date of conception (2 week intervals), gestation age (weeks), maternal tobacco use, education, age, and Medicaid status, and zip code-specific percent below poverty line and NO₂ (see text).

^bβ̂ is the estimated change in birth weight (gms), per 20 ppm change in NO₂.

Table 3.

Simulation Results. True (correct) Model^a stipulates An Effect of Exposure (PM_2.5) on Birth Weight.

Hypothetical Factors Cov1 – Cov3 were generated, with specified effects and correlations.

Scenario	Median β̂b (SE (β̂))	Percent Bias	Median (I)	Proportion of Simulations in which Null is Rejected (Ho: δf= 0)	AUC
1 (correct model)	−20.9 (17.2)	−	−0.01	0.050	−
2 (drop Covariate 1c)	−10.0 (17.4)	3	0.35	0.056	0.52
3 (drop Covariate 2c)	31.2 (18.9)	49	1.59	0.345	0.79
4 (drop Covariate 3c)	78.1 (20.8)	274	2.08	0.690	0.93

^aCorrect model includes indicator for estimated date of conception (2 week intervals), gestation age (weeks), maternal tobacco use, education, age, and Medicaid status, and zip code-specific percent below poverty line, PM_2.5 and hypothetical variables Covariates 1–3, with progressively stronger effect on birth weight (see text and footnote 4, below).

^bβ̂ is the estimated change in birth weight (gms), per 10 ppm change in PM_2.5.

^cβ for Covariates 1, 2, and 3: 50, 200, and 300 respectively.

Discussion

We extend a method to detect important residual confounding^2,3 by describing and evaluating the method for spatial studies. The ability to detect residual confounding was excellent for some scenarios, such as when race was intentionally omitted. As with any statistical technique, the ability to detect residual confounding improves with stronger confounding and larger sample size. We omitted measured variables, such as race, merely to illustrate possible scenarios based on relationships of real factors. In actual applications, the factor creating confounding, if any, could be completely unrecognized and unmeasured. Although few researchers would omit race from a study of air pollution and birth weight, an investigator could conceivably be unaware of, and therefore omit, some other factor that affected air pollution and birth weight in a way similar to race.

The validity of this approach depends on the assumptions. False-positive indications could arise if, for example, a factor affected both the outcome and future exposures but not the exposure of interest. Our simulations suggest that the method can discriminate situations where residual confounding is present from those where it is not, although the strength of this discrimination ability varies according to the situation.