A comparison of methods to estimate the hazard ratio under conditions of time-varying confounding and nonpositivity

Abstract

In occupational epidemiologic studies, the healthy-worker survivor effect refers to a process that leads to bias in the estimates of an association between cumulative exposure and a health outcome. In these settings, work status acts both as an intermediate and confounding variable, and may violate the positivity assumption (the presence of exposed and unexposed observations in all strata of the confounder). Using Monte Carlo simulation, we assess the degree to which crude, work-status adjusted, and weighted (marginal structural) Cox proportional hazards models are biased in the presence of time-varying confounding and nonpositivity. We simulate data representing time-varying occupational exposure, work status, and mortality. Bias, coverage, and root mean squared error (MSE) were calculated relative to the true marginal exposure effect in a range of scenarios. For a base-case scenario, using crude, adjusted, and weighted Cox models, respectively, the hazard ratio was biased downward 19%, 9%, and 6%; 95% confidence interval coverage was 48%, 85%, and 91%; and root MSE was 0.20, 0.13, and 0.11. Although marginal structural models were less biased in most scenarios studied, neither standard nor marginal structural Cox proportional hazards models fully resolve the bias encountered under conditions of time-varying confounding and nonpositivity.

The healthy worker effect has long been recognized as a potential source of bias when estimating the association between an occupational exposure and health outcome such as mortality.¹ Two aspects of the healthy worker effect can be distinguished from each other: the initial selection of healthy people into the work force; and the tendency for workers at increased risk of mortality to preferentially leave employment. The latter is known as the healthy-worker survivor effect.^2–3 Analytic methods to overcome the bias induced by the healthy-worker survivor effect began to appear in the early 1970s.² In 1986, Robins⁴ identified this bias as one due to confounding of the association between cumulative exposure and mortality by time-varying work status, which is affected by prior exposure. Standard Cox proportional hazards regression models produce biased exposure-disease associations whether or not one adjusts for time-varying confounders affected by prior exposure.^5–10 Marginal structural Cox proportional hazards regression models offer a viable alternative under such conditions.¹¹

In the context of occupational epidemiology, the healthy-worker survivor effect presents complications beyond what is encountered with standard time-varying confounding affected by prior exposure. Exposed persons who leave the workplace often have no chance of incurring work-based exposure at subsequent time points. If work status is a confounder, this situation results in a violation of the positivity assumption. The positivity assumption requires exposed and unexposed observations in all strata of the confounders at all time points.^12–14 Violations of the positivity assumption can arise in diverse research settings.^15–16 To make inferences that are not based on model interpolation or extrapolation, positivity is required.¹⁴

Formally, the positivity assumption, which is also known as the experimental-treatment assignment assumption,¹⁷ is met when Pr (X = x| L = l) > 0 for all l, where Pr(L= l) ≠ 0, X is the exposure variable, and L is a vector of confounders.¹² Violations of the positivity assumption (nonpositivity) are of two kinds: systematic and random. Systematic nonpositivity occurs when individuals cannot receive at least one level of the exposure within one or more of the confounder strata. Random nonpositivity occurs when no persons happen to be observed within one or more of the confounder strata.¹⁴ While both types of nonpositivity can threaten the validity of inferences made with respect to exposure-outcome associations, as a structural feature of the scenario under study, systematic nonpositivity is of greater concern.

Marginal structural models are known to yield unbiased results under conditions of time-varying confounding when the positivity assumption is met.¹¹ However, their performance under conditions of time-varying confounding with systematic nonpositivity (as occurs in the healthy-worker survivor effect) is unknown. Here, we use Monte Carlo simulation to compare the performance of hazard ratios derived from standard and marginal structural Cox models under conditions of time-varying confounding and nonpositivity in data that mimic sample sizes typically encountered in occupational epidemiology.

METHODS

Simulated Data

The Figure presents a causal directed acyclic graph¹⁸ representing the healthy worker survivor effect¹⁹ on which our simulated data are based. Baseline exposure status X(0), and an unmeasured confounder U, were each generated as independent Bernoulli random variables with a probability of 0.5. To mimic a study of an occupational cohort in which people enter follow-up at start of employment, work status at baseline, W(0) was always set to 1; as a constant this variable may be ignored without consequence. Follow-up work status was denoted W(1) and was defined as a Bernoulli random variable with a probability of being at work of Pr[W (1)= 1]= expit[θ+ ln(γ )U − ln(β)X(0)] , where expit(·) = exp(·)/[1+ exp(·)], whereθ represents the intercept (chosen such that the marginal probability of W(1) is approximately 0.5), and whereγ and β represent the log-odds of the association between the unmeasured confounder U and W(1), and between baseline exposure X(0) and W(1), respectively. To induce nonpositivity, follow-up exposure status X(1) was defined as zero if W(1) = 0, and a Bernoulli random variable with a probability of 0.5 if W(1) = 1. Survival time T followed an exponential distribution with a survival function exp(−λt) and a rate parameter of λ= exp [− ln(α) χ+ ln(δ) U ], where α is the log-hazard of the association between the cumulative exposure χ= X (0) + X(1) and T, and δ the log-hazard of the association between U and T. Finally, the data were administratively censored to induce a 0.25 event proportion.

Statistical Analysis

Performance was assessed using three regression models. Models 1 and 2 were a standard unadjusted Cox model, and a standard Cox model adjusted for work status, respectively. Model 3 was a marginal structural Cox model weighted by the inverse probability of exposure history, which, in our scenario (Figure) amounts to weighting for the inverse probability that X(1) = x given work status W(1).¹³ The hazard ratios derived from each of these models were compared with the true hazard ratio. An estimate of the true hazard ratio was obtained using a marginal structural Cox model weighted by the inverse probability that exposure status X(1) = x given the unmeasured confounder U, generated with a sample size of 20 million. According to the causal structure specified in the Figure, weighting for U, rather than W(1) as in model 3, allows us to ascertain the unbiased (or true) marginal association between cumulative exposure χ and outcome T. Of course, in real research settings one cannot weight for U because it is unobserved. Parameters for model 3 and the true hazard ratio were estimated using stabilized inverse probability of exposure weights, calculated as

FIGURE.

Causal directed acyclic graph representing the healthy worker survivor effect, with time-varying exposure [X(0) and X(1)], work status [W(1)], an unmeasured confounder (U), and a time to event variable (T). Nonpositivity arises due to the zero probability of exposure at follow-up [X(1)] for those who have left work (W(1) = 0).

$$M=\{\begin{array}{c}\frac{Pr[X(1)]}{Pr[X(1)\mid Z]}\text{if}X(1)=1\\ \frac{1-Pr[X(1)]}{1-Pr[X(1)\mid Z]}\text{if}X(1)=0\end{array}$$

where Z = W(1) when weighting for W(1), and Z = U when weighting for U. SAS version 9.2 was used for all analyses (SAS Institute, Inc, Cary, NC) .

Simulation summary

We investigated the behavior of the three Cox models (i.e., crude, adjusted, and weighted) under 13 different specifications ofα ,β ,δ , and γ representing the log-odds or log-hazards of association of 1, 2, 5, and 9. We defined the base-case scenario as: (i) a log-hazard of 9 for δ, the association between U and T; (ii) a log-odds of 9 for γ , the association between U and W(1); (iii) a log-odds of 9 for β, the association between X(0) and W(1); and a log-hazard of 2 for α, the association between cumulative exposure χ and T. Using Monte Carlo simulation, we assessed the performance of hazard ratios derived from the three modeling strategies based on 20,000 trials, each with a sample size of 1000 to mimic modest-sized occupational cancer cohort studies.

To compare the three models across different specifications of α ,β ,δ , and γ we calculated three summaries: (i) bias, computed as the mean of the estimator across 20,000 trials minus the mean of the true value obtained from 20 million trials; (ii) 95% confidence interval (CI) coverage, computed as the proportion of times the 95% CI from the three models contained the true value; and (iii) root mean squared error (root MSE), computed as the square root of the sum of the squared bias and the simulated variance for the estimator. All summaries were calculated for α, the association between cumulative exposure χ and the outcome T. Confidence intervals for the standard Cox models (crude and work-status-adjusted) were estimated using model-based standard errors. Confidence intervals for the marginal structural Cox model were estimated using robust standard errors. The simulated variance was computed as the square of the simulated standard deviation of the estimator across all 20,000 trials.

RESULTS

Each regression method resulted in biased estimates under some conditions. In general, the weighted association was least biased, the crude association most biased, and the adjusted association intermediate. Improved performance of the marginal structural Cox model relative to standard Cox regression may be expected, as the weighted association accounts for time-varying confounding affected by prior exposure, while neither the crude nor adjusted associations do so. The adjusted Cox model tended to be less biased than the crude Cox model, suggesting that, as expected,²⁰ the collider stratification bias induced in the adjusted model is smaller than the confounding in the crude model.

There were exceptions to this general pattern, however. First, in Table 1, rows 8 and 11, there was no time-varying confounding because either e^γ=1 or e^δ=1, respectively. In these two scenarios all associations were unbiased. Second, in Table 1, rows 5 and 6, the time-varying confounder was not affected by prior exposure (i.e., e^β=1), or was only modestly affected by prior exposure (i.e. e^β= 2). In these two scenarios the adjusted association was less biased than the weighted association.

TABLE 1.

Bias under various specifications of the causal structure in the Figure.

				Crude Cox Model		Adjusted Cox Model		Marginal Structural Model
eα	eβ	eγ	eδ	Absolute	Relative	Absolute	Relative	Absolute	Relative
2	9	9	9	−0.19	0.58	−0.09	0.79	−0.06	0.87
1	9	9	9	−0.17	3.10	−0.15	2.83	−0.06	1.70
5	9	9	9	−0.16	0.85	0.03	1.02	−0.01	0.99
9	9	9	9	−0.13	0.91	0.09	1.06	0.01	1.00
2	1	9	9	−0.29	0.50	0.02	1.04	−0.19	0.68
2	2	9	9	−0.27	0.50	−0.02	0.97	−0.15	0.72
2	5	9	9	−0.22	0.54	−0.06	0.86	−0.09	0.81
2	9	1	9	0.00	1.00	0.00	1.00	0.01	1.02
2	9	2	9	−0.07	0.87	−0.04	0.92	−0.01	0.98
2	9	5	9	−0.15	0.70	−0.09	0.82	−0.04	0.92
2	9	9	1	0.00	1.00	0.00	1.00	0.00	1.00
2	9	9	2	−0.07	0.88	−0.03	0.92	−0.02	0.96
2	9	9	5	−0.15	0.71	−0.07	0.87	−0.05	0.91

α represents the log hazard ratio of the association between cumulative exposure X and outcome T; β ,the log odds of the association between baseline exposure X(0) and work status at follow-up W(1); γ , the log odds of the association between an unmeasured confounder U and W(1); and δ , the log-odds of the association between U and T.

The same pattern can be seen with the root mean squared error. The weighted associations were more accurate (i.e., smaller root MSE) than the adjusted, which in turn were more accurate than the crude (Table 2). There were again two exceptions. First, when either e^γ=1 or e^δ=1, root MSE for the weighted association was slightly higher than the crude or adjusted associations. Because there was no bias in these settings, this higher root MSE reflects the effect of lower precision in the weighted model. This lower precision is the expected result of unnecessary protection against time-varying confounding, which is not present in these two settings. Second, when either e^β=1 or e^β= 2, the adjusted association was most accurate. These findings coincide with the patterns of bias observed in Table 1.

TABLE 2.

Root mean squared error under various specifications of the causal structure in the Figure.

α	β	γ	δ	Crude Cox	Adjusted Cox	MSM
2	9	9	9	0.206	0.137	0.113
1	9	9	9	0.202	0.189	0.127
5	9	9	9	0.177	0.099	0.090
9	9	9	9	0.158	0.134	0.089
2	1	9	9	0.305	0.109	0.207
2	2	9	9	0.280	0.102	0.175
2	5	9	9	0.235	0.118	0.131
2	9	1	9	0.104	0.105	0.113
2	9	2	9	0.118	0.111	0.106
2	9	5	9	0.174	0.135	0.111
2	9	9	1	0.101	0.103	0.110
2	9	9	2	0.120	0.106	0.107
2	9	9	5	0.177	0.124	0.112

α represents the log hazard ratio of the association between cumulative exposure X and outcome T;β , the log odds of the association between baseline exposure X(0) and work status at follow-up W(1);γ , the log odds of the association between an unmeasured confounder U and W(1); and δ , the log-odds of the association between U and T.

Finally, as demonstrated in Table 3, the 95% CI coverage for the weighted and adjusted associations proved on average to be roughly equivalent. For instance, across the 13 scenarios explored the mean coverage for the crude, adjusted, and weighted associations were 64.8%, 89.2%, and 88.8%, respectively. However, the 95% CI coverage for the weighted association was less than 90% only for the three scenarios where the association between baseline exposure and subsequent work status was null to moderate. These are exactly the scenarios where the time-varying confounder is affected by prior exposure only moderately to not at all.

TABLE 3.

Confidence limit coverage under various specifications of the causal structure in the Figure. Numbers represent the percentage of 95% confidence limits that included the true value in 20,000 simulations.

α	β	γ	δ	Crude Cox	Adjusted Cox	MSM
2	9	9	9	48.4	85.3	91.3
1	9	9	9	64.4	73.7	92.3
5	9	9	9	53.5	94.7	94.8
9	9	9	9	63.7	86.9	95.1
2	1	9	9	8.2	94.6	47.2
2	2	9	9	13.3	94.9	63.7
2	5	9	9	31.1	90.2	84.1
2	9	1	9	94.8	94.8	94.7
2	9	2	9	89.6	92.7	94.8
2	9	5	9	69.4	87.3	93.5
2	9	9	1	95.1	95.1	95.0
2	9	9	2	88.4	93.9	94.5
2	9	9	5	65.5	90.0	92.9

αrepresents the log hazard ratio of the association between cumulative exposure X and outcome T; β , the log odds of the association between baseline exposure X(0) and work status at follow-up W(1); γ , the log odds of the association between an unmeasured confounder U and W(1); and δ , the log-odds of the association between U and T.

DISCUSSION

Our findings indicate that neither standard nor marginal structural Cox models adequately deal with the bias induced by time-varying confounding with nonpositivity in the context of the healthy-worker survivor effect. As the results in Table 1 suggest, in the absence of time-varying confounding all methods explored recover the correct association, irrespective of nonpositivity. Furthermore, when time-varying confounding is not affected by prior exposure (i.e., when the association between baseline exposure and subsequent work status was null or small), standard adjustment appears to provide a reasonable estimate of the true association. Consequently, when the healthy-worker survivor effect is thought to be operating, researchers could assess the association between exposure and subsequent work status. Stronger associations would suggest use of non-standard methods, whereas null or modest associations might warrant the use of standard Cox models adjusted for work status. Finally, although the marginal structural Cox model proved more robust to increases in the strength of the association between the unmeasured confounder and either work status or mortality, the degree of this improvement over the standard work status adjusted model was modest.

Previous simulation research has considered the healthy-worker survivor effect under settings of no true exposure effect, and evaluated the ability to control for it using standard methods.²¹ As in that work, our simulation scenarios illustrate negative bias in estimates under a null exposure effect due to the healthy-worker survivor effect (Table 1, row 2). We show further that, while weighted methods perform better under a null exposure effect, both standard and weighted methods give biased results. Prior simulation work has not considered the performance of methods in settings with an actual exposure effect. With the few exceptions mentioned previously, marginal structural models consistently perform better in these settings.

The healthy-worker survivor effect has been recognized by epidemiologists for more than 40 years,²² yet most of the proposed solutions inadequately address the biases that may arise under such conditions.² For example, although restricting the analysis to survivors of a given period is likely to decrease confounding bias in certain settings,²³ it may induce a selection bias such that the net bias is increased after restriction.¹⁹ Furthermore, proposed solutions such as exposure lagging²⁴ and work status adjustment²² necessitate the unrealistic assumption that work status is not a risk factor for mortality.² However, as a time-varying confounder affected by prior exposure, adjusting for work status may induce collider-stratification bias.^19–20,25 Moreover, a key assumption underlying exposure lagging is that time off work is equivalent to time on work at zero exposure.²⁶ Yet, if workers leave their jobs for health-related reasons (whether induced by exposure, or some other cause) that also predispose them to higher rates of the event under study, then the risk of the event in persons no longer at work would be higher than the risk of the event in those at work but unexposed. Consequently, time off work is not likely to be equivalent to time on work under no exposure.²⁶ Despite these limitations, however, exposure lagging may successfully be used to specify the appropriate exposure given an assumed empirical induction period.

Though marginal structural models are able to control the bias induced by time-varying confounders that are affected by prior exposure,¹¹ as we show here, they do not account for the bias induced by nonpositivity. Because persons who have left the workplace cannot receive work-based exposures at subsequent time-points, the model used to estimate the inverse probability weights encounters a zero cell in the stratum where W(1) = 0 and X(1) = 1. Thus, the inverse of the probability of being exposed at time t = 1 when one has left the workplace is undefined. As a result, using the inverse probability weights fails to fully eliminate the bias induced by nonpositivity.

We have compared the performance of crude, adjusted and marginal structural Cox models under conditions of systematic nonpositivity, using the healthy-worker survivor effect as an example. We would expect our results to generalize to any research context with a causal structure similar to the one outlined in the Figure. Whether these results generalize to settings where nonpositivity is random¹⁴ remains uncertain. However, recent evidence suggests that, for marginal structural models, increased random nonpositivity results in less efficient estimators.²⁷ A lack of positivity may also arise in other research settings, such as when assessing the independent effects of neighborhood deprivation on preterm birth¹⁵ or the verbal ability of children.¹⁶ It is uncertain how marginal structural models would deal with nonpositivity bias encountered in such settings.

The present findings should be interpreted with limitations in mind. First, our scenarios were characterized by data with only two exposure time points. The bias may be amplified or attenuated under conditions where more than two time points are present. Second, in our simulations, when bias is present, we cannot partition the bias into components due to time-varying confounding and due to nonpositivity. Only when time-varying confounding is absent (e.g., when the association between the unmeasured confounder and either work status or mortality is null) can we attribute all the bias to nonpositivity. However, the results presented in Table 1, rows 5–7, do support the speculation that the inverse relationship between the bias for the weighted association and the effect of the baseline exposure on subsequent work status is due to the relationship between nonpositivity and time-varying confounding. Specifically, when the effect of baseline exposure on subsequent work status is small (or null), the time-varying confounder is affected only weakly (or not at all) by prior exposure, and the constant nonpositivity bias plays a proportionately larger role in biasing the estimates. Finally, the choice of parameterizations used to specify the magnitude of effects was only a small portion of all possible scenarios. As in all Monte Carlo research, results apply only to the scenarios studied.²⁸ Future research might assess these methods using a broader set of scenarios based on established cohorts, and using methodological approaches better suited to handle nonpositivity, such as history-restricted marginal structural models,²⁹ g-estimation of a structural nested accelerated failure-time model,⁶ or the parametric g-formula.^30–31

Despite these limitations, our approach does offer strengths. To our knowledge, this is the first assessment of the performance of marginal structural models in the context of the healthy-worker survivor effect. Our causal structure and simulated data are characterized by only two biases, time-varying confounding, and nonpositivity, allowing for a more clear understanding of the role that these biases play in occupational epidemiology. Finally, we compared the performance of a commonly used resolution to the healthy-worker survivor effect (adjustment for work status) to a less common, but perhaps more theoretically justified, approach (marginal structural models).

In conclusion, neither standard nor marginal structural Cox proportional hazards models can fully resolve the bias encountered under conditions of time-varying confounding with nonpositivity, such as in the healthy-worker survivor effect.