Counterpoint: Keeping the Demons at Bay When Handling Time-Varying Exposures—Beyond Avoiding Immortal Person-Time

Abstract

The potential for immortal time bias is pervasive in epidemiologic studies with left truncation or time-varying exposures. Unlike other biases in epidemiologic research (e.g., measurement bias, confounding due to unmeasured factors, and selection based on unmeasured predictors of the outcome), immortal time bias can and should be avoided by the correct assignment of person-time during follow up. However, even when handing person-time correctly, allowing late entry into a study or into an exposure group can open the door to more insidious sources of bias, some of which we explore here. Clear articulation of the study question, including the treatment plans of interest, can provide navigation around these sources of bias and elucidate the assumptions needed for inference given the available data. Here, we use simulated data to illustrate the assumptions required under various approaches to estimate the effect of a time-varying treatment and describe how these assumptions relate to the assumptions necessary to estimate single sample rates and risks in settings with censoring and truncation.

Editor's note: A counterpoint to this article appears on page 1013.

In this issue of the Journal, Harding and Weiss (1) describe study design and population characteristics that influence the magnitude of immortal time bias in cohort studies. The pervasiveness and importance of immortal time bias has been highlighted throughout the epidemiologic literature, explaining phenomena including the implausibly large protective effects of metformin on cancer outcomes and inhaled corticosteroids in chronic obstructive pulmonary disease patients (2, 3). We commend Harding and Weiss for drawing attention to this important issue. The authors appropriately point out that, where possible, inclusion of immortal time should be avoided, and, when not possible (e.g., when interpreting the work of others), a simple back calculation might provide insight into the true size of the association between exposure and outcome.

Immortal time bias is one of the many sources of bias that could explain implausibly large protective findings in real world evidence studies estimating the effect of medical interventions. Other sources include unmeasured confounding due to frailty (4, 5) and healthy-user bias due to selection of individuals who remain on treatment for prolonged periods of time (6, 7). However, rather than an inevitable consequence of missing important information on factors influencing treatment assignment or selection, bias due to the inclusion of immortal time can usually be avoided by clear articulation of the study question, appropriate choice of study design, and thoughtful handling of person-time during analyses.

Here, we delve into some issues beyond immortal person-time that arise when dealing with censored and truncated data, both when estimating risks or rates in a single sample and when estimating treatment effects. Briefly, in the first section, we review methods to estimate single sample risks and rates in both closed cohorts and cohorts that are open on the left. Specifically, we illustrate how the traditional approach used by Harding and Weiss to estimate rates directly from truncated data will fail if the hazard is not constant over time. In the second section, we review methods to estimate risks and rates under possible treatment plans, including how to specify a study question when observed treatments are time-varying.

ESTIMATING RISKS AND RATES FROM TRUNCATED DATA

To illustrate issues involved in estimating risks and rates from truncated data, we begin by analyzing a simulated data set designed roughly to mimic the study explored by Harding and Weiss. Consider a hypothetical study to estimate the 10-year risk of mortality after cataract diagnosis among a group of women diagnosed with cataracts (regardless of whether or not they had cataract surgery). Using the data reported in Tseng et al. (8; retracted and replaced (9)), we estimated that the overall mortality rate was about 2 deaths per 100 person-years. Based on this rate, we simulated data compatible with the summary data presented in the paper under 3 data-generating mechanisms: 1) a scenario with a constant hazard of mortality over the 10 years; 2) a scenario with a decreasing hazard function; and 3) a scenario with an increasing hazard function. Figure 1 displays the true hazard functions (panels A through C) and risk functions (panels D through F) under each of the 3 scenarios. In the simulated data, we estimated rates as the number of deaths divided by the number of person-years, where the number of person-years was simply the time from the origin to the first of death or the end of follow-up. We estimated risks as the complement of the Kaplan-Meier estimate of the survival function (10). In all scenarios, the 10-year risk of mortality was 18%, and the overall rate was 2.00 deaths per 100 person-years (Table 1). Further details on the data-generating mechanism can be found in Web Appendix 1 (available at https://academic.oup.com/aje).

Figure 1.

Hazard functions (A, B, and C) and risk functions (D, E, and F) for all-cause mortality after cataract diagnosis under simulated scenario 1 (constant hazard function; A and D), simulated scenario 2 (decreasing hazard function; B and E), and simulated scenario 3 (increasing hazard function; C and F).

Table 1.

Overall Rates Over 10 Years and 10-Year Risks of Mortality for a Simulated Cohort of 100,000 Women at Cataract Diagnosis Under 3 Hypothetical Scenarios for Study Entry

Analysis	Scenario 1a				Scenario 2b				Scenario 3c
	No. of Cases	No. of Person-Years	Rate	Risk	No. of Cases	No. of Person-Years	Rate	Risk	No. of Cases	No. of Person-Years	Rate	Risk
1: Immediate study entry	18,161	906,712	2.03	18.2	18,034	877,031	2.06	18.0	18,077	952,832	1.89	18.1
2: Late entry; include immortal time	11,395	890,217	1.28	12.2	7,379	861,796	0.86	8.3	16,454	944,203	1.74	16.8
3: Late entry; exclude immortal time	11,395	568,158	2.01	17.4	7,379	552,214	1.34	17.1	16,454	601,245	2.74	17.9
4: Severe late entry; exclude immortal time	9,864	416,582	2.01	16.6	5,182	405,943	1.28	12.3	13,350	440,356	3.03	17.6

^a Hazard function was constant over time.

^b Hazard function decreased over time.

^c Hazard function increased over time.

An ideal study design would recruit all participants at the origin (also known as time zero), here cataract diagnosis. However, in many studies, some participants are unable to be observed from the origin. For example, consider the hypothetical scenario in which data were unavailable for a subset of participants until they transferred into a study clinic at some time after cataract diagnosis. We induced this situation in the simulated data by generating a time of study entry for each participant W_i, and we set W_i to have a mean of 3 years after the origin.

Including these participants in the study is appealing to increase sample size and possibly extend the length of follow-up. However, when some participants enter the study after the origin, data for other women who would have entered the study—but who died (or had the event) prior to study entry—are truncated (11). Moreover, the time between the origin and study entry is “immortal” because people who enter the study at time W_i cannot have had the event prior to W_i, by definition. Inclusion of this immortal person-time in the estimation of risks and rates produces a downward bias in the estimates. To see why, consider that events contributed by truncated women to the risks and rates are not counted, but the person-time contributed by women who survive until their entry time is included, producing a downward bias in the absolute risks and rates.

Analysis 2 in Table 1 displays estimated risks and rates if the immortal person-time is included. Specifically, to estimate the rates in analysis 2, we calculated person-time as the time from the origin to the first of the time of death or end of follow-up for all participants (including those who entered the study late). When estimating the risks, we allowed participants who entered the study late to be included in the risks sets for event times prior to their study entry. As expected, these risks and rates are too low. The estimators for rates and risks used in each analysis are available in Web Appendix 2.

Analysis 3 of Table 1 displays risks and rates estimated by incorporating knowledge of the participants’ entry times into the study. In the estimation of rates, person-time was calculated as the time between study entry and the first of the event or end of follow-up. To estimate the risks, we applied an “extended” version of the Kaplan-Meier estimator that excluded participants from the risk sets for events occurring before their entry time (12). Estimated risks matched the true risks for each scenario, but the rate estimated using this approach was correct only in the scenario with a constant hazard function. In the scenario where the hazard was decreasing, the estimated rate was too low, and in the scenario where the hazard was increasing, the estimated rate was too high.

In analysis 4 in Table 1, we analyze a third data set in which we have induced severe late entry. Specifically, in this simulated data set, no participants enter the study until 6 months after the origin. In the scenario with a constant hazard function, the estimated rate was unbiased (although, as in analysis 3, estimated rates in scenarios 2 and 3 were biased). However, estimating risks in all 3 scenarios required an assumption that the risk prior to the first entry time was 0. The plausibility of this assumption is dependent on the subject area under study; however, in this simulated example, this assumption is not met and risks are biased. In most settings, universal late entry points to a need to redefine the study question to anchor the analysis at a more suitable origin.

Rates appear to be intuitive and straightforward summaries of the risk function. With complete data (i.e., no late entry or loss to follow-up), we were able to estimate the rate correctly under constant, increasing, or decreasing hazard functions. However, with late entry, estimated rates were too low or too high in settings with decreasing and increasing hazard functions, respectively, even after excluding the immortal person-time. In the next section, we explore the consequences of allowing late entry into specific exposure groups when estimating the effects of exposures that might vary over time.

CONSIDERATIONS WHEN ESTIMATING TREATMENT EFFECTS

Harding and Weiss (1) explored an example comparing 2 treatment plans among a target population of women diagnosed with cataracts. The potential for immortal time bias arises because, although all women were observed from the origin, treatment (cataract surgery) occurred at different times.

In any study, clear specification of the study question is important. For study questions with clear implications for clinical decision making, it is useful to think about mimicking a randomized trial as closely as possible using observational data (13). While the potential for unmeasured confounding exists in nonexperimental studies, framing the study question in terms of an idealized “target trial” helps clarify issues regarding the target population (who should be included in the analysis?), the origin (when would randomization have occurred?), the follow-up period (over which time period do the results apply?), and the treatment plans to be compared.

To explore issues related to estimating treatment effects in the setting explored by Harding and Weiss, we simulated a random exposure time for each woman included in the “full data” for scenarios 1 through 3, used in analysis 1 of Table 1. Specifically, 20% of women were assigned “immediate” treatment at cataract diagnosis, and the remaining women were assigned a random time for treatment drawn from a Weibull distribution, as described in Web Appendix 1. In the simulated data, treatment had no effect on mortality. In each scenario, we estimated absolute risks and rates under each treatment plan, as well as risk, rate, and hazard ratios corresponding to several contrasts of interest.

Contrast 1: Comparing “immediate surgery” with “no immediate surgery”

Rather than compare “people who had cataract surgery” with “people who did not have cataract surgery,” a hypothetical trial might compare a plan to provide cataract surgery immediately after cataract diagnosis in one group with a plan not to provide cataract surgery immediately after cataract diagnosis in the other group. In this hypothetical trial, participants randomized not to receive immediate cataract surgery might (or might not) later receive surgery.

To emulate this trial using the simulated “nonexperimental” data, we classified women who received surgery immediately after diagnosis as “treated” and women who did not receive immediate surgery as “untreated.” With these static exposure groups clearly defined, computation of rates or risks is straightforward. When estimating this contrast, we correctly recovered the true (null) risk, rate, and hazard ratios in all 3 scenarios (analysis 1 in Table 2).

Table 2.

Rates, Risks, and Rate, Risk, and Hazard Ratios Comparing Mortality Over 10 Years of Follow-Up Between 2 Treatment Plans Implemented at Cataract Diagnosis in a Simulated Cohort of 100,000 Women Under Various Analytical Approaches

Analysis	Scenario 1a					Scenario 2b					Scenario 3c
	Rate	Rate Ratio	Risk	Risk Ratio	Hazard Ratio	Rate	Rate Ratio	Risk	Risk Ratio	Hazard Ratio	Rate	Rate Ratio	Risk	Risk Ratio	Hazard Ratio
1: Immediate surgery vs. no immediate surgery
Immediate surgery	1.98	0.99	17.9	0.99	0.99	2.12	1.01	18.5	1.01	1.01	1.86	0.99	17.8	0.99	0.99
No immediate surgery	2.00	1.00	18.1	1.00	1.00	2.09	1.00	18.3	1.00	1.00	1.89	1.00	18.0	1.00	1.00
2: Immediate surgery vs. no surgery
Immediate surgery	1.98	0.98	17.9	0.98	0.97	2.12	0.66	18.5	1.00	1.00	1.86	2.52	17.8	1.00	0.99
No surgery during follow-up	2.01	1.00	18.4	1.00	1.00	3.21	1.00	18.6	1.00	1.00	0.73	1.00	17.7	1.00	1.00
3: Immediate surgery vs. no surgery (allowing late surgery)
Immediate surgery	1.99	0.99	17.4	1.00	0.97	1.62	0.50	17.5	1.03	1.01	2.37	3.21	17.8	1.01	0.97
No surgery during follow-up	2.01	1.00	17.3	1.00	1.00	3.21	1.00	17.0	1.00	1.00	0.74	1.00	17.6	1.00	1.00
4: Immediate surgery vs. no surgery (with only extremely late surgery)
Immediate surgery	1.96	0.99	10.4	0.61	1.00	1.12	0.50	7.8	0.43	0.95	3.71	2.84	17.3	0.98	1.04
No surgery during follow-up	1.98	1.00	17.2	1.00	1.00	2.37	1.00	18.0	1.00	1.00	1.30	1.00	17.7	1.00	1.00

^a Hazard function was constant over time.

^b Hazard function decreased over time.

^c Hazard function increased over time.

As a side note, there are many settings where true immediate treatment is infeasible. For example, even if a participant were randomized to receive immediate surgery, it is likely that there would be a short delay before the surgery could be scheduled and carried out. In such settings, trials might employ a grace period to allow a 2-week or 1-month window for women to receive the surgery. Nonexperimental studies might also allow such a grace period using a variety of methods beyond the scope of this commentary (13, 14).

Contrast 2: Comparing the “immediate surgery” plan with a plan that prohibits surgery throughout follow-up

Regardless of whether or not a grace period is employed, in this first study design, some women randomized not to receive immediate surgery will likely go on to receive surgery later in the course of follow-up. In the simulated data for scenario 1, 89% of untreated women who did not receive immediate surgery went on to receive surgery during follow-up. An alternative trial might compare more strict treatment plans that dictate whether or not a woman receives surgery for the entire duration of follow-up. Such a trial might compare the plan to receive immediate surgery (or surgery within some grace period) with a second plan that prevents a woman with cataracts from receiving surgery during the entire follow-up period (15).

There are several approaches to estimate contrast 2. In the first approach, we classified participants as “treated” or “untreated” according to whether or not they received the surgery immediately after diagnosis. Risks and rates under the “immediate surgery” plan were computed as in the first analysis. However, participants in the “no surgery” group were censored on the date that they later received surgery. Risks and rates were then estimated in this modified data set, meaning that, in the untreated group, person-time and events were not counted after a woman had surgery, and women were not included in the risk sets for events occurring after they had surgery. Typically, in both experimental and nonexperimental settings, such censoring is informative and requires handling using analytical approaches (e.g., inverse probability of censoring weights) (15, 16). In the simulated data, exposure assignment and timing were random.

Using this first approach to estimate the effect of immediate surgery compared with no surgery, we recovered the true (null) risk and hazard ratios in all 3 scenarios. The estimated rate ratio was correct in scenario 1 but too low in scenario 2 and too high in scenario 3 (analysis 2 in Table 2). To understand why, consider that the rates under the “immediate surgery” plan remained correct at about 2 deaths per 100 person-years. However, under the “no surgery” plan, women were censored when they had surgery. This means that some of the later person-time (and corresponding events), when the hazard was lower in scenario 2 and higher in scenario 3 was not included, leading to estimated rates that were too high and too low, respectively, even though the person-time was correctly allocated (i.e., immortal time was avoided). Note that, in settings where treatment affects the outcome, we would expect the difference in mortality between these plans to be larger than the difference in mortality between the “immediate surgery” and “no immediate surgery” plans.

In some nonexperimental settings, the number of participants receiving immediate treatment is small or the length of follow-up among participants meeting the criteria for immediate exposure is short. Under a set of assumptions, a second approach to estimate contrast 2 allows participants censored from the “untreated” group on the date they became exposed to enter into the treated group as “late entries” on that date.

Allowing late entries into the “immediate treatment” group requires the same analytical approach as allowing late entries into the study when estimating a single sample rate or risk. Specifically, as Harding and Weiss emphasize, the time between the origin and entry into the treated group should not be counted as treated (rather, this untreated person-time should remain with the untreated group). Moreover, allowing late entries into the treated group typically requires an assumption that there is no cumulative effect of treatment on the outcome of interest.

To implement this second approach to compare immediate surgery with no surgery, we again censored untreated women from the “no treatment” arm when they had the surgery, but we also allowed them to enter late into the treated arm at this time. Allowing them to enter late into the treated arm implies that we correctly allocated their person-time prior to surgery as untreated and did not include them in the risk sets for the treated group until after their surgery. After implementing this approach, we recovered the true (null) risk, rate, and hazard ratios in scenario 1 (analysis 3 in Table 2). In scenarios 2 and 3, estimated risk and hazard ratios were correct, but rate ratios were biased because the estimated rate in the treated arm was too low (scenario 2) or too high (scenario 3). Bias in estimated rates for the treated arm in scenarios 2 and 3 stems from preferential inclusion of later person-time in the treated group, in which the hazard was lower (scenario 2) or higher (scenario 3). Estimated risks, risk ratios, and hazard ratios were not biased because the estimators we used allowed us to appropriately assign those entering the treated group after the origin to risk sets for events occurring after their treatment times.

Estimating contrast 2 when no patients receive immediate treatment

Finally, we examined an extreme setting in which no women received immediate surgery. Specifically, we altered the data used above so that the first women to receive surgery did so at 4 years after cataract diagnosis. In this setting, contrast 1 could not be estimated because no women received immediate treatment. Similarly, the first approach to estimate contrast 2 could not be used, because no women were classified as treated from the origin. However, the second approach to estimate contrast 2, which allowed women to have late entry into the treated group, could be implemented.

Calculation of rates, rate ratios, and hazard ratios in this setting was straightforward: all women began the study as untreated and accrued person-time in the untreated group. After surgery, person-time and events were allocated to the treated group. Calculation of risks required the additional assumption that the true risk under the plan “provide immediate surgery” was 0 up until the time the first women received the surgery and entered the treated group.

Results are provided in analysis 4 in Table 2. As in analysis 3, the estimated rate ratio was correct (i.e., null) in scenario 1, while it was biased in scenarios 2 and 3. Estimated risk ratios were biased downward in all scenarios because risks under the “immediate surgery” arm were too low due to reliance on the assumption that risk was 0 prior to the first late entry into the treated group. Estimated hazard ratios were approximately null in all scenarios.

DISCUSSION

Harding and Weiss raised important points about the dangers of including immortal person-time when dealing with time-varying exposures. They illustrated the distortion of estimated rate ratios due to inappropriate categorization of person-time, providing an intuitive, hands-on example for readers.

Here, we’ve highlighted some additional considerations for epidemiologists when estimating the effects of time-varying exposures. First, the study question should be clearly articulated. Producers and consumers of research findings should be able to identify the origin, the event under study, the relevant follow-up period, and the treatment plans being compared. In addition, investigators should be up front about how epidemiologic measures under various plans are computed. For example, do all observed participants follow one of the prescribed plans exactly? If not, how are outcome histories handled for people who switch treatments during the course of follow-up, and what assumptions allow investigators to estimate risks or rates under a given treatment plan when some participants do not follow that plan for the entire follow-up period? Do at least some participants follow all plans of interest? Finally, investigators should summarize outcomes in a way that minimizes bias given the above considerations while remaining interpretable to readers.

While rates are convenient summaries of the risk functions and allow apparently straightforward calculation, they are likely to be biased in settings with censoring or truncation when the hazard function is not constant over time, even when person-time is allocated appropriately between treatment groups. In these settings, rates and rate ratios could be calculated separately for different points along the timescale of interest, or the hazard ratio could be used in place of the rate ratio. However, neither rate nor hazard ratios are ideal for estimating causal effects. Overall rate and hazard ratios are nearly always a function of the length of follow-up, and time-specific rate and hazard ratios are subject to selection bias (17).

Moreover, quantifying the effects of time-varying exposures using rate ratios and hazard ratios might obscure important limitations in the data. For example, in the setting described in analysis 4 in Table 2, the parameter of interest was a comparison of a plan to provide immediate surgery and a plan to prevent women from receiving surgery for all of follow-up. However, the first women to receive surgery did so extremely late during follow-up. Estimation of the rates was straightforward despite this extreme late entry into the exposed group: The analyst simply needed to add up the person-time and number of events occurring before versus after surgery, estimate the rates, and compare them. However, this simple calculation belies the more insidious issue of extreme nonpositivity: No one in the study sample followed one of the treatment plans of interest!

As a result, estimation of risk, rate, and hazard ratios relied on assumptions that could not be tested in the data. Specifically, estimation of rates and rate ratios relied on an assumption that the hazard was constant over time, and estimation of hazard ratios required an assumption that the hazard ratio was constant over time. Nonparametric estimation of the risk functions relied on the assumption that the risk under the “immediate surgery” plan was 0 until the first women received surgery. Without additional information, it is impossible to know which (if any) of these assumptions holds.

Situations in which no (or few) study participants follow the exposure plan of interest suggest that the data might be inadequate to answer the intended study question and that the study question itself might not be realistic. For example, if no women receive cataract surgery immediately after cataract diagnosis in a real-world setting, this might not be a feasible strategy due to logistical or other clinical constraints. In this setting, a more useful study question might compare mortality under exposure plans anchored to an alternative clinical decision point using a treatment-decision design (18).

Beyond improving interpretability and impact of results, moving to a treatment-decision design (that compares exposure plans actually followed in the observed data from a clinical “decision point”) can reduce the potential for bias due to accidental inclusion of immortal time or the other issues explored above in several ways. First, if the study is anchored on a true decision point, the data set likely contains enough individuals following each plan from baseline that contrast 1 can be estimated using standard methods, and contrast 2 can be estimated by simply censoring people from each plan when they deviate from that plan (as in analysis 2 in Table 2) and applying censoring weights as necessary. Without late entry into either exposure plan, the potential for immortal time bias is eliminated.

Traditional prospective cohort studies require investigators to specify an origin when individuals become eligible, recruit participants at or before this origin, and follow participants over time. Because such data are typically collected with the study question in mind, the origin is often clear and exposure plans are well defined. In contrast, with the increasing availability and utilization of routinely collected, large health-care databases for epidemiologic studies, investigators are more often creating analytical cohorts from existing data collected for a variety of purposes (e.g., clinical management and billing) (6). In these settings, it is up to the investigator to select the appropriate origin and identify participants following various exposure plans in the existing data. In contrast to traditional prospective cohort studies, cohorts constructed from existing databases allow investigators access to participant data unconstrained by a specific origin or follow-up period, which means that clear articulation of the study question might occur after one has seen the data. Moreover, access to entire exposure histories increases the number of opportunities for inappropriately assigning cohort membership and exposure based on future values, also referred to as “crystal ball” design (6, 19).

To summarize, clear articulation of the study question—including the origin, exposure plans compared, event definition, and follow-up period of interest—is an important part of epidemiologic data analysis (20) and minimizes the propensity for immortal time bias. If exposure categorization in the analytical data set is static (and based on information collected at baseline), estimation of risks and rates under each plan is often straightforward. However, estimating the effect of a time-varying exposure by simply comparing rates with events and person-time allocated to exposure plans in a time-varying fashion invokes strong, hidden assumptions that are brought to light by clearly articulating the plans being compared.