Comparison of Statistical Approaches for Dealing With Immortal Time Bias in Drug Effectiveness Studies

Abstract

In time-to-event analyses of observational studies of drug effectiveness, incorrect handling of the period between cohort entry and first treatment exposure during follow-up may result in immortal time bias. This bias can be eliminated by acknowledging a change in treatment exposure status with time-dependent analyses, such as fitting a time-dependent Cox model. The prescription time-distribution matching (PTDM) method has been proposed as a simpler approach for controlling immortal time bias. Using simulation studies and theoretical quantification of bias, we compared the performance of the PTDM approach with that of the time-dependent Cox model in the presence of immortal time. Both assessments revealed that the PTDM approach did not adequately address immortal time bias. Based on our simulation results, another recently proposed observational data analysis technique, the sequential Cox approach, was found to be more useful than the PTDM approach (Cox: bias = −0.002, mean squared error = 0.025; PTDM: bias = −1.411, mean squared error = 2.011). We applied these approaches to investigate the association of β-interferon treatment with delaying disability progression in a multiple sclerosis cohort in British Columbia, Canada (Long-Term Benefits and Adverse Effects of Beta-Interferon for Multiple Sclerosis (BeAMS) Study, 1995–2008).

In many observational studies of drug effectiveness, there may be a delay or waiting period before a subject begins to receive a treatment. Let us use multiple sclerosis (MS) as an example, with the disease-modifying drug β-interferon being the study drug of interest. If treated subjects entered a cohort at the time of their first β-interferon administration but untreated subjects entered the cohort at an earlier time, such as their first assessment at an MS clinic or study center conducting the observational study (as in the paper by Trojano et al. (1)), then the time period between the first clinic visit (baseline) and the first administration of β-interferon corresponds to “immortal time” (2, 3). This delay period is labeled “immortal” because, by design, subjects in the treated group cannot develop an outcome before receiving the treatment. Therefore, if some subjects develop the outcome before getting the opportunity to initiate treatment, they will be assigned to the untreated group.

If treatment status is misclassified during this delay period or if this delay period is excluded from the analysis, “immortal time bias” is introduced. Classifying subjects as “treated” or “untreated” with a time-invariant variable results in a spurious survival advantage (protective association) in favor of the treated group. This can considerably distort the estimated hazard ratio if a large number of failures (events or reaching outcome) occur before the initiation of treatment or if the duration of the immortal time is large (4). Although this bias was first identified in the 1970s (5), many studies still fail to account for this source of bias (2, 6, 7). Recently, the issue of immortal time bias has resurfaced in pharmacoepidemiology studies in the context of propensity score adjustment (3, 8–11), where treatment status must be defined at baseline.

The most common approach to accounting for immortal time bias is the adoption of a proper treatment exposure definition via time-dependent analyses (12), such as time-dependent Cox proportional hazards models (13–16). Prescription time-distribution matching (PTDM) (13) is a simple approach to adjusting for immortal time bias that has been frequently cited (17–21) and used (22–28) in recent years. The performance and statistical efficiency of this approach have been claimed to be similar to those of the Cox model with a binary time-dependent exposure (hereafter called the “time-dependent Cox model”) (13).

Several studies have quantified the amount and direction of bias due to “misclassifying” or “ignoring” immortal time by means of simulation (14, 29–31) or theoretically (15, 32–34); however, to the best of our knowledge, no attempt has been made to explore the appropriateness of the PTDM approach in minimizing immortal time bias. Our primary focus in this study was to assess the performance of the PTDM for dealing with this bias. We quantified the bias due to PTDM via simulation studies as well as theoretically (see Web Appendix 1 and Web Figure 1, available at http://aje.oxfordjournals.org/) For comparison, we considered a simplified version of the sequential Cox approach (35), which was originally proposed for dealing with time-dependent confounding (36). Additionally, misclassified and excluded immortal time approaches were also compared. We applied these methods to investigate the association of time-varying β-interferon treatment with delaying disability progression in subjects from the database of a British Columbia, Canada, MS study, the Long-Term Benefits and Adverse Effects of Beta-Interferon for Multiple Sclerosis (BeAMS) Study (1995–2008) (37, 38).

METHODS

Notation

Consider a hypothetical longitudinal study consisting of n subjects (i = 1, 2, …, n). Let t₀ = 0 be the start of follow-up (i.e., the time of the baseline visit). Data on baseline covariates L₀ (binary) are recorded at this time. Follow-up continues until the time of failure (reaching outcome) T or censoring T^C. Regular measurements of the binary treatment status $A_{t_m}$ ($A_{t_m}$ = 1 for treated and 0 otherwise) are recorded at intervals m = 0, 1, 2, …, K. Let [t_m, t_m+1) denote the mth interval. Because this study is focusing on the implications of immortal time, we assume that the subjects may initiate treatment no more than once and continue taking the treatment thereafter until the study ends (i.e., the value for treatment exposure changes only once, from 0 to 1). Let treatment initiation occur at time T^A.

For group-based comparison, suppose k = 1 indicates the ever-treated group, whereas k = 0 indicates the never-treated group. Let n_k indicate the size of these groups. Further, let N_k and T_k (k = 0, 1) indicate the observed number of failures and follow-up person-time in these groups, respectively. Let r = T₀:T₁, the ratio of the observed person-times in the never-treated and ever-treated subjects. Denote T_IT as the observed immortal time—that is, the aggregated follow-up time not under treatment in the ever-treated group—and set f = T_IT/T₁. We describe the analytical approaches in detail in the following sections.

Cox models with a time-invariant treatment variable

Two imprecise treatment exposure definitions are often used to estimate the log hazard ratio (log HR), where a Cox model with time-invariant treatment variable A (which does not depend on the time interval) is fitted while adjusting for the potential confounders measured at the original baseline as follows:

$${\lambda }_T(t_m|L_0)={\lambda }_0(t_m)\text{exp}({\psi }_1\mathit{A}\mathit{+}{\psi }_2{\mathit{L}}_{0)},$$

where m is the visit index, λ₀(t_m) is the unspecified baseline hazard function, ψ₁ is the log HR of the treatment status (A), and ψ₂ is the vector of log HRs for the baseline covariates L₀.

Included immortal time

In this approach, subjects who were ever exposed to treatment (“ever-treated”) are classified as treated for their whole duration of follow-up. Under the assumption of constant hazard, comparing the failure rate ratio from this approach (RR′) with the correct rate ratio (RR) obtained from a time-dependent analysis (see Suissa (15) and Web Appendix 2) yields

$$\frac{{\text{RR}}^{\mathrm{\prime }}}{\text{RR}}=\frac{{\displaystyle \frac{N_1/T_1}{N_0/T_0}}}{{\displaystyle \frac{N_1/(T_1-T_{\mathrm{IT}})}{N_0/(T_0+T_{\mathrm{IT}})}}}=(1-f)\times \frac{r}{(r+f)}.$$

From this equation, we can see that this approach always underestimates the correct failure rate ratio (also see Web Figure 2).

Excluded immortal time

In this approach, the immortal time—that is, time from cohort entry to the initiation of treatment—is excluded from the ever-treated subjects' follow-up, and time zero for these subjects is taken to be the time of treatment initiation, T^A. However, the follow-up period for the subjects who were never exposed to treatment (“never-treated”) remains the same; that is, time zero is the time of cohort entry t₀ = 0. Comparing the failure rate ratio from this approach (RR″) with the correct rate ratio obtained from a time-dependent analysis (see Suissa (15) and Web Appendix 2) yields

$$\frac{{\text{RR}}^{\mathrm{\prime }\mathrm{\prime }}}{\text{RR}}=\frac{{\displaystyle \frac{N_1/(T_1-T_{\mathrm{IT}})}{N_0/T_0}}}{{\displaystyle \frac{N_1/(T_1-T_{\mathrm{IT}})}{N_0/(T_0+T_{\mathrm{IT}})}}}=\frac{r}{(r+f)}.$$

From this equation, we can see that if the person-times in the ever-treated cohort are much smaller than the person-times in the never-treated cohort (i.e., if r is large), the bias from this approach may be negligible, even for large fractions of immortal time f (also see Web Figure 3).

Cox model with a time-dependent treatment variable

In the presence of the baseline covariates L₀, the hazard function can be expressed as the following time-dependent Cox model:

$${\lambda }_T(t_m|L_0)={\lambda }_0(t_m)\exp ({\psi }_1{\mathit{A}}_{{\mathit{t}}_{\mathit{m}}}\mathit{+}{\psi }_2{\mathit{L}}_0),$$ (1)

where ψ₁ is the log HR of the time-dependent treatment status $(A_{t_m}).$

The PTDM approach

The essence of the PTDM approach is to redefine time zero in both the ever-treated and the never-treated groups (13) by shifting the start of follow-up to the time of treatment initiation (the end of the immortal time period) T^A for the ever-treated subjects. First the immortal time (wait) periods for all of the ever-treated subjects are listed. For each never-treated subject, a time zero is selected at random (with replacement) from this list and assigned as the new time zero for the corresponding never-treated subjects. If a never-treated subject has an event before the end of the assigned immortal time (new time zero), he or she is excluded from further analysis. The analysis is performed on the new time zero points—that is, the newly defined baselines after exclusion of the observed or assigned immortal time from the follow-up for the ever-treated and never-treated groups, respectively (see Web Figure 1). This is expected to eliminate the imbalance in the excluded time distribution between the two treatment groups.

Let us denote the immortal time for the jth treated subject as T_j,IT. Similarly, the matched-immortal time for the j′th untreated subject is denoted by T_j′,IT. The aggregated immortal time is denoted by $T_{\mathrm{IT}}\equiv {\sum }_{j=1}^{n1}{T}_{j,\mathrm{IT}}$, and we denote the aggregated matched-immortal time by $T_{\mathrm{IT}}^{\mathrm{\prime }}\equiv {\sum }_{j^{\mathrm{\prime }}=1}^{n0}{T}_{j^{\mathrm{\prime }}}{,}_{\mathrm{IT}}.$ The number of failures observed during the aggregated immortal time T_IT is zero. Let $N_{\mathrm{IT}}^{\mathrm{\prime }}$ be the number of failures observed during the aggregated matched-immortal time $T_{\mathrm{IT}}^{\mathrm{\prime }}.$ For $N_{\mathrm{IT}}^{\mathrm{\prime }}\ge 0,$ the total untreated person-time T_x of the $N_{\mathrm{IT}}^{\mathrm{\prime }}$ patients who had failures within their assigned immortal times is excluded. Set x = T_x/T₁ ≥ 0. In addition, set $T_{\mathrm{IT}}^{\mathrm{\prime }}=q\times T_{\mathrm{IT}};$ that is, q is the ratio of assigned and observed immortal times. Here, q > 1 for the setting in which the aggregated matched-immortal time $T_{\mathrm{IT}}^{\mathrm{\prime }}$ is larger than the aggregated observed immortal time T_IT; otherwise, 0 < q < 1. Comparing the failure rate ratio from PTDM (RR‴) with the correct rate ratio obtained from a time-dependent analysis (detailed derivation shown in Web Appendix 1) yields

$$\frac{{\text{RR}}^{\mathrm{\prime }\mathrm{\prime }\mathrm{\prime }}}{\text{RR}}=\frac{{\displaystyle \frac{N_1/(T_1-T_{\mathrm{IT}})}{(N_0-{N^{\mathrm{\prime }}}_{\mathrm{IT}})/(T_0-{T^{\mathrm{\prime }}}_{\mathrm{IT}}-T_x)}}}{{\displaystyle \frac{N_1/(T_1-T_{\mathrm{IT}})}{N_0/(T_0+T_{\mathrm{IT}})}}}=\frac{N_0}{N_0-{N^{\mathrm{\prime }}}_{\mathrm{IT}}}\frac{r-q\times f-x}{r+f}.$$

Varying r, f, x, and q yields Figure 1. This approach shows a downward bias for increasing values of f, the fraction of the immortal person-time in the ever-treated subjects. The level of bias increases with both q and x. As with the excluded immortal time approach (described in Web Appendix 2), the bias from this approach is also small if the ever-treated cohort (or its person-time contribution) is much smaller than the never-treated cohort (i.e., r is large), even for large fractions of immortal time f (see Figure 1).

Figure 1.

Rate ratios for the prescription time-distribution matching method (RR‴) of dealing with immortal time bias in drug effectiveness studies as compared with rate ratios for a time-dependent analysis (RR) as a function of the fraction of immortal time f and for various ratios of the observed person-times in never-treated and ever-treated subjects, r. Specific cases: A) q = 1 (where q is the ratio of assigned and observed immortal times) and x = 0 (i.e., number of failures (reaching outcome) observed during the aggregated matched-immortal time, $N_{\mathrm{IT}}^{\mathrm{\prime }}=0$); B) q = 0.5 and x = 0; C) q = 2 and x = 0; D) q = 1 (assuming that $N_{\mathrm{IT}}^{\mathrm{\prime }}$ is very small compared with N₀) and x = 0.5. The “bias” is the deviation of RR‴/RR from the null value 1.

Further theoretical assessment of this approach is provided in Web Appendix 3.

The sequential Cox approach

The sequential Cox approach (35) mimics a randomized clinical trial for each of the intervals m (m = 0, 1, 2, …, K) during which subjects initiate treatment. Based on treatment initiation at the mth interval, the mth minitrial is created as follows: Only subjects who have not received any treatment before the mth interval are considered. Among these subjects, those initiating treatment during the mth interval (t_m < T^A ≤ t_m+1) are considered the treated group, while the remaining subjects are considered the control group (for an illustration, see Web Figure 4 in Web Appendix 4). This mode of data restructuring closely resembles the “sequence of trials” approach (39) but differs with regard to some details; for example, in the current approach, treatment weights are not used to avoid potential instability in the estimated weights (35, 36) and the control subjects are artificially censored at the time of later treatment initiation (T^A > t_m+1) to avoid confounding due to misclassification of treatment status. To address such artificial censoring, the analysis results are adjusted using inverse-probability-of-censoring weights (36, 40).

We create the pseudodata by aggregating the data from all minitrials (see Web Appendix 4 for implementation details). Under the assumption that the different minitrials may have different baseline hazard functions but all subjects in the same minitrial will have the same baseline hazard function, we can fit a weighted stratified (time-invariant) Cox model to the pseudodata, stratified by the treatment initiation interval and weighted by the inverse-probability-of-censoring weights. Because the same subject may be included in more than 1 minitrial, we use the robust (sandwich) estimator to obtain the standard error (41, p. 170; 42, 43).

Design of simulation

To simulate survival times, we adapt the permutation algorithm (44), which has been validated for generating survival times conditional on time-dependent variables (45, 46) (see Web Appendix 5).

Simulation specifications

In all of our simulations, we generated N = 1,000 data sets, each with n = 2,000 subjects followed for up to m = 30 subsequent visits. For a given percentage of subjects, treatment initiation time T^A is generated from a uniform distribution U(0, 30) (in months). In our simulations 1–3, the percentages of subjects who remain unexposed to treatment for the entire duration of follow-up (infinite T^A) are set to 0%, 25%, and 75%, respectively, for the case in which there is no failure or censoring during follow-up. When subjects fail (reach the outcome) or become censored before they get a chance to initiate treatment, the percentages of never-exposed subjects increase. In all of these simulation settings, we assume an exponential distribution for generating failure times T with the constant λ₀ = 0.01 rate of monthly events throughout the follow-up period. A uniform distribution U(1, 60) months is assumed to generate censoring times T^C. Administrative censoring is set at 30 months of follow-up for all subjects who remain at risk at that time. For each simulated data set, we can calculate the exact percentages of ever-treated and never-treated subjects. For any subject, treatment status is binary for any given month, either exposed or unexposed, where a person is considered exposed throughout the month of the treatment initiation. To focus on the immortal time issue, we assume there are no discontinuations or interruptions for persons who initiate treatment. Additionally, we consider sex as a baseline confounder. A subject's sex is generated based on a Bernoulli distribution where the probability of being male is 0.3 (as might occur in an MS population).

After generating values for the survival time T_i, the censoring time $T_i^C,$ and the treatment and covariate matrix X_im = (A_im, L_i0) for each subject i = 1, 2, …, n for up to m = 30 time periods, the permutation algorithm (44) is used to generate survival data, where treatment $A_{t_m}$ is time-dependent but the confounder L₀ is fixed at the baseline value. Arbitrarily, the parameters for the effects of treatment and sex on the survival outcome are set such that the treatment is associated with the outcome in a harmful direction (a log HR of ψ₁ = 0.5) and males are at a lower risk than females (a log HR of ψ₂ = −0.7).

In order to consider more frequent events, we repeat the Monte Carlo study (similar to simulation 1) with λ₀ = 0.10 (on a monthly scale) (simulation 4). We further assume a gamma distribution (scale = 1/0.01, shape = 0.4) and a Weibull distribution (scale = 1/0.01, shape = 2) for generating failure times T (simulations 5 and 6, respectively; all other settings are similar to simulation 1).

Performance metrics

We assess the performance of the various approaches by means of the following measures:

• $\text{Bias}(\widehat{{\psi }_1})={\sum }_{i=1}^N(\widehat{{\psi }_{1\mathit{i}}}-{\psi }_1)/\mathit{N}:$ the average difference between the true parameters and N = 1,000 estimated parameters (log HR).

• $\text{Standard}\text{deviation}(\widehat{{\psi }_1})=\sqrt{{\sum }_{i=1}^N{(\widehat{{\psi }_{1\mathit{i}}}-\overline{{\psi }_1})}^2/(N-1)}$, where $\overline{{\psi }_1}={\sum }_{i=1}^N{\displaystyle \frac{{\hat{\psi }}_{1i}}{N}}$.

• Model-based (average) standard error: The average of N = 1,000 estimated standard errors of the estimated causal association parameter.

• Coverage probabilities of model-based nominal 95% confidence intervals: proportion of N = 1,000 data sets in which the true parameter is contained in the nominal 95% confidence interval.

• $\text{Mean}\text{squared}\text{error}(\widehat{{\psi }_1})=\sqrt{{\sum }_{i=1}^N{(\widehat{{\psi }_{1\mathit{i}}}-{\psi }_1)}^2/N}$.

SIMULATION RESULTS

Description of the simulated data

To describe the data sets generated from each of the simulations, we generated 1 data set for each simulation with a larger number of subjects (n = 25,000). Figure 2 shows the cumulative percentages of the cohort having initiated treatment over the 30 follow-up periods.

Figure 2.

Cumulative percentages of a simulated cohort of 25,000 multiple sclerosis patients having initiated treatment over 30 follow-up periods. A) Simulation 1, with percentages of “failure,” “never-treated,” and “ever-treated” patients being 0.96%, 32.78%, and 67.22%, respectively; B) simulation 2, with the above percentages being 0.99%, 49.48%, and 50.52%, respectively; C) simulation 3, with the above percentages being 0.99%, 83.19%, and 16.81%, respectively. These percentages refer to percentages of subjects who fail (reach the outcome), remain unexposed to treatment, and initiate treatment, respectively, over the 30-month period.

Rare event condition

The results from the simulations of the rare event condition (λ₀ = 0.01 on a monthly time scale), simulations 1–3, are given in Tables 1–3, respectively.

Table 1.

Results Derived From Analytical Approaches to Adjusting for Immortal Time Bias in a Simulation of 1,000 Data Sets, Each Containing 2,000 Multiple Sclerosis Patients Followed for Up to 30 Time Intervals (Rare Event Case λ₀ = 0.01) (Simulation 1)

Approacha	Bias ($\widehat{{\psi }_{\mathbf{1}}}$)	SD ($\widehat{{\psi }_{\mathbf{1}}}$)	SE ($\widehat{{\psi }_{\mathbf{1}}}$)	CP	MSE ($\widehat{{\psi }_{\mathbf{1}}}$)
TD Coxb	−0.004	0.122	0.121	0.955	0.015
Included IT	−2.307	0.109	0.108	0.000	5.332
Excluded IT	−1.360	0.102	0.102	0.000	1.859
PTDM	−1.411	0.144	0.143	0.000	2.011
Sequential Coxc,d,e	−0.002	0.157	0.150	0.941	0.025

Abbreviations: CP, coverage probability; IT, immortal time; MSE, mean squared error; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a When estimating the treatment effect, the baseline covariate L₀ is included in all of the models under consideration. The results from the TD Cox model are considered the standard for comparison purposes. Results are provided on the log hazard ratio scale.

^b Cox proportional hazards model with time-dependent exposure.

^c In the sequential Cox approach, the corresponding inverse-probability-of-censoring weights model is fitted using Aalen's additive regression model (58, 59), adjusting for $A_{t_m}$ and L₀ to predict future censoring status.

^d A robust (sandwich) estimator was used to obtain the SE.

^e Event times were generated from an exponential distribution with rate 0.01.

Table 3.

Results Derived From Analytical Approaches to Adjusting for Immortal Time Bias in a Simulation of 1,000 Data Sets, Each Containing 2,000 Multiple Sclerosis Patients Followed for Up to 30 Time Intervals (Rare Event Case λ₀ = 0.01) (Simulation 3)^a

Approachb	Bias ($\widehat{{\psi }_{\mathbf{1}}}$)	SD ($\widehat{{\psi }_{\mathbf{1}}}$)	SE ($\widehat{{\psi }_{\mathbf{1}}}$)	CP	MSE ($\widehat{{\psi }_{\mathbf{1}}}$)
TD Coxc	−0.001	0.146	0.145	0.956	0.021
Included IT	−0.832	0.138	0.135	0.000	0.711
Excluded IT	−0.136	0.146	0.142	0.850	0.040
PTDM	−0.101	0.152	0.151	0.912	0.033
Sequential Coxd,e,f	−0.008	0.146g	0.145	0.954	0.021

Abbreviations: CP, coverage probability; IT, immortal time; MSE, mean squared error; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a At least 75% subjects unexposed at baseline remained untreated during the entire follow-up period.

^b When estimating the treatment effect, the baseline covariate L₀ is included in all of the models under consideration. The results from the TD Cox model are considered the standard for comparison purposes. Results are provided on the log hazard ratio scale.

^c Cox proportional hazards model with time-dependent exposure.

^d In the sequential Cox approach, the corresponding inverse-probability-of-censoring weights model is fitted using Aalen's additive regression model (58, 59), adjusting for $A_{t_m}$ and L₀ to predict future censoring status.

^e A robust (sandwich) estimator was used to obtain the SE.

^f Event times were generated from an exponential distribution with rate 0.01.

^g The SD from the sequential Cox model (0.1456) was slightly smaller than the SD from the TD Cox model fit (0.1460).

When the immortal time is misclassified as exposed time, we see a substantial downward bias. The bias improves slightly when excluded immortal time and PTDM approaches are used, but these estimate are still off the target. As the ratio for the subjects in the never-treated group versus the ever-treated group increases, the bias monotonically goes down (see Tables 2 and 3). The variability of the estimator is larger for PTDM than for the time-dependent Cox results. Interestingly, the standard errors of the estimates from both the misclassified and excluded immortal-time approaches are smaller than those for the time-dependent Cox results, but the mean squared errors are generally higher. When the sequential Cox approach is applied, in all 3 of these simulations, the biases are negligible and the average coverage probabilities of the model-based nominal 95% confidence intervals are close to 0.95; both are comparable to the time-dependent Cox results. This estimator is generally more variable than that from the time-dependent Cox model fit (from simulations 1 and 2). However, from simulation 3 (Table 3), we see that exceptions may be possible.

Table 2.

Results Derived From Analytical Approaches to Adjusting for Immortal Time Bias in a Simulation of 1,000 Data Sets, Each Containing 2,000 Multiple Sclerosis Patients Followed for Up to 30 Time Intervals (Rare Event Case λ₀ = 0.01) (Simulation 2)^a

Approachb	Bias ($\widehat{{\psi }_{\mathbf{1}}}$)	SD ($\widehat{{\psi }_{\mathbf{1}}}$)	SE ($\widehat{{\psi }_{\mathbf{1}}}$)	CP	MSE ($\widehat{{\psi }_{\mathbf{1}}}$)
TD Coxc	−0.000	0.117	0.116	0.949	0.014
Included IT	−1.394	0.099	0.100	0.000	1.953
Excluded IT	−0.698	0.104	0.104	0.000	0.499
PTDM	−0.558	0.134	0.131	0.012	0.329
Sequential Coxd,e,f	−0.001	0.127	0.126	0.950	0.016

Abbreviations: CP, coverage probability; IT, immortal time; MSE, mean squared error; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a At least 25% of subjects unexposed at baseline remained untreated during the entire follow-up period.

^b When estimating the treatment effect, the baseline covariate L₀ is included in all of the models under consideration. The results from the TD Cox model are considered the standard for comparison purposes. Results are provided on the log hazard ratio scale.

^c Cox proportional hazards model with time-dependent exposure.

^d In the sequential Cox approach, the corresponding inverse-probability-of-censoring weights model is fitted using Aalen's additive regression model (58, 59), adjusting for $A_{t_m}$ and L₀ to predict future censoring status.

^e A robust (sandwich) estimator was used to obtain the SE.

^f Event times were generated from an exponential distribution with rate 0.01.

When more events are available

Results from the more frequent event condition (λ₀ = 0.10 on a monthly time scale) are presented in Table 4. The bias for misclassified, excluded immortal-time, and PTDM approaches exhibiting large bias in Table 1 is considerably reduced but is still large, whereas the time-dependent Cox and sequential Cox approaches again exhibit negligible bias. For the PTDM approach, less bias is expected when the event rate is higher, as is discussed via an example of the analytical results in Web Appendix 3. In addition, the standard errors are much larger when failure rates are low.

Table 4.

Results Derived From Analytical Approaches to Adjusting for Immortal Time Bias in a Simulation of 1,000 Data Sets, Each Containing 2,000 Multiple Sclerosis Patients Followed for Up to 30 Time Intervals (More Frequent Event λ₀ = 0.10) (Simulation 4)^a,b

Approachc	Bias ($\widehat{{\psi }_{\mathbf{1}}}$)	SD ($\widehat{{\psi }_{\mathbf{1}}}$)	SE ($\widehat{{\psi }_{\mathbf{1}}}$)	CP	MSE ($\widehat{{\psi }_{\mathbf{1}}}$)
TD Coxd	−0.004	0.061	0.060	0.946	0.004
Included IT	−1.475	0.062	0.057	0.000	2.179
Excluded IT	−0.412	0.066	0.057	0.000	0.174
PTDM	−0.522	0.071	0.065	0.000	0.278
Sequential Coxe,f	−0.037	0.077	0.072	0.905	0.007

Abbreviations: CP, coverage probability; IT, immortal time; MSE, mean squared error; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a λ₀ = 0.10 on a monthly time scale implies that almost all subjects should reach the outcome within the first 10 months.

^b Event times were generated from an exponential distribution with rate 0.1.

^c When estimating the treatment effect, the baseline covariate L₀ is included in all of the models under consideration. The results from the TD Cox model are considered the standard for comparison purposes. Results are provided on the log hazard ratio scale.

^d Cox proportional hazards model with time-dependent exposure.

^e In the sequential Cox approach, the corresponding inverse-probability-of-censoring weights model is fitted using Aalen's additive regression model (58, 59), adjusting for $A_{t_m}$ and L₀ to predict future censoring status.

^f A robust (sandwich) estimator was used to obtain the SE.

When event times are generated from gamma and Weibull distributions

When event times are generated from a gamma distribution and a Weibull distribution, the results (Web Tables 1 and 2) closely resemble those obtained when event times are generated from an exponential distribution (Table 1).

Application to MS

We applied the methods described in this paper to data on the British Columbia MS cohort used earlier in the BeAMS Study (1995–2008) (38, 47–51). Web Appendix 6 shows the baseline characteristics of this MS cohort (see Web Table 3) and gives the study eligibility and exclusion criteria. In this retrospective observational study, the outcome measure was time from β-interferon eligibility to a “confirmed and sustained” Expanded Disability Status Scale score of 6, considered irreversible disability when all subsequent such scores were at least 6, with 1 or more measurements being taken at least 150 days later (52, 53). The goal was to assess the association of β-interferon with irreversible disease progression in MS patients.

Results are reported in Table 5. None of the approaches showed a significant hazard ratio. The hazard ratio estimated from the included immortal time approach was the lowest, and this was expected from the simulation. The hazard ratio estimated from the sequential Cox approach was slightly lower than that estimated from the time-dependent Cox analysis, while that estimated from the PTDM was even lower. The distributions of excluded times from both groups when using the PTDM method looked reasonably similar (Figure 3). Because the PTDM approach produces different estimates from the same data due to the random sampling of the control subjects, we estimated the hazard ratio from the British Columbia MS data 1,000 times (average hazard ratio = 1.26; listed in Table 5). The distribution of the estimated hazard ratios was moderately symmetrical (see Web Figure 5). Density plots of the estimated inverse-probability-of-censoring weights are shown in Web Figure 6.

Table 5.

Estimated Parameters (Using the Hazard Scale) for Multiple Sclerosis Patients From British Columbia, Canada, 1995–2008

Approacha	$\stackrel{\mathbf{ˆ}}{\mathbf{\text{HR}}}$	SE ($\stackrel{\mathbf{ˆ}}{\mathbf{\text{HR}}}$)	95% CI
TD Coxb	1.29	0.23	0.91, 1.82
Included IT	0.98	0.19	0.67, 1.42
Excluded IT	1.35	0.26	0.93, 1.96
PTDM	1.12	0.21	0.77, 1.63
PTDM (average)c	1.26	0.25	0.86, 1.85
Sequential Coxd	1.22	0.33	0.72, 2.08

Abbreviations: CI, confidence interval; HR, hazard ratio; IPCW, inverse-probability-of-censoring weights; IT, immortal time; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a To demonstrate the impact of the various immortal or immune time (pertains to nonfatal outcomes) adjustment methods, we assume here that once the subjects initiate β-interferon use, they continue taking the drug without any discontinuation until they develop the outcome (time to irreversible progression of disability) or are censored. All of the models under consideration adjust for the baseline confounders L₀ (age, sex, disease duration, and Expanded Disability Status Scale score) when estimating the treatment effect. The Cox model is used in all of these approaches.

^b Cox proportional hazards model with time-dependent exposure.

^c The HR was estimated from British Columbia multiple sclerosis data (38, 47) 1,000 times and averaged. The mean HR was 1.26 (ranging from 1.07 to 1.52; SD, 0.08), the average SE was 0.25 (SD, 0.02), and the average 95% CI was 0.86, 1.85.

^d To create the IPCW for the sequential Cox approach, Aalen's additive regression model (58, 59) is used. The IPCW denominator model adjusts for $A_{t_m}$ and L₀. The average weight is 1.00 (SD, 0.13), ranging from 0.66 to 3.45. Estimated IPCW distributions are unimodal and symmetrical.

Figure 3.

Matched wait periods (or matched immortal times), in years, from the prescription time-distribution matching approach in a simulated cohort of 25,000 multiple sclerosis patients initiating β-interferon treatment, British Columbia, Canada, 1995–2008.

To better understand the possible level of bias from each approach in this context, we conducted another Monte Carlo study, choosing data simulation settings similar to those of the MS data set. Simulation specifications are shown in Web Appendix 7 (simulated cohort characteristics are shown in Web Table 4 and Web Figure 7). The results are reported in Table 6. Here, the level of bias from each approach is much less than that reported for the previous simulations (Table 6 vs. Tables 1 and 2) but similar to that in simulation 3 (Table 3).

Table 6.

Comparison of Analytical Approaches to Adjusting for Immortal Time Bias in a Simulation Inspired by Multiple Sclerosis Data From 1,000 Data Sets, Each Containing 1,700 Subjects Followed for Up to 150 Months^a

Approachb	Bias ($\widehat{{\psi }_{\mathbf{1}}}$)	SD ($\widehat{{\psi }_{\mathbf{1}}}$)	SE ($\widehat{{\psi }_{\mathbf{1}}}$)	CP	MSE ($\widehat{{\psi }_{\mathbf{1}}}$)	95% CI for Mean Biasc
TD Coxd	0.000	0.136	0.135	0.949	0.018	−0.008, 0.008
Included IT	−0.833	0.122	0.122	0.000	0.709	−0.841, −0.825e
Excluded IT	−0.027	0.134	0.129	0.942	0.019	−0.035, −0.019e
PTDM	−0.105	0.139	0.139	0.890	0.030	−0.114, −0.096e
Sequential Cox	−0.001	0.231	0.207	0.928	0.053	−0.015, 0.013

Abbreviations: CI, confidence interval; CP, coverage probability; IT, immortal time; MSE, mean squared error; PTDM, prescription time-distribution matching; SD, standard deviation; SE, standard error; TD, time-dependent.

^a The simulation included 4 baseline covariates and a time-dependent treatment exposure.

^b Results are provided on the log hazard ratio scale.

^c The 95% CI for the mean bias was calculated as bias ± 1.96 × SD/$\sqrt{(N)}$ across the 1,000 simulations.

^d Cox proportional hazards model with time-dependent exposure.

^e Indicative of a finite-sample bias across 1,000 simulations.

DISCUSSION

In observational studies of drug effectiveness, subjects may not be exposed to the treatment at the beginning of follow-up. Statistical procedures, such as the time-dependent Cox model, are known to deal with time-dependent treatment. However, in the medical literature, it is common to see the Cox model applied on the basis of baseline treatment information, likely for the convenience of model-fitting and group-based interpretation (54). Therefore, there is a need for methods that are capable of handling the time-dependent nature of longitudinal data and will also help us to better understand the treatment-outcome mechanism. To this end, the PTDM approach has been proposed for the situation where treatment initiation occurs later than cohort entry (13). The sequential Cox approach has been proposed for more complex longitudinal settings (35), but we considered a simplified version of this approach to deal with the immortal time bias.

In the PTDM approach, treatment exposure is converted into a time-independent variable so that a simple Cox model for treatment-group comparison can be applied. New time zeros are defined after excluding the observed and assigned immortal times in the ever- and never-treated groups. The excluded times (wait periods) for the ever-treated and never-treated groups follow the same distribution. However, the baseline is not the same for all subjects. The question of whether this approach (assigning the baseline borrowed from the ever-treated cohort) adequately addresses the immortal time bias has not previously been investigated in a systematic fashion, to our knowledge. From our bias-quantifying equations (Web Appendices 1 and 3) and the results of our simulations, we can see that the bias is still substantial in the PTDM analysis and the direction of the bias is negative, highlighting the value of setting a well-defined time zero or baseline. The results of simulations 1–3 (also in Web Appendix 1) identify an increase in the ratio of subjects in the never-treated group to subjects in the ever-treated group as an important design parameter for reducing bias associated with the PTDM method (biases were −1.411, −0.558, and −0.101 for the ratios 0.49, 0.98, and 4.95, respectively). From simulation 4, we learned that less bias is expected for the PTDM approach when the event rate is higher. We also considered 2 other popular event-time-generating distributions, namely the gamma and Weibull distributions (simulations 5 and 6, respectively), and the results were consistent with those from simulation 1.

In contrast, the sequential Cox approach allows a consistent definition of baseline and utilizes all subjects. The baseline is set on the basis of treatment initiation, and the subjects are stratified accordingly. This approach recreates the covariate process at each treatment start using the minitrial approach (36) and works very well in our simulation studies in terms of bias. In practice, as the time-dependent Cox model approach can easily deal with the immortal time bias and is simpler to implement, one may not be inclined to use the sequential Cox approach, because additional steps are involved in the data-restructuring for the latter approach (i.e., to create minitrials and then pseudodata). The data associated with a given minitrial in a sequential Cox approach, however, can be extracted and separated quite easily from the pseudodata, and it is straightforward to compare the treatment associations (of early versus late treatment initiation) with the outcome. It is also possible to estimate the treatment association for patients with a specific level of a covariate at treatment initiation (35). Such detailed scrutiny using this minitrial approach could yield insights about the data which may be hard to gain using the time-dependent Cox approach.

We applied the methods under consideration to estimate the association of β-interferon with disease progression in the British Columbia MS cohort. In comparison with results from the time-dependent Cox model (hazard ratio (HR) = 1.29, 95% confidence interval (CI): 0.91, 1.81), the sequential Cox approach provided a slightly lower association estimate with a wider confidence interval (HR = 1.22, 95% CI: 0.72, 2.08). Repeating the PTDM analysis 1,000 times, the average results (HR = 1.26, average 95% CI: 0.86, 1.85) were quite similar to those from the time-dependent Cox approach, although the average confidence interval for the estimated treatment association was slightly wider. We designed another Monte Carlo study using simulation settings reflecting our MS data set to explore further the level of bias from each approach. The smaller amount of bias in this simulation for each approach may have been a consequence of the fact that the ever-treated group was much smaller than the never-treated group in this simulation (19% vs. 81%; ratio: 4.26). This phenomenon also explains why the PTDM approach provided estimates close to those from the time-dependent Cox model fit (29% user vs. 71% nonuser) (13). Even in this simulation setting, the time-dependent Cox model and sequential Cox approach were still associated with less bias than the PTDM approach.

Similar to other simulation studies, we investigated only a few scenarios. However, the assumptions underlying the data simulation were consistent with patterns typical of epidemiologic observational survival studies, where treatment initiation may happen later for some subjects than others (7). Furthermore, our assumption of no discontinuations or interruptions in the treatment was restrictive and may not be suitable for more complex disease scenarios in which an investigator might wish to assess different treatment strategies (i.e., switching between therapies) over the course of time. Substantial immortal time bias was introduced by group-based Cox model analyses in the scenarios investigated (analytical expressions for PTDM approach in Web Appendix 3 also support this finding). However, even in these scenarios, the sequential Cox method appeared to estimate the target parameter adequately. Variance estimation is still a challenge with the sequential Cox model and similar methods (55, 56), and the resampling methods suggested for estimating the variance are computationally demanding (35, 57). Future research could focus on assessing the ability of the sequential Cox approach to adjust for time-dependent confounders (37).