The Impact of Implementation Fidelity on Mortality Under a CD4-Stratified Timing Strategy for Antiretroviral Therapy in Patients With Tuberculosis

Abstract

Among patients with tuberculosis and human immunodeficiency virus type 1, CD4-stratified initiation of antiretroviral therapy (ART) is recommended, with earlier ART in those with low CD4 counts. However, the impact of implementation fidelity to this recommendation is unknown. We examined a prospective cohort study of 395 adult patients diagnosed with tuberculosis and human immunodeficiency virus between August 2007 and November 2009 in Kinshasa, Democratic Republic of the Congo. ART was to be initiated after 1 month of tuberculosis treatment at a CD4 count of <100 cells/mm³ or World Health Organization stage 4 (other than extrapulmonary tuberculosis) and after 2 months of tuberculosis treatment at a CD4 count of 100–350 cells/mm³. We used the parametric g-formula to estimate the impact of implementation fidelity on 6-month mortality. Observed implementation fidelity was low (46%); 54% of patients either experienced delays in ART initiation or did not initiate ART, which could be avoided under perfect implementation fidelity. The observed mortality risk was 12.0% (95% confidence interval (CI): 8.2, 15.7); under complete (counterfactual) implementation fidelity, the mortality risk was 7.8% (95% CI: 2.4, 12.3), corresponding to a risk reduction of 4.2% (95% CI: 0.3, 8.1) and a preventable fraction of 35.1% (95% CI: 2.9, 67.9). Strategies to achieve high implementation fidelity to CD4-stratified ART timing are needed to maximize survival benefit.

In 2012, an estimated 1.1 million (14%) of the 8.6 million patients diagnosed with active tuberculosis (TB) disease worldwide were also infected with human immunodeficiency virus (HIV) (1). Although, under current World Health Organization (WHO) guidelines, all people diagnosed with TB and HIV are eligible for antiretroviral therapy (ART), only 57% initiated ART during TB treatment in 2012 (1). Findings from randomized controlled trials demonstrated that early ART, defined as within 2–4 weeks of start of TB treatment, reduces mortality in patients with a CD4 count of <50 cells/mm³. In these trials, early ART initiation did not result in statistically significant lower mortality in patients with CD4 counts of >50 cells/mm³ (2–4). On the basis of these findings, the WHO recommends ART initiation within 2 weeks of TB treatment for patients with a CD4 count of <50 cells/mm³ and within 8 weeks for all people diagnosed with HIV and TB (5).

Findings from randomized controlled trials are not always replicable in less strictly controlled settings. Observational studies from sub-Saharan Africa have found that most patients initiate ART late, after 8 weeks of TB treatment (6–10). Lack of integration of TB and HIV treatment services has been identified as one of the key contributors to this delay (9, 10). Interventions to integrate TB and HIV services have been shown to reduce, but not eliminate, delay in ART initiation (11, 12).

Implementation fidelity, defined as the degree to which an intervention is implemented as intended, is a potential modifier of the relationship between an intervention and its intended outcome and is important to translation of evidence-based recommendations into clinical practice (13, 14). Achieving high implementation fidelity can help to replicate the success an intervention has achieved in randomized controlled trials (13). The relatively low coverage of timely ART initiation in patients with TB suggests that implementation fidelity to the 2012 WHO guidelines for timing of ART initiation in TB patients may be a challenge in routine clinical settings, even with integrated HIV/TB treatment. Data on implementation fidelity in resource-limited settings and its impact on desired outcomes are limited.

We aimed to quantify the impact of implementation fidelity to CD4-stratified timing of ART initiation for TB patients on mortality in a prospective cohort of patients receiving integrated TB/HIV treatment at the primary care level in Kinshasa, Democratic Republic of the Congo.

METHODS

Setting, study population, and study procedures

The Integrating Tuberculosis and Antiretroviral Treatment (ITART) Study was conducted at 5 primary health clinics in the Democratic Republic of the Congo and enrolled patients between August 2007 and November 2009 (15). All patients diagnosed with HIV and TB per national Democratic Republic of the Congo guidelines were offered study participation; 3 refused. At baseline, relevant demographic and clinical data were collected, and CD4 count testing was performed.

Study nurses (1 per clinic) were trained to implement a CD4-stratified timing strategy for ART initiation in accord with the 2006 WHO guidelines. Patients were eligible to initiate ART at completion of the first month of TB treatment if the baseline CD4 count was below 100 cells/mm³ or WHO stage 4 condition other than extrapulmonary TB, as well as at 2 months of TB treatment if the baseline CD4 count was between 100 and 350 cells/mm³. Patients with a baseline CD4 count of >350 cells/mm³ were scheduled for repeat CD4 testing at month 5 of TB treatment and were eligible to initiate ART at the end of TB treatment if the CD4 count had dropped to ≤350 cells/mm³ or when an incident WHO stage 4 condition occurred. When CD4 counts were not available (because of a shortage of reagents), all patients were considered eligible for ART initiation at 1 month. According to the legal framework of the Democratic Republic of the Congo, nurses’ decisions to initiate ART needed approval by a consulting physician. Providers assessed whether the patient was or was not tolerating TB treatment at the time of ART eligibility on the basis of clinical judgment as the 2006 WHO guidelines did not define what constitutes tolerance (or intolerance) of TB treatment.

Patients were seen by study nurses weekly during the 2-month intensive phase, monthly during the subsequent 4-month continuation phase, and at completion of TB treatment. Patients were considered lost to follow-up if they were more than 3 days late for a scheduled clinic visit and could not be located by phone or home visit.

The ITART Study was approved by institutional review boards at both the University of North Carolina at Chapel Hill and the University of Kinshasa. All patients provided written informed consent for participation in the study.

Study population and definitions

This analysis was limited to individuals aged ≥13 years who were enrolled in the ITART Study within 1 month of TB treatment start, were treated with ART, were naïve, and had a CD4 count of ≤350 cells/mm³ within 30 days of the start of TB treatment (at baseline).

Outcome

Mortality was defined as all-cause mortality during the first 6 months of TB treatment, the duration of first-line TB treatment. Using the standard approach for observational cohort studies, we calculated the 6-month risk of mortality on the basis of the observed outcomes. Patients who were lost to follow-up prior to 6 months were assigned a missing outcome and did not contribute to the numerator or denominator of the observed risk.

Risk factors for mortality

We used a logistic regression model to assess baseline covariates, including the timing of ART initiation, as potential predictors of mortality. We first ran a full logistic model containing all the by selected covariates. Subsequently, we used a backwards elimination stepwise method to generate a final (reduced) predictive model. Covariates were assessed in order from the highest to the lowest Wald χ² and eliminated from the model by using the likelihood ratio test with an α = 0.10. We estimated crude and adjusted odds ratios with 95% confidence intervals.

Assessment of implementation fidelity

“Implementation fidelity” is defined as the degree to which programs are implemented as intended, with a focus on content or frequency of the intervention (13). In this study, we confined implementation fidelity to the CD4-stratified ART initiation strategy as the proportion of individuals who initiated timely ART, that is, according to a priori–defined CD4 criteria. We calculated each patient's timing of ART initiation by comparing the ART start date with the TB treatment start date. We categorized the timing of ART initiation as per CD4-stratified strategy (“per strategy”) or deviating from CD4-stratified strategy (“not per strategy”). To accommodate a combination of scheduling limitations, clinic closures on weekends and holidays, and limited availability of a consulting physician as required for ART initiation per the Democratic Republic of the Congo legal framework, a 5-day grace period was added to the 1- or 2-month TB treatment to define ART initiation per strategy. Participants who initiated ART prior to the time they became eligible plus 5 days were categorized as per strategy. Participants who died or were lost to follow-up prior to eligibility for ART and had not initiated ART were categorized as initiating ART per strategy, since not initiating ART prior to death or loss to follow-up did not constitute deviation from the CD4-stratified strategy. In sensitivity analyses, we explored the impact of narrowing the definition of ART initiation per strategy to exclude patients who were lost to follow-up prior to the time of ART eligibility, a subset of patients who could have started timely ART had they been retained in care.

Differences in the proportions and medians of baseline characteristics between patients initiating ART per strategy and those initiating not per strategy were assessed by using χ² or Fisher's exact tests and Kruskal-Wallis tests, respectively.

Estimation of the causal effect of implementation fidelity on mortality

To estimate the causal effect of implementation fidelity, we compared mortality in the study population under observed intervention fidelity with mortality in the study population with complete implementation fidelity (Figure 1) (16, 17). Standard multivariable regression would not easily allow us to estimate the difference in risk in mortality at the population level attributable to implementation fidelity. We overcame this by using the parametric g-formula to estimate mortality in the cohort under the counterfactual scenario of complete implementation fidelity (18–20). A step-by-step overview of this methodological approach is presented in Appendix 1, and the worked example is presented as Appendix 2 (18).

Figure 1.

Impact on mortality of perfect versus observed implementation fidelity to CD4-stratified timing of antiretroviral therapy (ART), Integrating Tuberculosis and Antiretroviral Treatment Study, 2007–2009. All individuals were assigned to the timing per protocol (top, all black), but only 44% were observed to adhere to timing per protocol (middle; diagonal lines represent nonadherence to protocol). In analysis, we wished to understand the difference between what was observed (bottom, left) and the counterfactual exposure distribution in which all subjects adhered to timing per protocol (bottom, right). Note that probability of outcome (mortality) is not explicitly shown in these figures.

We built a logistic regression model to assess the association between initiating ART per strategy and mortality (step 1), including baseline covariates identified as potential confounders using a directed acyclic graph. We then used parameter estimates from the model to calculate the predicted probability of death for each patient based on their baseline covariates and observed ART timing (step 2). This modeling method imputes an outcome for each patient on the basis of the average risk across patients with observed outcomes with the same baseline characteristics. Consequently, the outcome of participants who were lost to follow-up is no longer missing, as these participants are assigned an outcome on the basis of their baseline characteristics. By averaging these predicted probabilities of death across all participants, we estimated the risk of mortality in the full cohort under the observed, real-life level of implementation fidelity (step 3).

To estimate the causal effect of implementation fidelity, we estimated a (counterfactual) probability of death for each participant, corresponding to what would have happened to each participant had he or she initiated ART per strategy. For participants who did initiate ART per strategy, this predicted probability of death is the same as that calculated in step 2; for participants who did not initiate ART per strategy, we estimated this probability based on the outcomes of patients with similar baseline characteristics who did initiate ART per strategy (step 4). By averaging these predicted probabilities, we estimated the risk of mortality in the full cohort under a scenario of complete (100%) implementation fidelity (step 5).

We then calculated the risk difference by subtracting this mortality risk estimate in the cohort with complete fidelity from the mortality risk estimate in the cohort with observed fidelity (step 6). Bootstrapping was used to generate the 95% confidence interval around the risk difference. This was done by creating multiple (n = 500) data sets through random selection of 395 individuals with replacement from the original ITART Study population, followed by rerunning step 1 through step 6, and using the standard error across all the risk difference estimates (step 7).

We estimated the preventable fraction similarly, by dividing the risk difference by the estimated risk in the cohort under observed, real-life implementation fidelity and bootstrapping the corresponding 95% confidence interval. This measure is interpreted as the fraction of mortality that could be prevented if 100% implementation fidelity were achieved.

RESULTS

Baseline characteristics of the analytical cohort

Between August 2007 and November 2009, 599 participants enrolled in the ITART Study. Among them, 204 were sequentially excluded on the basis of age <13 years (n = 18), lack of baseline CD4 count (n = 88), enrollment more than 1 month after TB treatment initiation (n = 1), exposure to ART prior to enrollment (n = 0), and CD4 count >350 (n = 97). The remaining 395 participants were included in the analysis.

Baseline characteristics of the patients are presented in Table 1. The type of TB was smear-positive pulmonary in 35%, smear-negative pulmonary in 45%, and extrapulmonary in 20% of patients. Just over half (59%) were female, and the median age was 38 (interquartile range (IQR), 32–45) years. Most patients were underweight (median body mass index (weight (kg)/height (m)²), 17.8; IQR, 16.5–19.7); 59.2% had a body mass index of <18.5. Patients presented late in the HIV disease process, with a median CD4 count of 131 cells/mm³ (IQR, 63–224). Of the 143 (36%) patients eligible for ART at 1 month, 77 (54%) had a CD4 count of <50 cells/mm³, 64 (45%) had a CD4 count of 50–99 cells/mm³, and 2 (1%) had a CD4 count of >100 cells/mm³ and WHO stage 4. Of the 252 (64%) patients eligible for ART at 2 months, 136 (54%) had a CD4 count of 100–199 cells/mm³ and 116 (46%) had a CD4 count of 200–350 cells/mm³. A few patients (n = 24) (6%) had a contraindication to 1 or more antiretroviral drugs.

Table 1.

Baseline Characteristics of Participants by Timing of Antiretroviral Therapy Initiation per CD4-Stratified Strategy, Integrating Tuberculosis and Antiretroviral Treatment Study, 2007–2009

Baseline Characteristic	All Patients (n = 395)		Timing of Antiretroviral Therapy Initiation				P Value
			Per Strategy (n = 183)		Not Per Strategy (n = 212)
	No.	%	No.	%	No.	%
Sex
Female	231	58.5	103	56.3	128	60.4	0.41
Male	164	41.5	80	43.7	84	39.6
Age, yearsa	38 (32–45)		38 (32–45)		38 (32–45)		0.97
Age, years
<30	65	16.5	32	17.5	33	15.6	0.61
30–39	158	40.0	73	39.9	85	40.1	0.97
40–49	129	32.7	56	30.6	73	34.4	0.42
≥50	43	10.9	22	12.0	21	9.9	0.50
Tuberculosis type
Smear positive, pulmonary	139	35.2	71	38.8	68	32.1	0.16
Smear negative, pulmonary	176	44.6	77	42.1	99	46.7	0.36
Extrapulmonary	80	20.3	35	19.1	45	21.2	0.60
CD4 count, cells/mm3a	131 (63–224)		151 (77–243)		113 (60–190)		0.002
CD4 count, cells/mm3
<50	77	19.5	33	18.0	44	20.8	0.50
50–99	64	16.2	21	11.5	43	20.3	0.02
100–199	136	34.4	61	33.3	75	35.4	0.67
200–350	118	29.9	68	37.2	50	23.6	0.003
Body mass indexa,b	17.8 (16.5–19.7)		17.7 (16.4–19.6)		17.8 (16.6–19.8)		0.35
Body mass indexb
Underweight (<18.5)	234	59.2	114	62.3	120	56.6	0.25
Normal (18.5–24.9)	148	37.5	64	35.0	84	39.6	0.34
Overweight (25.0–29.9)	11	2.8	4	2.2	7	3.3	0.56
Obese (≥30.0)	2	0.5	1	0.6	1	0.5	1.0
Toleration of tuberculosis drugs	322	81.5	175	95.6	147	69.3	<0.001
Contraindication to any ARV drug	24	6.1	7	3.8	17	8.0	0.09
WHO clinical stage 4	90	22.8	41	22.4	49	23.1	0.87

Abbreviations: ARV, antiretroviral; WHO, World Health Organization.

^a Median (interquartile range).

^b Weight (kg)/height (m)².

Implementation fidelity to CD4-stratified timing of ART initiation

Overall, 183 (46%) participants initiated ART per strategy. Among the 212 (54%) participants who initiated ART not per strategy, 53 (25%) never initiated ART, and 159 (75%) initiated ART with a median delay of 11 days (IQR, 4–24) beyond the time of eligibility plus 5 days. The median delay did not differ by eligibility category (12 days for those eligible at 1 month vs. 10 days for those at 2 months) (P = 0.62). Patients whose timing of ART initiation was per strategy had a higher CD4 count (151 cells/mm³ vs. 113 cells/mm³) (P = 0.002) and a higher frequency of tolerating their TB drugs (96% vs. 70%) (P < 0.0001) at the scheduled time of ART initiation than did patients whose timing of ART initiation was not per strategy.

Predictors of mortality in the first 6 months of TB treatment

Results of predictive modeling of mortality are presented in Table 2. In a series of crude (unadjusted) models, a CD4 count of <50 cells/mm³ (crude odds ratio (OR) = 6.0, 95% confidence interval (CI): 2.2, 16.4) and TB treatment intolerance (crude OR = 12.3, 95% CI: 5.6, 27.3) were predictive of mortality. In the final (reduced) model, TB treatment intolerance (adjusted OR = 12.7, 95% CI: 4.8, 33.2), a CD4 count of <50 cells/mm³ (adjusted OR = 7.3, 95% CI: 2.3, 23.3), and male sex (adjusted OR = 2.4, 95% CI: 1.0, 5.6) were predictive of mortality. In addition, underweight (adjusted OR = 2.2, 95% CI: 0.9, 5.6) and not initiating ART per strategy (adjusted OR = 2.5, 95% CI: 0.9, 6.6) doubled the risk of mortality, although not statistically significantly.

Table 2.

Predictors of Mortality in 395 HIV- and TB-infected Participants Under Observed Implementation Fidelity to CD4-Stratified Timing of Antiretroviral Therapy Initiation, Integrating Tuberculosis and Antiretroviral Treatment Study, 2007–2009

Predictor	Crude		Adjusted
			Full Model		Reduced Model
	OR	95% CI	OR	95% CI	OR	95% CI
Sex
Male	1.78	0.86, 3.65	1.97	0.76, 5.09	2.37	1.00, 5.61
Female	1.00	Referent	1.00	Referent	1.00	Referent
Age, years
<30	1.36	0.44, 4.17	1.26	0.32, 4.96
30–39	1.00	Referent	1.00	Referent
40–49	1.72	0.71, 4.14	1.86	0.64, 5.37
≥50	2.40	0.82, 7.07	2.96	0.77, 11.40
Type of tuberculosis
Smear positive, pulmonary	1.00	Referent	1.00	Referent
Smear negative, pulmonary	2.01	0.81, 4.98	1.43	0.48, 4.24
Extrapulmonary	2.05	0.71, 5.91	1.11	0.07, 17.79
CD4 count, cells/mm3
<50	5.99	2.19, 16.36	7.66	2.27, 25.80	7.30	2.29, 23.30
50–99	1.81	0.53, 6.19	1.92	0.47, 7.86	1.94	0.49, 7.59
100–199	1.00	Referent	1.00	Referent	1.00	Referent
200–350	1.36	0.44, 4.17	2.19	0.58, 8.20	1.95	0.56, 6.79
Body mass indexa
Underweight (<18.5)	1.92	0.87, 4.27	2.46	0.94, 6.20	2.21	0.89, 5.62
Not underweight (≥18.5)	1.00	Referent	1.00	Referent	1.00	Referent
TB treatment intolerance
Yes	12.33	5.56, 27.31	12.75	4.69, 34.71	12.65	4.82, 33.20
No	1.00	Referent	1.00	Referent	1.00	Referent
Contraindication to any ARV
Yes	0.90	0.20, 4.04	1.43	0.48, 4.24
No	1.00	Referent	1.00	Referent
WHO stage
4	1.34	0.59, 3.01	1.28	0.10, 16.10
3	1.00	Referent	1.00	Referent
ART initiation
Not per strategy	4.25	1.79, 10.07	2.52	0.93, 6.86	2.47	0.93, 6.55
Per strategy	1.00	Referent	1.00	Referent	1.00	Referent

Abbreviations: ART, antiretroviral therapy; ARV, antiretroviral; CI, confidence interval; HIV, human immunodeficiency virus; OR, odds ratio; TB, tuberculosis; WHO, World Health Organization.

^a Weight (kg)/height (m)².

Mortality under observed implementation fidelity to CD4-stratified ART initiation

During the first 6 months of TB treatment, 33 participants died, and 47 (11.9%) were lost to follow-up. Among the 348 (88.1%) patients with an observed outcome, the 6-month mortality risk was 9.5% (95% CI: 6.4, 12.6). The majority (n = 26) of these deaths occurred in participants whose timing of ART initiation deviated from the CD4-stratified strategy.

When estimating the risk in the full cohort (i.e., including those lost to follow-up) using the predicted outcome probabilities from the logistic regression model, we found that the 6-month mortality risk under observed implementation fidelity was 12.0% (95% CI: 8.2, 15.7).

Estimated causal effect of implementation fidelity on 6-month mortality

As shown in Table 3, complete fidelity to the CD4-stratified timing strategy for ART initiation in this population was estimated to result in a 6-month mortality risk of 7.8% (95% CI: 2.4, 12.3), corresponding to a −4.2% risk difference (95% CI: −8.1, −0.3). The preventable fraction of mortality due to nonfidelity to the CD4-stratified ART initiation strategy was 35.1% (95% CI: 2.9, 67.9), suggesting that just over one third of the mortality is preventable by complete implementation fidelity. These mortality estimates were robust to sensitivity analyses in which the definition of ART initiation per strategy was narrowed to exclude patients who were lost to follow-up prior to the time of eligibility (risk difference, −4.1%, 95% CI: −7.8, −0.5), and the preventable fraction was 34.6% (95% CI: 3.8, 65.3).

Table 3.

Six-Month Mortality Risk Under Observed and Complete Implementation Fidelity to Timing of Antiretroviral Therapy Initiation per CD4-Stratified Strategy, Integrating Tuberculosis and Antiretroviral Treatment Study, 2007–2009

	Estimate	95% CI
Risks in ITART Study population (n = 395)
Observed riska	0.095	0.064, 0.126
Estimated (modeled) risk under observed implementation fidelity to the CD4-stratified ART initiation strategyb	0.120	0.082, 0.157
Estimated risk under complete implementation fidelity to the CD4-stratified ART initiation strategyb	0.078	0.030, 0.128
Contrast measures
Estimated risk difference in mortality between observed and complete implementation fidelityb	−0.042	−0.081, −0.003
Estimated preventable fraction of mortality under complete implementation fidelityb	0.351	0.029, 0.679

Abbreviation: ART, antiretroviral therapy; CI, confidence interval; ITART, Integrating Tuberculosis and Antiretroviral Treatment.

^a Excludes 47 patients missing 6-month mortality outcome.

^b Includes estimated/predicted outcomes for those missing 6-month mortality outcome.

DISCUSSION

Despite full integration of TB treatment and ART, only 46% of all HIV-infected TB patients initiated ART per CD4-stratified timing strategy (i.e., within 1–2 months of TB treatment initiation). The remaining 54% of patients either failed to start or experienced a median additional delay of 11 (IQR, 4–24) days. We estimated that, if these failures to initiate and delays could be avoided, the 6-month mortality risk would fall by 4.2%, representing a 35.1% preventable fraction of mortality in this population. These findings suggest that strategies are needed to improve the implementation of the 2012 WHO recommendation that calls for CD4-stratified timing of ART initiation (5).

Under the condition of low (46%) implementation fidelity to the CD4-stratified ART initiation strategy, the 6-month estimated mortality risk in the full cohort was 12.0%. This estimated risk was higher than the observed risk (9.5%), because patients who were lost to follow-up and missing an outcome had a high risk of mortality based on their baseline characteristics. Although not unexpected given the many common risk factors for lost to follow-up and mortality, this demonstrates that a complete-case analysis may underestimate mortality risk. Under the scenario of 100% fidelity to the CD4 count–stratified ART initiation strategy, we found a 4.2% (95% CI: 0.03, 8.1) reduction in mortality risk, representing a 35.1% preventable fraction of mortality. Interventions aimed at factors associated with deviation from CD4-stratified timing are thus needed to improve implementation fidelity (21). However, even under complete implementation fidelity, there were 10 patients whose predicted probability of death was greater than 50%. All 10 had a baseline CD4 count of <50 cells/mm³ and did not tolerate TB treatment, suggesting that interventions other than improving implementation fidelity to CD4-stratified ART initiation may be needed to prevent their death. Because the definition of TB treatment intolerance is inherently nonspecific, patients with severe immunosuppression who are assessed as not tolerating TB treatment may, in fact, have other underlying clinical conditions that contribute to mortality and require attention beyond that available at the primary care level. In fact, many factors likely contribute to mortality in our population; we focused our analysis on the timing of ART initiation.

Few studies have quantified implementation fidelity to international guidelines for resource-limited settings, and we are not aware of any studies attempting to estimate the incremental gain achievable through 100% implementation fidelity. As such, it remains unclear whether any positive outcomes following implementation of new guidelines would be improved further if greater attention were paid to implementation fidelity. It has therefore been suggested that, to discern the true impact of an intervention, research should evaluate implementation fidelity (13).

Our study provides 1 of the first examples of using modern epidemiologic methods to measure the impact of implementation fidelity on desired outcomes. The parametric g-formula generates unbiased estimates of the causal effect of an intervention by using observational data when assumptions of exchangeability (no uncontrolled confounding or uncontrolled selection bias), positivity (patients in each treatment group across all strata of each covariate), and consistency (for a given treatment, the counterfactual outcome is equivalent to the observed outcome for each patient) are met. This allows investigators to extend the analysis of observational study beyond the traditional all versus none comparison and, instead, estimate the population-attributable risk difference and fraction, which contrast counterfactual scenarios of more relevance. We used this approach to study the risk difference between different levels (observed vs. 100%) of implementation fidelity to 1 intervention. This approach could also be used to compare the estimated impact of multiple or complex interventions (18, 20, 22). We therefore believe that the application of the parametric g-formula has great potential to contribute to the field of implementation science (22, 23).

Our study has several limitations that must be considered. First, the nurses making decisions on ART timing were trained and closely monitored as part of the study. Implementation fidelity may be even lower in routine settings, and our estimates of the causal effect of implementation fidelity may therefore be an underestimate. Second, similar to most implementation fidelity studies, our assessment focused exclusively on the adherence component by assessing whether ART was initiated on time. We did not evaluate other components of implementation fidelity, such as intervention complexity, facilitation strategies, quality of delivery, and patient responsiveness, which can influence or moderate the level of adherence to guideline implementation by health-care workers (13). Third, because of limited follow-up data, we focused on reduction in short-term mortality, which is only one of the desired outcomes of TB/HIV care. Other potentially relevant outcomes include long-term mortality, medication adherence, and biologically confirmed treatment success. Finally, although we adjusted for baseline covariates identified as potential confounders, we cannot exclude the possibility of residual confounding.

In conclusion, despite full integration of TB treatment and ART, delayed initiation of ART continues to occur and results in increased mortality risk. Strategies to achieve high implementation fidelity to CD4-stratified timing of ART initiation for patients with TB at primary-care clinics in resource-limited settings are thus needed.

APPENDIX 1

Steps of the Parametric g-Formula to Estimate the Risk Difference Between the Observed and Complete Implementation Fidelity to Timing of ART Initiation per CD4-Stratified Strategy

1. Fit a logistic regression model of the association between initiating ART per strategy and mortality, including confounders derived from a causal directed acyclic graph.

2. Use parameter estimates from the model to estimate predicted probabilities of death for each patient on the basis of their baseline factors and observed ART timing.

3. Estimate the risk of death in the ITART Study population with observed fidelity to ART timing per CD4-stratified strategy, by taking the average of the predicted probabilities of death calculated in step 2 across all patients.

4. Recalculate the predicted probabilities for each patient under the scenario of complete fidelity to ART timing per CD4-stratified strategy.

5. Estimate the risk of death in the ITART Study population with complete fidelity to ART timing per CD4-stratified strategy, by taking the average of the predicted probabilities of death calculated in step 4 across all patients.

6. Estimate the risk difference by subtracting the average risk of death in the ITART Study population with complete fidelity (step 5) from the average risk of death in the same study population with observed fidelity (step 3).

7. Estimate the 95% confidence interval through bootstrap estimation: Create multiple (B) data sets through random selection with replacement from the original ITART Study population, rerun steps 1–6 on each data set to estimate B risk differences, and use the standard deviation of the B risk differences to estimate the standard error of the original point estimate.

APPENDIX 2

Application of the Parametric g-Formula to Estimate the Risk Difference Between Observed and Complete Fidelity to CD4-Stratifed Timing of ART Initiation

We used logistic regression and the parametric g-formula to calculate the predicted probability of death for each patient in the analytical population of 395 patients under 2 scenarios of implementation of the CD4-stratified timing strategy for ART initiation: scenario 1 (observed fidelity) and scenario 2 (complete fidelity).

As an example, we applied this method for patient X, who was 32 years of age, diagnosed with smear-positive pulmonary TB, tolerating TB treatment, and WHO stage 3; whose ART initiation was not per strategy; who had a CD4 count of 45 cells/mm³ and an underweight body mass index of <18.5; and who did not have any contraindication to antiretroviral drugs.

The logistic regression model, which includes a variable corresponding to whether the patient initiated ART per strategy and covariates included in Table 1, generated regression parameters such that

$$\begin{array}{cc}\text{Ln}[P(D=1|X=x)]=\hfill & -2.6590+0.0305(\text{age})+-0.6731(\text{female})\hfill \\ & +-0.9064(\text{per\_strategy})+-2.5365(\text{tb\_tx\_tolerated})\hfill \\ & +2.0222(\text{cd}4\mathrm{\_}\mathrm{less}\mathrm{\_}50)+0.6315(\text{cd}4\mathrm{\_}50\mathrm{\_}99)\hfill \\ \hfill & +0.7738(\text{cd}4\mathrm{\_}200\mathrm{\_}350)+0.2346(\text{contraindication})\hfill \\ & +0.3727(\text{smear\_neg})+0.0296(\text{extrapulmonary})\hfill \\ & +0.3744(\text{who\_4})+0.8832(\text{underweight}).\hfill \end{array}$$

Scenario 1: observed fidelity to CD4-stratified strategy for timing of ART initiation

As shown in the calculation below, for patient X, the (factual) predicted probability of death under his observed fidelity to ART timing was 21%:

$$\begin{array}{cc}\text{Ln}[P(D=1|X=x)]=\hfill & -2.6590+0.0305(32)+-0.6731(0)+-0.9064(0)\hfill \\ & +-2.5365(1)+2.0222(1)+0.6315(0)+0.7738(0)\hfill \\ & +0.2346(0)+0.3727(0)+0.0296(0)+0.3744(0)+0.8832(1)\hfill \end{array}$$

$$\text{Ln}[P(D=1|X=x)]=-1.3141$$

$$\begin{array}{cc}P(D=1|X=x)\hfill & =[\exp (-1.3141)]/[1+\text{exp}(-1.3141)]=\mathrm{0.212.}\hfill \end{array}$$

Scenario 2: complete fidelity to CD4-stratified timing of ART initiation

In order to estimate the (counterfactual) predicted probability of death for patient X under complete fidelity to ART timing, we used the same regression parameters and baseline characteristics; however, we enter a “1” instead of a “0” for the coefficient corresponding to the regression parameter for ART per strategy. As shown in the calculation below, for patient X, the predicted probability of death under complete fidelity was 9.5%:

$$\begin{array}{cc}\text{Ln}[P(D=1|X=x)]=\hfill & -2.6590+0.0305(32)+-0.6731(0)+-0.9064(1)\hfill \\ & +-2.5365(1)+2.0222(1)+0.6315(0)+0.7738(0)+0.2346(0)\hfill \\ & +0.3727(0)+0.0296(0)+0.3744(0)+0.8832(1)\hfill \end{array}$$

$$\text{Ln}[P(D=1|X=x)]=-2.2205$$

$$P(D=1|X=x)=[\exp (-2.251)]/[1+\exp (-2.251)]=\mathrm{0.098.}$$

From this example, it is clear that, for patients (unlike this one) whose observed ART initiation was per strategy, the predicted probability under complete fidelity (scenario 2) would be the same as the predicted probability under observed fidelity (scenario 1).

To estimate the population risk under observed and complete fidelity, we took the average of the individual patient probabilities of death under scenario 1 and the average of the individual population probabilities of death under scenario 2. Then, we subtracted the average population risk under scenario 2 from the average population risk under scenario 1 to calculate the risk difference, as a measure of effect of complete implementation fidelity compared with observed implementation fidelity. The 95% confidence interval was bootstrapped by sampling with replacement from the original analytical population to generate 500 data sets (n = 395). In each data set, we estimated the risk difference and then used the standard deviation across these 500 estimates to estimate the 95% confidence interval around the risk difference from the original analytical population.