Abstract
Among patients with tuberculosis and human immunodeficiency virus type 1, CD4-stratified initiation of antiretroviral therapy (ART) is recommended, with earlier ART in those with low CD4 counts. However, the impact of implementation fidelity to this recommendation is unknown. We examined a prospective cohort study of 395 adult patients diagnosed with tuberculosis and human immunodeficiency virus between August 2007 and November 2009 in Kinshasa, Democratic Republic of the Congo. ART was to be initiated after 1 month of tuberculosis treatment at a CD4 count of <100 cells/mm3 or World Health Organization stage 4 (other than extrapulmonary tuberculosis) and after 2 months of tuberculosis treatment at a CD4 count of 100–350 cells/mm3. We used the parametric g-formula to estimate the impact of implementation fidelity on 6-month mortality. Observed implementation fidelity was low (46%); 54% of patients either experienced delays in ART initiation or did not initiate ART, which could be avoided under perfect implementation fidelity. The observed mortality risk was 12.0% (95% confidence interval (CI): 8.2, 15.7); under complete (counterfactual) implementation fidelity, the mortality risk was 7.8% (95% CI: 2.4, 12.3), corresponding to a risk reduction of 4.2% (95% CI: 0.3, 8.1) and a preventable fraction of 35.1% (95% CI: 2.9, 67.9). Strategies to achieve high implementation fidelity to CD4-stratified ART timing are needed to maximize survival benefit.
Keywords: antiretroviral therapy; causal inference, HIV, human immunodeficiency virus, implementation fidelity, parametric g-formula, tuberculosis
In 2012, an estimated 1.1 million (14%) of the 8.6 million patients diagnosed with active tuberculosis (TB) disease worldwide were also infected with human immunodeficiency virus (HIV) (1). Although, under current World Health Organization (WHO) guidelines, all people diagnosed with TB and HIV are eligible for antiretroviral therapy (ART), only 57% initiated ART during TB treatment in 2012 (1). Findings from randomized controlled trials demonstrated that early ART, defined as within 2–4 weeks of start of TB treatment, reduces mortality in patients with a CD4 count of <50 cells/mm3. In these trials, early ART initiation did not result in statistically significant lower mortality in patients with CD4 counts of >50 cells/mm3 (2–4). On the basis of these findings, the WHO recommends ART initiation within 2 weeks of TB treatment for patients with a CD4 count of <50 cells/mm3 and within 8 weeks for all people diagnosed with HIV and TB (5).
Findings from randomized controlled trials are not always replicable in less strictly controlled settings. Observational studies from sub-Saharan Africa have found that most patients initiate ART late, after 8 weeks of TB treatment (6–10). Lack of integration of TB and HIV treatment services has been identified as one of the key contributors to this delay (9, 10). Interventions to integrate TB and HIV services have been shown to reduce, but not eliminate, delay in ART initiation (11, 12).
Implementation fidelity, defined as the degree to which an intervention is implemented as intended, is a potential modifier of the relationship between an intervention and its intended outcome and is important to translation of evidence-based recommendations into clinical practice (13, 14). Achieving high implementation fidelity can help to replicate the success an intervention has achieved in randomized controlled trials (13). The relatively low coverage of timely ART initiation in patients with TB suggests that implementation fidelity to the 2012 WHO guidelines for timing of ART initiation in TB patients may be a challenge in routine clinical settings, even with integrated HIV/TB treatment. Data on implementation fidelity in resource-limited settings and its impact on desired outcomes are limited.
We aimed to quantify the impact of implementation fidelity to CD4-stratified timing of ART initiation for TB patients on mortality in a prospective cohort of patients receiving integrated TB/HIV treatment at the primary care level in Kinshasa, Democratic Republic of the Congo.
METHODS
Setting, study population, and study procedures
The Integrating Tuberculosis and Antiretroviral Treatment (ITART) Study was conducted at 5 primary health clinics in the Democratic Republic of the Congo and enrolled patients between August 2007 and November 2009 (15). All patients diagnosed with HIV and TB per national Democratic Republic of the Congo guidelines were offered study participation; 3 refused. At baseline, relevant demographic and clinical data were collected, and CD4 count testing was performed.
Study nurses (1 per clinic) were trained to implement a CD4-stratified timing strategy for ART initiation in accord with the 2006 WHO guidelines. Patients were eligible to initiate ART at completion of the first month of TB treatment if the baseline CD4 count was below 100 cells/mm3 or WHO stage 4 condition other than extrapulmonary TB, as well as at 2 months of TB treatment if the baseline CD4 count was between 100 and 350 cells/mm3. Patients with a baseline CD4 count of >350 cells/mm3 were scheduled for repeat CD4 testing at month 5 of TB treatment and were eligible to initiate ART at the end of TB treatment if the CD4 count had dropped to ≤350 cells/mm3 or when an incident WHO stage 4 condition occurred. When CD4 counts were not available (because of a shortage of reagents), all patients were considered eligible for ART initiation at 1 month. According to the legal framework of the Democratic Republic of the Congo, nurses’ decisions to initiate ART needed approval by a consulting physician. Providers assessed whether the patient was or was not tolerating TB treatment at the time of ART eligibility on the basis of clinical judgment as the 2006 WHO guidelines did not define what constitutes tolerance (or intolerance) of TB treatment.
Patients were seen by study nurses weekly during the 2-month intensive phase, monthly during the subsequent 4-month continuation phase, and at completion of TB treatment. Patients were considered lost to follow-up if they were more than 3 days late for a scheduled clinic visit and could not be located by phone or home visit.
The ITART Study was approved by institutional review boards at both the University of North Carolina at Chapel Hill and the University of Kinshasa. All patients provided written informed consent for participation in the study.
Study population and definitions
This analysis was limited to individuals aged ≥13 years who were enrolled in the ITART Study within 1 month of TB treatment start, were treated with ART, were naïve, and had a CD4 count of ≤350 cells/mm3 within 30 days of the start of TB treatment (at baseline).
Outcome
Mortality was defined as all-cause mortality during the first 6 months of TB treatment, the duration of first-line TB treatment. Using the standard approach for observational cohort studies, we calculated the 6-month risk of mortality on the basis of the observed outcomes. Patients who were lost to follow-up prior to 6 months were assigned a missing outcome and did not contribute to the numerator or denominator of the observed risk.
Risk factors for mortality
We used a logistic regression model to assess baseline covariates, including the timing of ART initiation, as potential predictors of mortality. We first ran a full logistic model containing all the by selected covariates. Subsequently, we used a backwards elimination stepwise method to generate a final (reduced) predictive model. Covariates were assessed in order from the highest to the lowest Wald χ2 and eliminated from the model by using the likelihood ratio test with an α = 0.10. We estimated crude and adjusted odds ratios with 95% confidence intervals.
Assessment of implementation fidelity
“Implementation fidelity” is defined as the degree to which programs are implemented as intended, with a focus on content or frequency of the intervention (13). In this study, we confined implementation fidelity to the CD4-stratified ART initiation strategy as the proportion of individuals who initiated timely ART, that is, according to a priori–defined CD4 criteria. We calculated each patient's timing of ART initiation by comparing the ART start date with the TB treatment start date. We categorized the timing of ART initiation as per CD4-stratified strategy (“per strategy”) or deviating from CD4-stratified strategy (“not per strategy”). To accommodate a combination of scheduling limitations, clinic closures on weekends and holidays, and limited availability of a consulting physician as required for ART initiation per the Democratic Republic of the Congo legal framework, a 5-day grace period was added to the 1- or 2-month TB treatment to define ART initiation per strategy. Participants who initiated ART prior to the time they became eligible plus 5 days were categorized as per strategy. Participants who died or were lost to follow-up prior to eligibility for ART and had not initiated ART were categorized as initiating ART per strategy, since not initiating ART prior to death or loss to follow-up did not constitute deviation from the CD4-stratified strategy. In sensitivity analyses, we explored the impact of narrowing the definition of ART initiation per strategy to exclude patients who were lost to follow-up prior to the time of ART eligibility, a subset of patients who could have started timely ART had they been retained in care.
Differences in the proportions and medians of baseline characteristics between patients initiating ART per strategy and those initiating not per strategy were assessed by using χ2 or Fisher's exact tests and Kruskal-Wallis tests, respectively.
Estimation of the causal effect of implementation fidelity on mortality
To estimate the causal effect of implementation fidelity, we compared mortality in the study population under observed intervention fidelity with mortality in the study population with complete implementation fidelity (Figure 1) (16, 17). Standard multivariable regression would not easily allow us to estimate the difference in risk in mortality at the population level attributable to implementation fidelity. We overcame this by using the parametric g-formula to estimate mortality in the cohort under the counterfactual scenario of complete implementation fidelity (18–20). A step-by-step overview of this methodological approach is presented in Appendix 1, and the worked example is presented as Appendix 2 (18).
We built a logistic regression model to assess the association between initiating ART per strategy and mortality (step 1), including baseline covariates identified as potential confounders using a directed acyclic graph. We then used parameter estimates from the model to calculate the predicted probability of death for each patient based on their baseline covariates and observed ART timing (step 2). This modeling method imputes an outcome for each patient on the basis of the average risk across patients with observed outcomes with the same baseline characteristics. Consequently, the outcome of participants who were lost to follow-up is no longer missing, as these participants are assigned an outcome on the basis of their baseline characteristics. By averaging these predicted probabilities of death across all participants, we estimated the risk of mortality in the full cohort under the observed, real-life level of implementation fidelity (step 3).
To estimate the causal effect of implementation fidelity, we estimated a (counterfactual) probability of death for each participant, corresponding to what would have happened to each participant had he or she initiated ART per strategy. For participants who did initiate ART per strategy, this predicted probability of death is the same as that calculated in step 2; for participants who did not initiate ART per strategy, we estimated this probability based on the outcomes of patients with similar baseline characteristics who did initiate ART per strategy (step 4). By averaging these predicted probabilities, we estimated the risk of mortality in the full cohort under a scenario of complete (100%) implementation fidelity (step 5).
We then calculated the risk difference by subtracting this mortality risk estimate in the cohort with complete fidelity from the mortality risk estimate in the cohort with observed fidelity (step 6). Bootstrapping was used to generate the 95% confidence interval around the risk difference. This was done by creating multiple (n = 500) data sets through random selection of 395 individuals with replacement from the original ITART Study population, followed by rerunning step 1 through step 6, and using the standard error across all the risk difference estimates (step 7).
We estimated the preventable fraction similarly, by dividing the risk difference by the estimated risk in the cohort under observed, real-life implementation fidelity and bootstrapping the corresponding 95% confidence interval. This measure is interpreted as the fraction of mortality that could be prevented if 100% implementation fidelity were achieved.
RESULTS
Baseline characteristics of the analytical cohort
Between August 2007 and November 2009, 599 participants enrolled in the ITART Study. Among them, 204 were sequentially excluded on the basis of age <13 years (n = 18), lack of baseline CD4 count (n = 88), enrollment more than 1 month after TB treatment initiation (n = 1), exposure to ART prior to enrollment (n = 0), and CD4 count >350 (n = 97). The remaining 395 participants were included in the analysis.
Baseline characteristics of the patients are presented in Table 1. The type of TB was smear-positive pulmonary in 35%, smear-negative pulmonary in 45%, and extrapulmonary in 20% of patients. Just over half (59%) were female, and the median age was 38 (interquartile range (IQR), 32–45) years. Most patients were underweight (median body mass index (weight (kg)/height (m)2), 17.8; IQR, 16.5–19.7); 59.2% had a body mass index of <18.5. Patients presented late in the HIV disease process, with a median CD4 count of 131 cells/mm3 (IQR, 63–224). Of the 143 (36%) patients eligible for ART at 1 month, 77 (54%) had a CD4 count of <50 cells/mm3, 64 (45%) had a CD4 count of 50–99 cells/mm3, and 2 (1%) had a CD4 count of >100 cells/mm3 and WHO stage 4. Of the 252 (64%) patients eligible for ART at 2 months, 136 (54%) had a CD4 count of 100–199 cells/mm3 and 116 (46%) had a CD4 count of 200–350 cells/mm3. A few patients (n = 24) (6%) had a contraindication to 1 or more antiretroviral drugs.
Table 1.
Baseline Characteristic | All Patients (n = 395) |
Timing of Antiretroviral Therapy Initiation |
P Value | ||||
---|---|---|---|---|---|---|---|
Per Strategy (n = 183) |
Not Per Strategy (n = 212) |
||||||
No. | % | No. | % | No. | % | ||
Sex | |||||||
Female | 231 | 58.5 | 103 | 56.3 | 128 | 60.4 | 0.41 |
Male | 164 | 41.5 | 80 | 43.7 | 84 | 39.6 | |
Age, yearsa | 38 (32–45) | 38 (32–45) | 38 (32–45) | 0.97 | |||
Age, years | |||||||
<30 | 65 | 16.5 | 32 | 17.5 | 33 | 15.6 | 0.61 |
30–39 | 158 | 40.0 | 73 | 39.9 | 85 | 40.1 | 0.97 |
40–49 | 129 | 32.7 | 56 | 30.6 | 73 | 34.4 | 0.42 |
≥50 | 43 | 10.9 | 22 | 12.0 | 21 | 9.9 | 0.50 |
Tuberculosis type | |||||||
Smear positive, pulmonary | 139 | 35.2 | 71 | 38.8 | 68 | 32.1 | 0.16 |
Smear negative, pulmonary | 176 | 44.6 | 77 | 42.1 | 99 | 46.7 | 0.36 |
Extrapulmonary | 80 | 20.3 | 35 | 19.1 | 45 | 21.2 | 0.60 |
CD4 count, cells/mm3a | 131 (63–224) | 151 (77–243) | 113 (60–190) | 0.002 | |||
CD4 count, cells/mm3 | |||||||
<50 | 77 | 19.5 | 33 | 18.0 | 44 | 20.8 | 0.50 |
50–99 | 64 | 16.2 | 21 | 11.5 | 43 | 20.3 | 0.02 |
100–199 | 136 | 34.4 | 61 | 33.3 | 75 | 35.4 | 0.67 |
200–350 | 118 | 29.9 | 68 | 37.2 | 50 | 23.6 | 0.003 |
Body mass indexa,b | 17.8 (16.5–19.7) | 17.7 (16.4–19.6) | 17.8 (16.6–19.8) | 0.35 | |||
Body mass indexb | |||||||
Underweight (<18.5) | 234 | 59.2 | 114 | 62.3 | 120 | 56.6 | 0.25 |
Normal (18.5–24.9) | 148 | 37.5 | 64 | 35.0 | 84 | 39.6 | 0.34 |
Overweight (25.0–29.9) | 11 | 2.8 | 4 | 2.2 | 7 | 3.3 | 0.56 |
Obese (≥30.0) | 2 | 0.5 | 1 | 0.6 | 1 | 0.5 | 1.0 |
Toleration of tuberculosis drugs | 322 | 81.5 | 175 | 95.6 | 147 | 69.3 | <0.001 |
Contraindication to any ARV drug | 24 | 6.1 | 7 | 3.8 | 17 | 8.0 | 0.09 |
WHO clinical stage 4 | 90 | 22.8 | 41 | 22.4 | 49 | 23.1 | 0.87 |
Abbreviations: ARV, antiretroviral; WHO, World Health Organization.
a Median (interquartile range).
b Weight (kg)/height (m)2.
Implementation fidelity to CD4-stratified timing of ART initiation
Overall, 183 (46%) participants initiated ART per strategy. Among the 212 (54%) participants who initiated ART not per strategy, 53 (25%) never initiated ART, and 159 (75%) initiated ART with a median delay of 11 days (IQR, 4–24) beyond the time of eligibility plus 5 days. The median delay did not differ by eligibility category (12 days for those eligible at 1 month vs. 10 days for those at 2 months) (P = 0.62). Patients whose timing of ART initiation was per strategy had a higher CD4 count (151 cells/mm3 vs. 113 cells/mm3) (P = 0.002) and a higher frequency of tolerating their TB drugs (96% vs. 70%) (P < 0.0001) at the scheduled time of ART initiation than did patients whose timing of ART initiation was not per strategy.
Predictors of mortality in the first 6 months of TB treatment
Results of predictive modeling of mortality are presented in Table 2. In a series of crude (unadjusted) models, a CD4 count of <50 cells/mm3 (crude odds ratio (OR) = 6.0, 95% confidence interval (CI): 2.2, 16.4) and TB treatment intolerance (crude OR = 12.3, 95% CI: 5.6, 27.3) were predictive of mortality. In the final (reduced) model, TB treatment intolerance (adjusted OR = 12.7, 95% CI: 4.8, 33.2), a CD4 count of <50 cells/mm3 (adjusted OR = 7.3, 95% CI: 2.3, 23.3), and male sex (adjusted OR = 2.4, 95% CI: 1.0, 5.6) were predictive of mortality. In addition, underweight (adjusted OR = 2.2, 95% CI: 0.9, 5.6) and not initiating ART per strategy (adjusted OR = 2.5, 95% CI: 0.9, 6.6) doubled the risk of mortality, although not statistically significantly.
Table 2.
Predictor | Crude |
Adjusted |
||||
---|---|---|---|---|---|---|
Full Model |
Reduced Model |
|||||
OR | 95% CI | OR | 95% CI | OR | 95% CI | |
Sex | ||||||
Male | 1.78 | 0.86, 3.65 | 1.97 | 0.76, 5.09 | 2.37 | 1.00, 5.61 |
Female | 1.00 | Referent | 1.00 | Referent | 1.00 | Referent |
Age, years | ||||||
<30 | 1.36 | 0.44, 4.17 | 1.26 | 0.32, 4.96 | ||
30–39 | 1.00 | Referent | 1.00 | Referent | ||
40–49 | 1.72 | 0.71, 4.14 | 1.86 | 0.64, 5.37 | ||
≥50 | 2.40 | 0.82, 7.07 | 2.96 | 0.77, 11.40 | ||
Type of tuberculosis | ||||||
Smear positive, pulmonary | 1.00 | Referent | 1.00 | Referent | ||
Smear negative, pulmonary | 2.01 | 0.81, 4.98 | 1.43 | 0.48, 4.24 | ||
Extrapulmonary | 2.05 | 0.71, 5.91 | 1.11 | 0.07, 17.79 | ||
CD4 count, cells/mm3 | ||||||
<50 | 5.99 | 2.19, 16.36 | 7.66 | 2.27, 25.80 | 7.30 | 2.29, 23.30 |
50–99 | 1.81 | 0.53, 6.19 | 1.92 | 0.47, 7.86 | 1.94 | 0.49, 7.59 |
100–199 | 1.00 | Referent | 1.00 | Referent | 1.00 | Referent |
200–350 | 1.36 | 0.44, 4.17 | 2.19 | 0.58, 8.20 | 1.95 | 0.56, 6.79 |
Body mass indexa | ||||||
Underweight (<18.5) | 1.92 | 0.87, 4.27 | 2.46 | 0.94, 6.20 | 2.21 | 0.89, 5.62 |
Not underweight (≥18.5) | 1.00 | Referent | 1.00 | Referent | 1.00 | Referent |
TB treatment intolerance | ||||||
Yes | 12.33 | 5.56, 27.31 | 12.75 | 4.69, 34.71 | 12.65 | 4.82, 33.20 |
No | 1.00 | Referent | 1.00 | Referent | 1.00 | Referent |
Contraindication to any ARV | ||||||
Yes | 0.90 | 0.20, 4.04 | 1.43 | 0.48, 4.24 | ||
No | 1.00 | Referent | 1.00 | Referent | ||
WHO stage | ||||||
4 | 1.34 | 0.59, 3.01 | 1.28 | 0.10, 16.10 | ||
3 | 1.00 | Referent | 1.00 | Referent | ||
ART initiation | ||||||
Not per strategy | 4.25 | 1.79, 10.07 | 2.52 | 0.93, 6.86 | 2.47 | 0.93, 6.55 |
Per strategy | 1.00 | Referent | 1.00 | Referent | 1.00 | Referent |
Abbreviations: ART, antiretroviral therapy; ARV, antiretroviral; CI, confidence interval; HIV, human immunodeficiency virus; OR, odds ratio; TB, tuberculosis; WHO, World Health Organization.
a Weight (kg)/height (m)2.
Mortality under observed implementation fidelity to CD4-stratified ART initiation
During the first 6 months of TB treatment, 33 participants died, and 47 (11.9%) were lost to follow-up. Among the 348 (88.1%) patients with an observed outcome, the 6-month mortality risk was 9.5% (95% CI: 6.4, 12.6). The majority (n = 26) of these deaths occurred in participants whose timing of ART initiation deviated from the CD4-stratified strategy.
When estimating the risk in the full cohort (i.e., including those lost to follow-up) using the predicted outcome probabilities from the logistic regression model, we found that the 6-month mortality risk under observed implementation fidelity was 12.0% (95% CI: 8.2, 15.7).
Estimated causal effect of implementation fidelity on 6-month mortality
As shown in Table 3, complete fidelity to the CD4-stratified timing strategy for ART initiation in this population was estimated to result in a 6-month mortality risk of 7.8% (95% CI: 2.4, 12.3), corresponding to a −4.2% risk difference (95% CI: −8.1, −0.3). The preventable fraction of mortality due to nonfidelity to the CD4-stratified ART initiation strategy was 35.1% (95% CI: 2.9, 67.9), suggesting that just over one third of the mortality is preventable by complete implementation fidelity. These mortality estimates were robust to sensitivity analyses in which the definition of ART initiation per strategy was narrowed to exclude patients who were lost to follow-up prior to the time of eligibility (risk difference, −4.1%, 95% CI: −7.8, −0.5), and the preventable fraction was 34.6% (95% CI: 3.8, 65.3).
Table 3.
Estimate | 95% CI | |
---|---|---|
Risks in ITART Study population (n = 395) | ||
Observed riska | 0.095 | 0.064, 0.126 |
Estimated (modeled) risk under observed implementation fidelity to the CD4-stratified ART initiation strategyb | 0.120 | 0.082, 0.157 |
Estimated risk under complete implementation fidelity to the CD4-stratified ART initiation strategyb | 0.078 | 0.030, 0.128 |
Contrast measures | ||
Estimated risk difference in mortality between observed and complete implementation fidelityb | −0.042 | −0.081, −0.003 |
Estimated preventable fraction of mortality under complete implementation fidelityb | 0.351 | 0.029, 0.679 |
Abbreviation: ART, antiretroviral therapy; CI, confidence interval; ITART, Integrating Tuberculosis and Antiretroviral Treatment.
a Excludes 47 patients missing 6-month mortality outcome.
b Includes estimated/predicted outcomes for those missing 6-month mortality outcome.
DISCUSSION
Despite full integration of TB treatment and ART, only 46% of all HIV-infected TB patients initiated ART per CD4-stratified timing strategy (i.e., within 1–2 months of TB treatment initiation). The remaining 54% of patients either failed to start or experienced a median additional delay of 11 (IQR, 4–24) days. We estimated that, if these failures to initiate and delays could be avoided, the 6-month mortality risk would fall by 4.2%, representing a 35.1% preventable fraction of mortality in this population. These findings suggest that strategies are needed to improve the implementation of the 2012 WHO recommendation that calls for CD4-stratified timing of ART initiation (5).
Under the condition of low (46%) implementation fidelity to the CD4-stratified ART initiation strategy, the 6-month estimated mortality risk in the full cohort was 12.0%. This estimated risk was higher than the observed risk (9.5%), because patients who were lost to follow-up and missing an outcome had a high risk of mortality based on their baseline characteristics. Although not unexpected given the many common risk factors for lost to follow-up and mortality, this demonstrates that a complete-case analysis may underestimate mortality risk. Under the scenario of 100% fidelity to the CD4 count–stratified ART initiation strategy, we found a 4.2% (95% CI: 0.03, 8.1) reduction in mortality risk, representing a 35.1% preventable fraction of mortality. Interventions aimed at factors associated with deviation from CD4-stratified timing are thus needed to improve implementation fidelity (21). However, even under complete implementation fidelity, there were 10 patients whose predicted probability of death was greater than 50%. All 10 had a baseline CD4 count of <50 cells/mm3 and did not tolerate TB treatment, suggesting that interventions other than improving implementation fidelity to CD4-stratified ART initiation may be needed to prevent their death. Because the definition of TB treatment intolerance is inherently nonspecific, patients with severe immunosuppression who are assessed as not tolerating TB treatment may, in fact, have other underlying clinical conditions that contribute to mortality and require attention beyond that available at the primary care level. In fact, many factors likely contribute to mortality in our population; we focused our analysis on the timing of ART initiation.
Few studies have quantified implementation fidelity to international guidelines for resource-limited settings, and we are not aware of any studies attempting to estimate the incremental gain achievable through 100% implementation fidelity. As such, it remains unclear whether any positive outcomes following implementation of new guidelines would be improved further if greater attention were paid to implementation fidelity. It has therefore been suggested that, to discern the true impact of an intervention, research should evaluate implementation fidelity (13).
Our study provides 1 of the first examples of using modern epidemiologic methods to measure the impact of implementation fidelity on desired outcomes. The parametric g-formula generates unbiased estimates of the causal effect of an intervention by using observational data when assumptions of exchangeability (no uncontrolled confounding or uncontrolled selection bias), positivity (patients in each treatment group across all strata of each covariate), and consistency (for a given treatment, the counterfactual outcome is equivalent to the observed outcome for each patient) are met. This allows investigators to extend the analysis of observational study beyond the traditional all versus none comparison and, instead, estimate the population-attributable risk difference and fraction, which contrast counterfactual scenarios of more relevance. We used this approach to study the risk difference between different levels (observed vs. 100%) of implementation fidelity to 1 intervention. This approach could also be used to compare the estimated impact of multiple or complex interventions (18, 20, 22). We therefore believe that the application of the parametric g-formula has great potential to contribute to the field of implementation science (22, 23).
Our study has several limitations that must be considered. First, the nurses making decisions on ART timing were trained and closely monitored as part of the study. Implementation fidelity may be even lower in routine settings, and our estimates of the causal effect of implementation fidelity may therefore be an underestimate. Second, similar to most implementation fidelity studies, our assessment focused exclusively on the adherence component by assessing whether ART was initiated on time. We did not evaluate other components of implementation fidelity, such as intervention complexity, facilitation strategies, quality of delivery, and patient responsiveness, which can influence or moderate the level of adherence to guideline implementation by health-care workers (13). Third, because of limited follow-up data, we focused on reduction in short-term mortality, which is only one of the desired outcomes of TB/HIV care. Other potentially relevant outcomes include long-term mortality, medication adherence, and biologically confirmed treatment success. Finally, although we adjusted for baseline covariates identified as potential confounders, we cannot exclude the possibility of residual confounding.
In conclusion, despite full integration of TB treatment and ART, delayed initiation of ART continues to occur and results in increased mortality risk. Strategies to achieve high implementation fidelity to CD4-stratified timing of ART initiation for patients with TB at primary-care clinics in resource-limited settings are thus needed.
ACKNOWLEDGMENTS
Author affiliations: Department of Epidemiology (Monita R. Patel, Daniel Westreich, Marcel Yotebieng, Frieda Behets, Annelies Van Rie), Center for AIDS Research (Joseph J. Eron), and Division of Infectious Diseases (Joseph J. Eron), University of North Carolina at Chapel Hill, Chapel Hill, North Carolina; School of Public Health, University of Kinshasa, Kinshasa, Democratic Republic of the Congo (Marcel Yotebieng, Mbonze Nana); and Division of Epidemiology, College of Public Health, The Ohio State University, Columbus, Ohio (Marcel Yotebieng).
This work was supported by grants from the Centers for Disease Control and Prevention (U62/CCU422422), the President's Emergency Plan for AIDS Relief (5U2GPS001179-01), and the National Institute for Allergy and Infectious Diseases, US National Institutes of Health (2T32AI070114 to M.R.P.). D.W. was supported in part by grant DP2-HD-08-4070.
We acknowledge the HIV and TB Programs of the Ministry of Health of the Democratic Republic of the Congo and Martine Tabala, Koen Vanden Driessche, Steven Mpuate, and Liz Cromwell for their contributions to the success of this study.
D.W. engages in ad hoc consulting on epidemiologic methods for the Eunice Kennedy Shriver National Institute of Child Health and Human Development, US National Institutes of Health.
APPENDIX 1
Steps of the Parametric g-Formula to Estimate the Risk Difference Between the Observed and Complete Implementation Fidelity to Timing of ART Initiation per CD4-Stratified Strategy
Fit a logistic regression model of the association between initiating ART per strategy and mortality, including confounders derived from a causal directed acyclic graph.
Use parameter estimates from the model to estimate predicted probabilities of death for each patient on the basis of their baseline factors and observed ART timing.
Estimate the risk of death in the ITART Study population with observed fidelity to ART timing per CD4-stratified strategy, by taking the average of the predicted probabilities of death calculated in step 2 across all patients.
Recalculate the predicted probabilities for each patient under the scenario of complete fidelity to ART timing per CD4-stratified strategy.
Estimate the risk of death in the ITART Study population with complete fidelity to ART timing per CD4-stratified strategy, by taking the average of the predicted probabilities of death calculated in step 4 across all patients.
Estimate the risk difference by subtracting the average risk of death in the ITART Study population with complete fidelity (step 5) from the average risk of death in the same study population with observed fidelity (step 3).
Estimate the 95% confidence interval through bootstrap estimation: Create multiple (B) data sets through random selection with replacement from the original ITART Study population, rerun steps 1–6 on each data set to estimate B risk differences, and use the standard deviation of the B risk differences to estimate the standard error of the original point estimate.
APPENDIX 2
Application of the Parametric g-Formula to Estimate the Risk Difference Between Observed and Complete Fidelity to CD4-Stratifed Timing of ART Initiation
We used logistic regression and the parametric g-formula to calculate the predicted probability of death for each patient in the analytical population of 395 patients under 2 scenarios of implementation of the CD4-stratified timing strategy for ART initiation: scenario 1 (observed fidelity) and scenario 2 (complete fidelity).
As an example, we applied this method for patient X, who was 32 years of age, diagnosed with smear-positive pulmonary TB, tolerating TB treatment, and WHO stage 3; whose ART initiation was not per strategy; who had a CD4 count of 45 cells/mm3 and an underweight body mass index of <18.5; and who did not have any contraindication to antiretroviral drugs.
The logistic regression model, which includes a variable corresponding to whether the patient initiated ART per strategy and covariates included in Table 1, generated regression parameters such that
Scenario 1: observed fidelity to CD4-stratified strategy for timing of ART initiation
As shown in the calculation below, for patient X, the (factual) predicted probability of death under his observed fidelity to ART timing was 21%:
Scenario 2: complete fidelity to CD4-stratified timing of ART initiation
In order to estimate the (counterfactual) predicted probability of death for patient X under complete fidelity to ART timing, we used the same regression parameters and baseline characteristics; however, we enter a “1” instead of a “0” for the coefficient corresponding to the regression parameter for ART per strategy. As shown in the calculation below, for patient X, the predicted probability of death under complete fidelity was 9.5%:
From this example, it is clear that, for patients (unlike this one) whose observed ART initiation was per strategy, the predicted probability under complete fidelity (scenario 2) would be the same as the predicted probability under observed fidelity (scenario 1).
To estimate the population risk under observed and complete fidelity, we took the average of the individual patient probabilities of death under scenario 1 and the average of the individual population probabilities of death under scenario 2. Then, we subtracted the average population risk under scenario 2 from the average population risk under scenario 1 to calculate the risk difference, as a measure of effect of complete implementation fidelity compared with observed implementation fidelity. The 95% confidence interval was bootstrapped by sampling with replacement from the original analytical population to generate 500 data sets (n = 395). In each data set, we estimated the risk difference and then used the standard deviation across these 500 estimates to estimate the 95% confidence interval around the risk difference from the original analytical population.
REFERENCES
- 1.World Health Organization. Global Tuberculosis Report 2013. Geneva, Switzerland: World Health Organization; 2013. [Google Scholar]
- 2.Blanc FX, Sok T, Laureillard D, et al. Earlier versus later start of antiretroviral therapy in HIV-infected adults with tuberculosis. N Engl J Med. 2011;36516:1471–1481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Havlir DV, Kendall MA, Ive P, et al. Timing of antiretroviral therapy for HIV-1 infection and tuberculosis. N Engl J Med. 2011;36516:1482–1491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Abdool Karim SS, Naidoo K, Grobler A, et al. Integration of antiretroviral therapy with tuberculosis treatment. N Engl J Med. 2011;36516:1492–1501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.World Health Organization. WHO Policy on Collaborative TB/HIV Activities: Guidelines for National Programmes and Other Stakeholders. Geneva, Switzerland: World Health Organization; 2012. [PubMed] [Google Scholar]
- 6.Manosuthi W, Chottanapand S, Thongyen S, et al. Survival rate and risk factors of mortality among HIV/tuberculosis-coinfected patients with and without antiretroviral therapy. J Acquir Immune Defic Syndr. 2006;431:42–46. [DOI] [PubMed] [Google Scholar]
- 7.Varma JK, Nateniyom S, Akksilp S, et al. HIV care and treatment factors associated with improved survival during TB treatment in Thailand: an observational study. BMC Infect Dis. 2009;9:42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Henegar C, Behets F, Vanden Driessche K, et al. Mortality among tuberculosis patients in the Democratic Republic of Congo. Int J Tuberc Lung Dis. 2012;169:1199–1204. [DOI] [PubMed] [Google Scholar]
- 9.Lawn SD, Campbell L, Kaplan R, et al. Time to initiation of antiretroviral therapy among patients with HIV-associated tuberculosis in Cape Town, South Africa. J Acquir Immune Defic Syndr. 2011;572:136–140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lawn SD, Campbell L, Kaplan R, et al. Delays in starting antiretroviral therapy in patients with HIV-associated tuberculosis accessing non-integrated clinical services in a South African township. BMC Infect Dis. 2011;11:258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kerschberger B, Hilderbrand K, Boulle AM, et al. The effect of complete integration of HIV and TB services on time to initiation of antiretroviral therapy: a before-after study. PLoS One. 2012;710:e46988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Huerga H, Spillane H, Guerrero W, et al. Impact of introducing human immunodeficiency virus testing, treatment and care in a tuberculosis clinic in rural Kenya. Int J Tuberc Lung Dis. 2010;145:611–615. [PubMed] [Google Scholar]
- 13.Carroll C, Patterson M, Wood S, et al. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Breitenstein SM, Gross D, Garvey CA, et al. Implementation fidelity in community-based interventions. Res Nurs Health. 2010;332:164–173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Van Rie A, Patel MR, Nana M, et al. Integration and task shifting for TB/HIV care and treatment in highly resource-scarce settings: one size may not fit all. J Acquir Immune Defic Syndr. 2014;653:e110–e117. [DOI] [PubMed] [Google Scholar]
- 16.Hernan MA, Robins JM. Causal Inference. Boca Raton, FL: Chapman & Hall/CRC; (in press). [Google Scholar]
- 17.Westreich D. From exposures to population interventions: pregnancy and response to HIV therapy. Am J Epidemiol. 2014;1797:797–806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Petersen ML, Wang Y, van der Laan MJ, et al. Assessing the effectiveness of antiretroviral adherence interventions. Using marginal structural models to replicate the findings of randomized controlled trials. J Acquir Immune Defic Syndr. 2006;43(suppl 1):S96–S103. [DOI] [PubMed] [Google Scholar]
- 19.Robins J. A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. J Chronic Dis. 1987;40(suppl 2):139S–161S. [DOI] [PubMed] [Google Scholar]
- 20.Westreich D, Cole SR, Young JG, et al. The parametric g-formula to estimate the effect of highly active antiretroviral therapy on incident AIDS or death. Stat Med. 2012;3118:2000–2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Patel MR, Nana M, Yotebieng M, et al. Delayed antiretroviral therapy despite integrated treatment for tuberculosis and HIV infection. Int J Tuberc Lung Dis. 2014;186:694–699. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ahern J, Hubbard A, Galea S. Estimating the effects of potential public health interventions on population disease burden: a step-by-step illustration of causal inference methods. Am J Epidemiol. 2009;1699:1140–1147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Taubman SL, Robins JM, Mittleman MA, et al. Intervening on risk factors for coronary heart disease: an application of the parametric g-formula. Int J Epidemiol. 2009;386:1599–1611. [DOI] [PMC free article] [PubMed] [Google Scholar]