Abstract
Background
Although randomized controlled trials are the gold standard design for cause-effect analysis, high costs and challenges around practicability, feasibility, and ethics may limit their use. In such situations, causal inference methods can improve the rigor of cause-effect analysis using observational data but such methods have infrequently been applied in tuberculosis (TB) research. We conducted a parallel comparison across three causal inference methods in order to assess the causal association between missed clinic visit/s and treatment success among people with drug-susceptible bacteriologically confirmed pulmonary TB.
Methods
We used causal inference methods to analyze cross-sectional data of adults with drug-susceptible bacteriologically confirmed pulmonary TB at clinics in rural eastern Uganda. We compared effect estimates from three causal inference methods, namely instrumental variable analysis, propensity-score analysis (adjustment, matching, weighting, and stratification), and double-robust estimation for cause-effect analysis. The exposure was missing a TB clinic visit/s and the outcome was treatment success defined as cure or treatment completion, both measured on a binary scale. Covariates were selected based on the literature, and their social and biological relevance to the outcome. We report the odds ratio and 95% confidence interval from each causal analysis.
Results
Of 762 participants (mean age of 39.3 ± 15.8 years) included, 186 (24.4%) had missed a clinic visit/s while 687 (90.2%) were successfully treated for TB. Missed clinic visit/s lowered treatment success across all analyses with instrumental variable analysis (OR 0.41, 95% CI 0.20–0.82), propensity-score analysis (adjustment [OR 0.49, 95% CI 0.30–0.82], matching [OR 0.43, 95% CI 0.21–0.91)], weighting [OR 0.52, 95% CI 0.30–0.91], and stratification [OR 0.34, 95% CI 0.19–0.62]), and double-robust estimation (OR 0.49, 95% CI 0.28–0.85).
Conclusions
Missed clinic visit/s reduced the likelihood of TB treatment success rate across all causal inference methods, supporting a causal relationship. Studies are needed to examine interventions that enhance retention in TB treatment.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12874-025-02553-x.
Keywords: Causal inference, Double-robust Estimation, Propensity score analysis, Treatment success, Tuberculosis
Background
Classic epidemiology methods provide a basis for analysis to establish a relationship between exposure and outcome. Although associations can be established, causality cannot be confirmed. For tuberculosis (TB) research, causality provides compelling evidence of the effectiveness of interventions. Questions on causality are best answered using randomized controlled trials as the gold standard design because of their potential to achieve comparability on both measured and unmeasured factors [1], thereby providing an unbiased cause-effect estimate. However, randomized controlled trials have limitations. In certain situations, they may be unethical, impractical, or simply very expensive to undertake [2]. Findings from randomized controlled trials often have limited generalizability due to the recruitment of specific populations and post-randomization confounding [3]. As an alternative, TB researchers may use observational data to answer cause-effect questions [4]. Observational data may offer greater external validity as they emerge from diverse and real-world settings.
Data from observational studies have inherent challenges that limit their generalizability. The data carry selection bias (systematic differences between the exposed and unexposed groups) and confounding of the exposure and the outcome relationship by both known and unknown extraneous factors due to a lack of randomization [4]. To mitigate these limitations, TB researchers conventionally use multivariable regression analysis, adjusting for possible measured confounders to answer cause-effect questions [5]. However, multivariable regression analysis only adjusts for measured confounders and the model might easily be misspecified [6], hence limiting its use in causal analysis. Accordingly, causal inference methods have been developed to mitigate selection bias and confounding associated with observational data in order to emulate a randomized controlled trial.
Although a variety of causal inference methods are available, for TB research, few studies have conducted parallel comparisons of the causal inference methodologies in the estimation of the measure of effect. Accordingly, we demonstrate the application of causal inference methods to evaluate the effect of a missed clinic visit/s on treatment success among people with drug-susceptible pulmonary TB in rural eastern Uganda. We theorized that missed clinic visit/s interrupt TB care continuity hence compromising treatment adherence and lowering the TB treatment success rate—a measure of the optimal performance of TB control programs. We applied three causal inference methods, namely instrumental variable analysis, propensity-score analysis, and double-robust estimation in the assessment.
The exposure (missed clinic visit/s) preceded the outcome (treatment success) so causal claims are possible provided selection bias and confounding have been mitigated.
Methods
Data source description
We analyzed real-world TB data described in previous studies [7–12]. The Research Ethics Committee at Mbarara University of Science and Technology (reference number 03/11–18) and the Uganda National Council for Science and Technology (reference number HS 2531) approved the parent study. For the current data analysis, a waiver from the ethics committee was granted. Briefly, the data are from a cohort of people with drug-susceptible bacteriologically confirmed pulmonary TB (PTB) who had received TB care across 10 TB clinics in rural eastern Uganda between 2015 and 2018. The data were drawn from the routine TB register using a standardized data abstraction tool. The participants were treated with the 6-month anti-TB regimen that comprised isoniazid (H), rifampicin (R), pyrazinamide (Z), and ethambutol (E) administered daily as a fixed-dose combination for 2 months during the intensive phase, followed by 4 months of RH only during the continuation phase (2RHZE/4RH). The 6-month anti-TB drugs required people with TB to visit the TB clinic 8 times—bi-weekly for 2 months and then monthly for 4 months. Some of the participants treated for TB before August 2018 received an 8-month anti-TB regimen that comprised RHZE and Streptomycin (S) for 2 months followed by RHZE for one month during the intensive phase, and RHE for 5 months during the continuation phase (2RHZES/1RHZE/5RHE). The eight-month regimen required people with TB to visit the TB clinic 10 times—bi-weekly for two months and monthly for 6 months.
Design and measurements
We used the observational data to design a quasi-experimental study. The exposure was missing one or more scheduled TB clinic appointments—missed TB clinic visit/s. The outcome was treatment success defined per the World Health Organization (WHO) and the Uganda Ministry of Health (MoH) guidelines as cure or treatment completion. Both the exposure and outcome were measured on a binary scale. Individuals were considered cured of TB if they had a negative sputum smear test result at the end of TB treatment (at month 6 for the 6-month regimen and month 8 for the 8-month regimen) and on one previous occasion (at month 2 or 5 for the 6-month anti-TB regimen, and at month 3 and 5 for the 8-month regimen). Individuals with treatment outcomes such as death, treatment failure (positive sputum smear test result at month 5 or thereafter), and lost-to-follow-up were considered unsuccessfully treated for TB.
Individuals whose treatment outcome was not reported or had been transferred to another health facility (treatment outcome not evaluated) were excluded as we could not assign definitive outcomes.
The covariates included the level of health facility (hospital vs. health center), health facility location (rural vs. urban), health facility ownership (private vs. government), age, sex, type of person with TB (newly diagnosed or previously treated for TB), whether transferred in or not, whether tested for HIV or not, HIV serostatus, type of directly observed therapy short course (DOTS) being facility or community, treatment supporter availability, and geographic access to the TB clinic measured by residence in the same sub-county as the TB clinic.
Clinical trial number
Not applicable.
Statistical analysis
The data analysis was done in Stata version 15 (College Station, Texas 77845 USA) and R version 4.2.1 (2022-06-23 ucrt). In the descriptive analysis, the mean and standard deviation were used to summarize normally distributed numerical data like age, otherwise the median and interquartile range were used. Categorical data such as sex were summarized using frequencies and percentages. Bivariate analysis was conducted by cross-tabulating the outcome (treatment success) with the covariates. Differences in the outcome with categorical data were tested using the Chi-square test when cell frequencies were at least five (≥ 5) or Fisher’s exact test when it was less than five (< 5). Mean differences in the outcome with numerical data were evaluated using the student’s t-test when the data were normally distributed, else the Wilcoxon rank-sum test was used when the data were skewed.
For cause-effect analysis, we applied three causal inference methods, including instrumental variable analysis, propensity score analysis (adjustment, weighting, and matching), and double-robust estimation. We describe the methods as follows:
-
Instrumental variable analysis: This method uses a variable known as an instrument or instrumental variable for cause-effect analysis. For the instrument to be valid, it must be correlated with the exposure, directly uncorrelated with the outcome except through its effect on the exposure, and uncorrelated with measured and unmeasured confounders [13]. The instrument isolates variation in the exposure independent of the selection process as if the exposure was randomly assigned, and removes bias resulting from confounding of the exposure-outcome relationship.
This way, instrumental variable analysis simulates a randomized controlled trial. Unlike multivariable regression analysis, which might yield biased estimates due to confounding, unmeasured confounders, and selection bias, instrumental variable analysis controls for both known and unknown confounders.
An instrumental variable analysis must satisfy three assumptions: (1) relevance criterion—the instrument must be correlated with the exposure; (2) exclusion restriction criterion—the instrument must be uncorrelated with the outcome except through its effect on the exposure; and (3) exogeneity criterion—the instrument must be uncorrelated with confounders. These assumptions are presented in Fig. 1. We considered participant residence in the same sub-county as the TB clinic as the instrument for the cause-effect analysis as it fulfilled all three criteria. We assessed the relevance criteria through correlation analysis, considering F-statistics greater than 10 as confirmatory [14]. We assessed distributional similarity in covariates and treatment success across the instrument as confirming the exogeneity and exclusion restriction criteria [15]. We then applied the two-stage least squares regression approach for causal analysis, with the first stage comprising a logistic regression model for the exposure as a function of the instrument and covariates to generate predicted coefficients. The second stage was a binary logistic regression model for the outcome as a function of the predicted coefficients in the first stage, adjusted for all covariates [16]. In a previous study [12], we showed that the exogeneity and exclusion restriction criteria were met.
-
Propensity score analysis: This analytic approach was appropriate as the exposed and unexposed groups were systematically different on measured covariates. Propensity score analyses remove systematic differences between groups thereby ensuring an unbiased measure of effect. The propensity score is the probability of assignment to the exposure conditional on measured covariates and the scores range from 0 to 1 [17]. First, we used a binary logistic regression model for the exposure as a function of covariates known from the literature to influence the outcome or to account for differences in the exposure (conditional independence assumption). Second, the coefficients from the regression output were used to generate propensity scores. The propensity scores could equally be generated using a generalized boosted model allowing the automatic adding of polynomials or interaction terms whenever needed [18]. Lastly, we used the propensity scores for adjustment, weighting, matching, and stratification as recommended [17].
- In propensity-score adjustment, we performed a binary logistic regression analysis for the outcome as a function of the exposure, adjusted for the propensity scores to establish causal effects. By adjusting for the propensity score alone (single measure), all measured confounders are controlled for in the analysis [17].
- In propensity-score weighting, we created a pseudo-population in which all measured covariates are balanced by weighting the exposed group using the inverse of the propensity score (1/propensity score) and the unexposed group using 1/1-propensity score [19] hence simulating a randomized controlled trial [6]. Covariate balance was confirmed with standardized mean difference (SMD) being < 0.1 [18] and propensity score mirror histogram to show distributional similarity in propensity scores between the exposed and unexposed groups. The SMD was preferred over the probability value (p-values) as the latter is sample size dependent [20]. Once covariate balance was confirmed, a binary logistic regression analysis was fitted for the outcome as a function of the exposure, adjusted for the propensity score weights. We assessed the correctness of the propensity score model specification based on the null hypothesis that the model was correctly specified [21].
-
In propensity-score matching, we matched the exposed and unexposed groups based on similar propensity scores and excluded observations that could not be matched on the propensity scores. Doing so created identical exposed and unexposed groups as if it were a randomized controlled trial. Of the several matching approaches such as exact matching, optimal matching, full matching, coarsened exact matching, and nearest neighbor matching with or without replacement and with or without a caliper, we used the 1:1 nearest neighbor matching with caliper as it achieved balance for all the measured covariates. The literature on how these matching methods work and how they are implemented is beyond the scope of this paper. We implore TB researchers to contact additional resources. For the matching, we used a caliper computed as 20% of the standard deviation of the propensity scores as recommended [20] and in our previous studies [7, 22, 23]. We assessed covariate balance using SMD being < 0.1 and distributional similarity in covariates using a histogram and jitter plot [24]. Once we confirmed covariate balance, we performed causal analysis by fitting a conditional logistic regression analysis for the outcome as a function of the exposure, adjusted for the matched pairs [20]. We checked the robustness of the findings using the Rosenbaum-Wilcoxon test.We considered distant gamma values from the point of unmeasured confounders (hidden bias) as confirmatory of robust results [24].
- In propensity score stratification, we used the propensity scores to create 4 strata within which the covariates were anticipated to achieve balance. The strata were based on the percentiles of the propensity scores [25]. Within each stratum, we assessed covariate balance using a propensity-score histogram and established cause-effect using a binary logistic regression. The overall cause-effect was computed as the weighted average, with the weights taken as the proportion of the sample in each stratum [25].
Fig. 1.
Directed Acyclic Graph (DAG) illustrating the use of an instrumental variable to estimate the causal effect of exposure on outcome in the presence of measured and unmeasured confounding. The DAG depicts the causal relationships in a study where an Instrumental Variable (IV) is used to estimate the effect of an Exposure (X) on an Outcome (Y), while accounting for both measured confounders (C1, C2, etc.) and unmeasured confounders (U)
-
c.
Double robust estimation: This causal analysis combined both exposure and outcome regression models, and provided two options instead of one to correctly estimate the causal effect of the exposure on the outcome [26]. Estimates from double robust estimation are consistently correct as long as one of the two models is correctly specified but not necessarily both [5]. The double robust estimation approach has one major advantage over the other causal inference approaches as it prevents model misspecification [27]. We fitted the exposure and outcome model as a function of a priori-determined covariates.
The odds ratio and its 95% confidence interval were the measure of cause-effect in all analyses. The statistical analysis codes are provided as Additional file 1.
Results
Participant characteristics and TB treatment outcomes by exposure status
We analyzed data from 762 participants, with a mean age of 39.3 ± 15.8 years (Table 1). Of the participants, 186 (24.4%) missed a TB clinic visit/s while 687 (90.2%) were successfully treated for TB. Systematic differences between participants with and without a missed TB clinic visit/s were noticeable concerning the health facility, residence, HIV testing, HIV test results, presence of a child under 5 years in the household, treatment supporter availability, and treatment outcomes, including treatment success.
Table 1.
Participant characteristics and TB treatment outcomes by exposure status
| Variables | Level | All (n = 762) | Missed TB Clinic visit/s | P-value | |
|---|---|---|---|---|---|
| No (n = 576) | Yes (n = 186) | ||||
| Health facility | District Hospital | 368 (48.3) | 234 (40.6) | 134 (72.0) | < 0.001 |
| Health center IV | 185 (24.3) | 143 (24.8) | 42 (22.6) | ||
| Regional Referral Hospital | 209 (27.4) | 199 (34.5) | 10 (5.4) | ||
| Ownership | Private not-for-profit | 659 (86.5) | 505 (87.7) | 154 (82.8) | 0.117 |
| Government | 103 (13.5) | 71 (12.3) | 32 (17.2) | ||
| Location | Rural | 254 (33.3) | 150 (26.0) | 104 (55.9) | < 0.001 |
| Peri-urban | 508 (66.7) | 426 (74.0) | 82 (44.1) | ||
| Age group (years) | 15–34 | 343 (45.0) | 262 (45.5) | 81 (43.5) | 0.856 |
| 35–50 | 254 (33.3) | 189 (32.8) | 65 (34.9) | ||
| 51 and over | 165 (21.7) | 125 (21.7) | 40 (21.5) | ||
| mean (SD) | 39.3 (15.8) | 39.3 (16.0) | 39.2 (15.3) | 0.920 | |
| Sex | Male | 502 (65.9) | 379 (65.8) | 123 (66.1) | 1.000 |
| Female | 260 (34.1) | 197 (34.2) | 63 (33.9) | ||
| Person with TB | Newly diagnosed | 671 (88.1) | 502 (87.2) | 169 (90.9) | 0.220 |
| Previously treated | 91 (11.9) | 74 (12.8) | 17 (9.1) | ||
| Transferred from another TB Unit | No | 688 (90.3) | 526 (91.3) | 162 (87.1) | 0.121 |
| Yes | 74 (9.7) | 50 (8.7) | 24 (12.9) | ||
| Tested for HIV | No | 9 (1.2) | 3 (0.5) | 6 (3.2) | 0.010 |
| Yes | 753 (98.8) | 573 (99.5) | 180 (96.8) | ||
| HIV test result | Negative | 543 (71.3) | 425 (73.8) | 118 (63.4) | 0.001 |
| Positive | 210 (27.6) | 148 (25.7) | 62 (33.3) | ||
| Known positive | 9 (1.2) | 3 (0.5) | 6 (3.2) | ||
| Has a child under 5 years in the household | No | 430 (56.4) | 297 (51.6) | 133 (71.5) | < 0.001 |
| Yes | 332 (43.6) | 279 (48.4) | 53 (28.5) | ||
| Type of DOTS | Facility | 43 (5.6) | 30 (5.2) | 13 (7.0) | 0.464 |
| Community | 719 (94.4) | 546 (94.8) | 173 (93.0) | ||
| Treatment supporter availability | No | 109 (14.3) | 70 (12.2) | 39 (21.0) | 0.004 |
| Yes | 653 (85.7) | 506 (87.8) | 147 (79.0) | ||
| Treatment outcomes | Cured | 413 (54.2) | 367 (63.7) | 46 (24.7) | < 0.001 |
| Completed | 274 (36.0) | 162 (28.1) | 112 (60.2) | ||
| Failed | 14 (1.8) | 9 (1.6) | 5 (2.7) | ||
| Dead | 61 (8.0) | 38 (6.6) | 23 (12.4) | ||
| Treatment success | No | 75 (9.8) | 47 (8.2) | 28 (15.1) | 0.009 |
| Yes | 687 (90.2) | 529 (91.8) | 158 (84.9) | ||
Note: DOTS: Directly Observed Therapy Short Course; HIV: Human Immunodeficiency Virus; SD: standard deviation
Effect of a missed clinic visit/s on treatment success
Instrumental variable analysis
We found an F-statistics of 22.4 (p < 0.001). Since the F-statistics was greater than 10, the instrument strongly correlated with the exposure and was thus relevant.
We found distributional similarity in baseline covariates and treatment success across the instrument thus confirming the exogeneity and exclusion restriction criteria were met as earlier mentioned. In the cause-effect analysis, missed TB clinic visit/s reduced the TB treatment success rate by 59% (OR 0.41, 95% CI 0.20–0.82).
Propensity-score analysis
Propensity-score adjustment
After adjusting for propensity scores, we found that missed clinic visits/s reduced the TB treatment success by 51% (OR 0.49, 95% CI 0.30–0.82).
Propensity-score matching
We matched 282 observations in a 1:1 ratio, and all the covariates were balanced between the exposed and unexposed groups. Figure 2 confirmed the covariate balance following propensity score matching as the histograms became compared to before the matching. Further covariate balance was revealed by the jitter plot (Fig. 3, middle section) as the clustering of the propensity scores in the exposed (matched treated units) and unexposed (matched control units) groups were identical after matching compared to before matching (unmatched treated units vs. unmatched control units).
Fig. 2.
Propensity-score histograms before and after propensity-score matching. The “raw treated” and “row control” are the exposed and unexposed groups before propensity score matching (left column), respectively. The “matched treated” and “matched control” are the exposed and unexposed groups after propensity score matching (right column), respectively
Fig. 3.
Jitter plot showing propensity scores before and after matching. The middle section (highlighted using a black rectangle) shows the propensity score distribution after matching. The “matched treated units” and the “matched control units” show the exposed and unexposed groups after propensity score matching. The “unmatched treated units” and the “unmatched control units” (outermost sections) show the exposed and unexposed groups before propensity score matching, respectively
The propensity-score matched analysis showed that treatment success was reduced by 57% among those with a missed TB clinic visit/s compared to those without any missed clinic visit (OR 0.43, 95% CI 0.21–0.91). Sensitivity analysis using the Rosenbaum sensitivity test indicated that this effect estimate was robust to unmeasured confounders as the lower bound of the odds shifted from a statistically insignificant value (Gamma value = 1, point of no hidden bias) to a statistically significant value (p = 0.03) at Gamma value of 7.75. This value is distant from the point of no hidden bias (Gamma value = 1), suggesting that the finding was robust to unmeasured confounders.
Propensity-score weighted analysis
Table 2 shows that the covariates, namely health facility level characteristics (level, location, and ownership), residence HIV serostatus, and treatment supporter availability were imbalanced before weighting as all SMD exceeded 0.1. After propensity-score weighting, the groups became similar on all measured covariates as the SMDs were less than 0.1. The propensity-score weighted analysis indicated that the TB treatment success rate was reduced by 48% among participants with a missed TB clinic visit/s compared to those without any missed clinic visit (OR 0.52, 95% CI 0.30–0.91).
Table 2.
Covariate distribution before and after propensity score weighting
| Variables | Covariate distribution before propensity score weighting | ||||
|---|---|---|---|---|---|
| Exposed group | Unexposed group | SMD | |||
| mean | SD | mean | SD | ||
| Level of health facility | 1.82 | 0.51 | 2.10 | 0.77 | -0.427 |
| Health facility ownership type | 1.82 | 0.38 | 1.88 | 0.33 | -0.156 |
| Health facility location | 0.45 | 0.50 | 0.74 | 0.44 | -0.626 |
| Residence | 1.59 | 0.49 | 1.40 | 0.49 | 0.385 |
| Sex | 1.66 | 0.48 | 1.66 | 0.48 | 0.010 |
| MTB load | 2.73 | 0.97 | 2.81 | 1.03 | -0.080 |
| HIV serostatus | 1.34 | 0.48 | 1.26 | 0.44 | 0.188 |
| Treatment supporter availability | 1.79 | 0.41 | 1.88 | 0.32 | -0.251 |
| Type of DOTS | 1.07 | 0.26 | 1.05 | 0.22 | 0.090 |
| Covariate distribution after propensity score weighting | |||||
| Variables | mean | SD | mean | SD | SMD |
| Level of health facility | 1.82 | 0.52 | 1.80 | 0.64 | 0.032 |
| Health facility ownership type | 1.81 | 0.39 | 1.81 | 0.39 | 0.010 |
| Health facility location | 0.47 | 0.50 | 0.48 | 0.50 | -0.012 |
| Residence | 1.57 | 0.50 | 1.55 | 0.50 | 0.039 |
| Sex | 1.66 | 0.47 | 1.66 | 0.48 | 0.007 |
| MTB load | 2.76 | 0.97 | 2.76 | 1.04 | -0.004 |
| HIV serostatus | 1.33 | 0.47 | 1.32 | 0.47 | 0.014 |
| Treatment supporter availability | 1.80 | 0.40 | 1.77 | 0.42 | 0.050 |
| Type of DOTS | 1.07 | 0.25 | 1.07 | 0.26 | -0.021 |
Note: DOTS: Directly Observed Therapy Short Course; HIV: Human Immunodeficiency Virus; MTB: Mycobacterium TB; SD: standard deviation. SMD: Standardized mean difference; SMD < 0.1 confirms covariate balance
Propensity-score stratification
Figure 4 is a histogram showing covariate balance based on propensity score stratification. All the strata show differences in the distribution of the propensity score distribution after the stratification, suggesting covariate imbalance. The stratum-specific effect estimates for the TB treatment success rate in the first stratum were an odds ratio of 0.40 (95% CI 0.18, 0.87), an OR of 0.15 in the second stratum (95% CI 0.03, 0.81), an OR of 0.37 (95% CI 0.10, 1.37) in the third stratum, and an OR of 0.63 (95% CI 0.15, 2.64) in the fourth stratum. The overall weighted effect estimate was 0.34 (95% CI 0.19–0.62), representing a 66% decline in the TB treatment success as a result of a missed clinic visit/s.
Fig. 4.
Propensity-score histograms before and after propensity-score stratification. The raw treated and raw control (left column) are the exposed and unexposed groups before propensity score stratification, respectively. The “matched treated” is the exposed group while the “matched control” is the unexposed group after propensity score stratification (right column). The vertical dotted lines in the “matched treated” and “matched control” show the four strata
Double robust Estimation
In double-robust estimation, findings showed that missed clinic visit/s reduced the TB treatment success rate by 51% (OR 0.49, 95% CI 0.28–0.85).
Effect of missed clinic visit/s on treatment success
Table 3 summarizes the cause-effect analysis findings. Findings showed that missed clinic visit/s reduced treatment success at varying degrees based on instrumental variable analysis, propensity-score analysis, and double robust estimation approaches.
Table 3.
Effect of missed clinic visit/s on treatment success
| Statistical method | Effect estimates (OR, 95% CI) |
|---|---|
| Instrumental variable analysis | 0.41* (0.20–0.82) |
| Double-robust estimation | 0.49* (0.28–0.85) |
| Propensity score analysis | |
| a. Weighting | 0.52* (0.30–0.91) |
| b. Adjustment | 0.49** (0.30–0.82) |
| c. Matching (+) | 0.43* (0.21–0.91) |
| d. Stratification | 0.34** (0.19–0.62) |
Note: All odds ratios are the exponentiated coefficients with the 95% confidence interval in brackets. Statistical significance codes at the 5% level: * p < 0.05, ** p < 0.01, *** p < 0.001. (+) denotes that the analysis included 282 observations (141 participants in the exposed groups vs. 141 participants in the unexposed group)
Discussion
We conducted a parallel comparison of the measure of effect across three causal-inference methods in evaluating the effect of a missed clinic visit/s on treatment success rate among people with drug-susceptible bacteriologically confirmed pulmonary TB in rural eastern Uganda. Across all three methods, missing a TB clinic visit/s reduces the TB treatment success rate. The findings across the different methodologies converged hence strengthening the hypothesis that missed clinic visit/s negatively impact treatment success among people with TB. The differences in the cause-effect estimates between the methods were minimal. These differences may be explained by variations in the assumptions and adjustments used to mitigate confounding and selection bias. The instrumental variable analysis corrects for endogeneity by using an instrument that influences missed clinic visit/s. This instrument affects treatment success only through its effect on missed clinic visit/s [12–16, 28]. Instrumental variable analysis is particularly useful in addressing reverse causality and unmeasured confounding. Its estimates ofen reflect a more conservative and robust cause-effect.
Key limitations of the instrumental variable method include the challenges of identifying a valid instrument, the requirement to satisfy all its assumptions, and the difficulty in ensuring these assumptions hold. However, when a valid instrument exists, the instrumental variable analysis provides the most accurate estimate in observational data. Therefore, it should be prioritized as it controls for all confounders—measured, unmeasured, and unknown. The propensity score methods, namely adjustment, weighting, matching, and stratification essentially balance the covariates between the exposed and unexposed groups to approximate a randomized controlled trial.
Discrepancies in effect estimates arise due to differences in analytic procedures. For example, the propensity score matching excluded approximately 63% of the observations as they could not get perfect matches. The reduction in sample sizes may have resulted in a less precise estimate compared to other methods. Propensity score adjustment is easy to implement and hence tempting to use but does not permit covariate balance checks [25]. The downside of weighting is that participants with treatment probabilities nearer to 0 or 1 may experience unstable weights. Weight stabilization has been recommended to normalize the range of propensity scores [19]. Weighting methods like standardized mortality ratio weighting and weighted odds ratio have key limitations. Standardized mortality ratio assumes homogeneity of treatment effects across subgroups, which may not hold, potentially leading to biased conclusions. The choice of reference group can also influence results, particularly if it is poorly matched or confounded [29]. The weighted odds ratio is difficult to interpret, especially when combined with methods like the inverse probability of treatment weighting or standardized mortality ratio, and is sensitive to model specification and extreme weights, which can distort treatment effects and increase variance [30]. The propensity-score stratification did not balance all the covariates as shown in previous studies [25, 31]. One approach to correct the covariate imbalance would be to re-specify the propensity score model. In general, the propensity score methods assume all relevant confounders have been measured and accounted for, which is often not the case [32]. The double robust estimation combines both propensity score-based adjustment and regression models, offering protection against misspecification of either model and producing reliable estimates, assuming both models are correctly specified [27, 33]. Researchers need to consider these limitations as they plan to use these approaches.
Data quality issues, such as measurement errors, may contribute to discrepancies in effect estimates, particularly in propensity score analyses, which are highly sensitive to such limitations. Therefore, ensuring accurate data measurement is critical across TB control programs. The exclusion of many participants in the propensity score-matched analysis may have inadvertently removed individuals whose data could have provided valuable insights. This may have affected the generalizability and precision of the findings. Model misspecification, especially in the selection of covariates, may also explain variations in effect estimates. In both propensity score analysis and double robust estimation approaches, we relied on the conditional independence assumption by including variables known to affect both the exposure and the outcome [34]. The sensitivity of causal inference methods to model specification underscores the importance of rigorously defining and testing analytical assumptions. In TB research, where data are often complex and sometimes incomplete, even minor changes in model specifications can significantly impact results, underscoring the importance of careful design and analytic rigor.
Our data show a strong causal relationship between missed clinic visit/s and treatment success, with consistent effect estimates across the analytic methods. The convergence of results from multiple causal inference techniques strengthens the validity of the findings, indicating that missed clinic visit/s are not a spurious result of a single method. Analysis of data using causal inference methods should consider applying multiple statistical approaches to enhance the validity of findings. Our finding revealed that interventions aimed at improving patient adherence to clinic visit/s should be prioritized in tackling missed clinic visit/s. For instance, reminders, educational campaigns, or increased access to clinic services through person-centered measures like flexible hours and multi-month dispensing of anti-TB drugs. Such initiatives would improve treatment outcomes and reduce the burden of TB in settings with high rates of missed appointments.
Comparative effectiveness of causal inference methods in observational studies
The effectiveness of the various causal inference methods is based on several considerations. Instrumental variable analysis is particularly suitable when unmeasured confounding is present, and a valid instrument exists. Double robust estimation is advantageous in minimizing the risk of model misspecification.
Propensity score weighting is notable for its efficiency and generalizability while propensity score matching is effective in reducing confounding and enhancing interpretability. Propensity score stratification is straightforward to implement but less efficient in achieving covariate balance. Propensity score adjustment is the least effective in adequately addressing confounders. In summary, when a valid instrument exists, instrumental variable analysis would be regarded as the gold standard, as it effectively accounts for unmeasured confounding. In the absence of a valid instrument, double robust estimation provides the strongest protection against model misspecification. Among the propensity score methods, weighting is typically favored for its efficiency, followed by matching due to its transparency. Propensity score adjustment and stratification, while useful, are generally considered less robust options.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Acknowledgements
We thank the District Health Offices of Soroti, Kumi, Ngora, and Serere for their logistical support during the data collection. We thank the TB focal persons at the respective TB clinics for their invaluable help to the research assistants and the Principal Investigator.
Abbreviations
- CI
Confidence Interval
- DOTS
Directly Observed Therapy Short Course
- OR
Odds ratio
- PTB
Pulmonary tuberculosis
- RHZE
Rifampicin, Isoniazid, Pyrazinamide, and Ethambutol
- RHZES
Rifampicin, Isoniazid, Pyrazinamide, Ethambutol, and Streptomycin
- SMD
Standardized mean difference
- MTB
Mycobacterium tuberculosis
- TB
Tuberculosis
- WHO
World Health Organization
Author contributions
JI and FB conceptualized the study. JI acquired and analyzed the data. JI, AC, and FB interpreted the findings and drafted the manuscript. AC and FB revised the manuscript for intellectual content. All authors (JI, AC, and FB) approved the final version of the manuscript.
Funding
None.
Data availability
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Declarations
Ethics approval and consent to participate
We analyzed real-world TB data described in previous studies. The Research Ethics Committee at Mbarara University of Science and Technology (reference number 03/11–18) and the Uganda National Council for Science and Technology (reference number HS 2531) approved the parent study. For the current data analysis, a waiver from the ethics committee was granted.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Kendall JM. Designing a research project: randomised controlled trials and their principles. Emerg Med J. 2003;20(2):164–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ohlsson H, Kendler KS. Applying causal inference methods in psychiatric epidemiology: A review. JAMA Psychiatry. 2020;77(6):637–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Fernainy P, Cohen AA, Murray E, Losina E, Lamontagne F, Sourial N. Rethinking the pros and cons of randomized controlled trials and observational studies in the era of big data and advanced methods: a panel discussion. BMC Proceedings. 2024;18(2):1. [DOI] [PMC free article] [PubMed]
- 4.Igelström E, Craig P, Lewsey J, Lynch J, Pearce A, Katikireddi SV. Causal inference and effect Estimation using observational data. J Epidemiol Commun Health. 2022;76(11):960. [Google Scholar]
- 5.Tchetgen Tchetgen EJ. On a closed-form doubly robust estimator of the adjusted odds ratio for a binary exposure. Am J Epidemiol. 2013;177(11):1314–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Li X, Shen C. Doubly robust Estimation of causal effect: upping the odds of getting the right answers. Circulation Cardiovasc Qual Outcomes. 2020;13(1):e006065. [DOI] [PubMed] [Google Scholar]
- 7.Izudi J, Tamwesigire IK, Bajunirwe F. Association between GeneXpert diagnosis and Same-Day initiation of tuberculosis treatment in rural Eastern Uganda. Am J Trop Med Hyg. 2020;103(4):1447–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Izudi J, Tamwesigire IK, Bajunirwe F. Treatment success and mortality among adults with tuberculosis in rural Eastern Uganda: a retrospective cohort study. BMC Public Health. 2020;20(1):501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Izudi J, Tamwesigire IK, Bajunirwe F. Treatment supporters and level of health facility influence completion of sputum smear monitoring among tuberculosis patients in rural Uganda: A mixed-methods study. Int J Infect Dis. 2020;91:149–55. [DOI] [PubMed] [Google Scholar]
- 10.Izudi J, Tamwesigire IK, Bajunirwe F. Sputum smear non-conversion among adult persons with bacteriologically confirmed pulmonary tuberculosis in rural Eastern Uganda. J Clin Tuberculosis Other Mycobact Dis. 2020;20:100168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Izudi J, Tamwesigire IK, Bajunirwe F. Surveillance for multi-drug and rifampicin resistant tuberculosis and treatment outcomes among previously treated persons with tuberculosis in the era of GeneXpert in rural Eastern Uganda. J Clin Tuberculosis Other Mycobact Dis. 2020;19:100153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Izudi J, Tamwesigire IK, Bajunirwe F. Effect of missed clinic visits on treatment outcomes among people with tuberculosis: a quasi-experimental study utilizing instrumental variable analysis. IJID Reg. 2024;13:100461. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Widding-Havneraas T, Zachrisson HD. A gentle introduction to instrumental variables. J Clin Epidemiol. 2022;149:203–5. [DOI] [PubMed] [Google Scholar]
- 14.Ertefaie A, Small DS, Flory JH, Hennessy S. A tutorial on the use of instrumental variables in pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2017;26(4):357–67. [DOI] [PubMed] [Google Scholar]
- 15.Huang HH, Cagle PJ Jr., Mazumdar M, Poeran J. Statistics in brief: instrumental variable analysis: an underutilized method in orthopaedic research. Clin Orthop Relat Res. 2019;477(7):1750–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Greenland S. An introduction to instrumental variables for epidemiologists. Int J Epidemiol. 2018;47(1):358. [DOI] [PubMed] [Google Scholar]
- 17.Okoli GN, Sanders RD, Myles P. Demystifying propensity scores. Br J Anaesth. 2014;112(1):13–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Olmos A, Govindasamy P. A practical guide for using propensity score weighting in R. Practical Assess Res Evaluation. 2019;20(1):13. [Google Scholar]
- 19.Ali MS, Groenwold RH, Klungel OH. Best (but oft-forgotten) practices: propensity score methods in clinical nutrition research. Am J Clin Nutr. 2016;104(2):247–58. [DOI] [PubMed] [Google Scholar]
- 20.Staffa SJ, Zurakowski D. Five steps to successfully implement and evaluate propensity score matching in clinical research studies. Anesth Analg. 2018;127(4):1066–73. [DOI] [PubMed] [Google Scholar]
- 21.Izudi J, Bajunirwe F, Cattamanchi A. Negative effects of undernutrition on sputum smear conversion and treatment success among retreatment cases in Uganda: A quasi-experimental study. J Clin Tuberculosis Other Mycobact Dis. 2024;35:100422. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Izudi J, Okoboi S, Lwevola P, Kadengye D, Bajunirwe F. Effect of disclosure of HIV status on patient representation and adherence to clinic visits in Eastern Uganda: A propensity-score matched analysis. PLoS ONE. 2021;16(10):e0258745. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Izudi J, Tamwesigire IK, Bajunirwe F. Does completion of sputum smear monitoring have an effect on treatment success and cure rate among adult tuberculosis patients in rural Eastern Uganda? A propensity score-matched analysis. PLoS ONE. 2019;14(12):e0226919. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Olmos A, Govindasamy P. Propensity scores: a practical introduction using R. J MultiDisciplinary Evaluation. 2015;11(25):68–88. [Google Scholar]
- 25.Lanza ST, Moore JE, Butera NM. Drawing causal inferences using propensity scores: a practical guide for community psychologists. Am J Community Psychol. 2013;52(3–4):380–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Orsini N, Bellocco R, Sjölander A. Doubly robust Estimation in generalized linear models. Stata J. 2013;13(1):185–205. [Google Scholar]
- 27.Funk MJ, Westreich D, Wiesen C, Stürmer T, Brookhart MA, Davidian M. Doubly robust Estimation of causal effects. Am J Epidemiol. 2011;173(7):761–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Earle CC, Tsai JS, Gelber RD, Weinstein MC, Neumann PJ, Weeks JC. Effectiveness of chemotherapy for advanced lung cancer in the elderly: instrumental variable and propensity analysis. J Clin Oncology: Official J Am Soc Clin Oncol. 2001;19(4):1064–70. [DOI] [PubMed] [Google Scholar]
- 29.Brookhart MA, Wyss R, Layton JB, Stürmer T. Propensity score methods for confounding control in nonexperimental research. Circulation Cardiovasc Qual Outcomes. 2013;6(5):604–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Heagerty PJ, Zeger SL. Multivariate continuation ratio models: connections and caveats. Biometrics. 2000;56(3):719–32. [DOI] [PubMed] [Google Scholar]
- 31.Steiner PM, Cook TD, Shadish WR, Clark MH. The importance of covariate selection in controlling for selection bias in observational studies. Psychol Methods. 2010;15(3):250–67. [DOI] [PubMed] [Google Scholar]
- 32.Lee J, Little TD. A practical guide to propensity score analysis for applied clinical research. Behav Res Ther. 2017;98:76–90. [DOI] [PubMed] [Google Scholar]
- 33.Kurz CF. Augmented inverse probability weighting and the double robustness property. Med Decis Making: Int J Soc Med Decis Mak. 2022;42(2):156–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Gertler PJ, Martinez S, Premand P, Rawlings LB, Vermeersch CMJ. Impact evaluation in practice: World Bank; 2016.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.