Estimating treatment effects from non-overlapping cohorts with application to antimicrobial resistance

Abstract

Background

The positivity assumption, also known as overlap, is one of the key assumptions for causal inference. However, in certain observational studies it does not hold. This work is motivated by such a scenario, where the goal was to compare the effect of ceftriaxone versus cefuroxime treatment on subsequent 3rd generation cephalosporin resistance within one year post-treatment in Gram-negative bacteria among hospitalized patients. Our data consisted of electronic medical records from 24,969 patients hospitalized between 2015–2025 in two hospitals with distinct policies: one prescribed cefuroxime, while the other prescribed ceftriaxone. These prescription policies created two non-overlapping patient cohorts, as each hospital treated patients with either cefuroxime or ceftriaxone. We developed and applied a statistical framework that leverages additional data, for which positivity holds, and clinically driven assumptions, to identify the average treatment effect among these non-overlapping cohorts, without removing any observations.

Methods

Our framework is based on a logistic regression model for the conditional expected bacterial resistance outcome, and it can be applied to any generalized linear model. The model is identifiable under a clinically driven assumption that the effect modification of the hospital on cefazolin equals that of cefuroxime and ceftriaxone, when compared to penicillins. We rely on cefazolin and penicillins being prescribed in both hospitals, and on penicillins inducing negligible resistance to 3rd generation cephalosporins. We also implemented inverse probability of censoring weighting for censored outcomes, and utilized aminoglycoside resistance as a negative control outcome to assess sufficient confounding adjustment.

Results

The estimated average treatment effect, on the risk difference scale, of ceftriaxone compared to cefuroxime was 5.83% (95% CI: -2.06%, 13.39%). The conditional odds ratio for ceftriaxone compared to cefuroxime, adjusted for confounders, was 1.31 (95% CI: 0.91, 1.89). No evidence of unmeasured confounding was found using the negative control outcome falsification test.

Conclusions

We develop and apply a statistical framework applicable in settings of non-overlapping cohorts that violate the positivity assumption. Our framework allows identification of the average treatment effect thanks to parametric assumptions about the conditional expected outcome, additional data and clinically driven assumptions. The empirical findings, though not statistically significant, may point to a modest resistance increase with ceftriaxone compared to cefuroxime use; and suggest further research about the preferential use of cefuroxime is needed.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12874-026-02947-5.

Introduction

The positivity assumption requires that all treatment levels could appear in every subgroup defined by the measured confounders [1, 2]. When this assumption is violated, the absence of comparable patients across treatment levels renders the treatment effect unidentifiable from the data. Positivity might be violated in two forms: 1. Random violation, where all treatment levels can theoretically appear in every subgroup of the study population, but are not observed due to finite sample size. 2. Structural violation, where some subgroups have a zero probability of receiving a given treatment. A common solution is trimming, which redefines the study population by excluding subgroups where positivity is violated [1–5]. Trimming alters the target of inference, and is undesirable if the excluded observations are central to the research question. When redefining the study population is impossible, regression models can extrapolate into regions where treatment groups do not overlap [4]. A recently developed parametric approach addresses this scenario but assumes no interactions between the treatment and the subgroups in which positivity is violated [6], an assumption that is implausible when such interactions exist. In this paper, we present a statistical framework accommodating such interactions.

Antibiotic resistance is a pressing challenge in modern healthcare [7], driven in large part by the overuse and misuse of antibiotics [7–10]. Although prior antibiotic exposure is a well-established risk factor for resistant infections [8, 11, 12], quantitative causal estimates attributable to specific antibiotics are limited. Consequently, antibiotics are often ranked by their purported risks, relying on expert opinions [13–15]. Recent applications of causal inference methodology, including matching, instrumental variables, and principal stratification, have begun to address this gap [16–19].

Here, we focus on cefuroxime and ceftriaxone, 2nd and 3rd generation cephalosporins, respectively, routinely prescribed as first-line therapy for hospitalized patients in Israel [20–22]. Several mechanisms suggest that ceftriaxone may carry a greater risk of selecting for resistance than cefuroxime. First, ceftriaxone has broader Gram-negative coverage [23], exerting greater selective and ecological pressure on bacterial populations. Second, ceftriaxone reaches the gut microbiota at substantially higher concentrations than parenteral cefuroxime: cefuroxime is eliminated by the kidneys with negligible fecal levels [24], whereas a substantial fraction of intravenous ceftriaxone is excreted in bile as an active drug [25]. This exposure has been linked to an increase in AmpC beta-lactamase overproducers in the gut [26]. Beyond AmpC induction, the use of 3rd generation cephalosporins, and particularly ceftriaxone, has been associated with the emergence of extended-spectrum β-lactamase (ESBL)-producing organisms [27–29] and with elevated risk of Clostridioides difficile infection [30, 31].

These two considerations converge in the present study. We compared between treatment with ceftriaxone and cefuroxime with respect to subsequent risk of antibiotic resistance in Gram-negative bacteria among inpatients within one-year post-treatment. We used electronic medical records (EMR) from two secondary Israeli hospitals with distinct prescribing policies: Meir Medical Center (MMC), where cefuroxime but not ceftriaxone is predominantly prescribed, and Samson Assuta Ashdod Hospital (SAAH), where ceftriaxone but not cefuroxime is primarily prescribed. These policies yield two non-overlapping cohorts, and a structural violation of the positivity assumption [1, 32]. In this setting, trimming was impractical, as it would exclude the patients treated with cefuroxime or ceftriaxone. Assuming no treatment-hospital interaction [6] was also untenable, given the hospitals' distinct bacterial environments. We therefore developed a framework that combines a statistical identification assumption with a biological property of antibiotics prescribed in both hospitals: cefazolin (a 1 st generation cephalosporin), and the penicillins amoxicillin, ampicillin and penicillin. The identification assumption is that the effect modification by the hospitals on cefuroxime and ceftriaxone equals that of cefazolin. The biological property is that these penicillins induce negligible resistance to 3rd generation cephalosporins. We used a logistic regression model to estimate the Average Treatment Effect (ATE) of ceftriaxone versus cefuroxime on subsequent 3rd generation cephalosporin resistance, on the Risk Difference (RD) scale, with the biological property enabling a clear interpretation of the model. We implemented Inverse Probability of Censoring Weighting (IPCW) for censored outcomes, and utilized aminoglycoside resistance as a Negative Control Outcome (NCO), to ascertain sufficient confounding adjustment.

Methods

Data and study design

This retrospective cohort study was based on the EMR of all patients above the age of 18 who had at least one positive clinical culture containing Gram-negative bacteria obtained between July 2015-June 2024 and July 2018-June 2024, in MMC, Kfar-Saba and SAAH, respectively. MMC and SAAH are secondary hospitals with 797 and 344 beds, respectively, located in central and southern Israel. Clinical data comprised antibiotic susceptibility test results for all Gram-negative bacteria isolated, alongside comprehensive demographic and clinical covariates describing the patients characteristics and clinical history.

The treatment variable was the first use of cefuroxime or ceftriaxone, the main antibiotics of interest, during the study periods. Additionally, as mentioned earlier, treatment by four other antibiotics prescribed in both hospitals was considered as well: cefazolin, amoxicillin, ampicillin and penicillin. Any treatment durations and all administration routes were considered. In SAAH, ceftriaxone was administered almost exclusively intravenously (98.37%), with the remaining cases receiving it intramuscularly. At Meir Medical Center (MMC), cefuroxime was administered intravenously in 88.0% of cases, with the remainder administered orally. The outcome variable was binary susceptibility test results of subsequent clinical cultures obtained up to one-year post-treatment, to any of the following 3rd generation cephalosporins: ceftriaxone, cefdinir, cefixime, cefpodoxime, cefditoren, ceftibuten, ceftazidime, cefotaxime, cefoperazone, and ceftizoxime. The time period of one-year post-treatment was chosen as the increased risk of Antimicrobial Resistance (AMR) has been shown to remain elevated for up to a year post-treatment [33–35]. Resistance to 3rd generation cephalosporins was selected as the outcome variable, as it serves as a reliable marker for Extended Spectrum Beta-Lactamase (ESBL) resistance [20–22, 36, 37]. Cultures collected from patients up to three days post-treatment were not considered, as these cultures might have resulted from treatment failure.

Confounders were selected from the recorded demographic and clinical data up to 6 months prior to treatment. The demographic confounders were sex, age, and the comorbidities diabetes, Chronic Obstructive Pulmonary Disease (COPD), and Chronic Renal Failure (CRF). The clinical confounders were in-hospital prior use of cephalosporins, penicillins, carbapenems and fluoroquinolones, prior use of a urinary catheter, prior isolation of ESBL Enterobacterales and Methicillin-Resistant Staphylococcus aureus (MRSA) infections, and the hospital ward in which antibiotic prescriptions were administered. To address potential confounding by bacterial resistance patterns at the institutional-level, the models were also adjusted for covariates that measure the average 3rd generation cephalosporin-resistance of Gram-negative bacteria during the 7 and 30 days prior to treatment allocation, at the unit-level. As these covariates did not change the results materially, their inclusion appears as sensitivity analyses (see Supplementary material, Tables S9-S10).

Ceftriaxone was scarcely prescribed in MMC (2.12% of all antibiotic prescriptions), as was cefuroxime in SAAH (2.34% of all antibiotic prescriptions). In MMC, ceftriaxone was mainly used to treat meningitis and severe pneumonia; and in SAAH, cefuroxime was mainly administered as surgical prophylaxis. These antibiotic treatments were not used for first-line therapy, and occurred in patients with disparate characteristics from the rest of the study population (see Table S1). For example, cefuroxime users in SAAH were mainly men hospitalized for surgery (73.3%), a rate more than double that of such patients among the rest of the study population in SAAH (see Table 1, Table S1). Hence, these cases were excluded from the study to avoid potential confounding from variables we cannot control for, as they represent a different patient subpopulation. As an additional sensitivity analysis, we restricted the analysis to antibiotic prescriptions administered in internal medicine units (16,009 prescriptions across 11,030 patients). This allowed us to examine the robustness of our results in a more homogeneous population, less susceptible to unmeasured confounding stemming from baseline clinical patient heterogeneity (see Supplementary material, Table S11).

Table 1.

Characteristics of the study population at first antibiotic prescription, stratified by hospital

	Meir Medical Center	Samson Assuta Ashdod Hospital
N	18,271	6698
Sex (Male) (%)	8198 (44.9)	2884 (43.1)
Age (median (IQR))	72.67 (23.73)	71.57 (32.77)
Treatment level (%)
Amoxicillin	104 (0.6)	126 (1.9)
Ampicillin	1307 (7.2)	1198 (17.9)
Cefazolin	4000 (21.9)	2348 (35.1)
Ceftriaxone	0 (0.0)	2962 (44.2)
Cefuroxime	12,554 (68.7)	0 (0.0)
Penicillin	306 (1.7)	64 (1.0)
Treatment unit (%)
ER	2978 (16.3)	968 (14.5)
ICU	385 (2.1)	522 (7.8)
Internal	7223 (39.5)	2048 (30.6)
Ob/Gyn	1226 (6.7)	691 (10.3)
Orthopedic	394 (2.2)	58 (0.9)
Other	1325 (7.3)	10 (0.1)
Surgical	4740 (25.9)	2401 (35.8)
Censoring status (%)
Censored by antibiotic use	7455 (40.8)	3696 (55.2)
Censored by death	1087 (5.9)	183 (2.7)
Did not return	7092 (38.8)	2107 (31.5)
Uncensored	2637 (14.4)	712 (10.6)
Had a urinary catheter prior 6 months (%)	4299 (23.5)	2983 (44.5)
Has diabetes (%)	3911 (21.4)	1085 (16.2)
Has COPD (%)	1061 (5.8)	342 (5.1)
Has CRF (%)	1900 (10.4)	897 (13.4)
Had ESBL prior 6 months (%)	631 (3.5)	430 (6.4)
Had MRSA prior 6 months (%)	190 (1.0)	66 (1.0)
Used cephalosporin prior 6 months (%)	465 (2.5)	420 (6.3)
Used penicillin prior 6 months (%)	679 (3.7)	409 (6.1)
Used carbapenem prior 6 months (%)	30 (0.2)	55 (0.8)
Used fluoroquinolone prior 6 months (%)	133 (0.7)	254 (3.8)
Cultures with resistance (%)	1528 (29.8)	719 (41.1)

Abbreviations: MRSA Methicillin-resistant Staphylococcus aureus, ESBL Extended Spectrum Beta-Lactamase producing bacteria, ICU Intensive Care Unit, IQR Interquartile Range, Ob/Gyn Obstetrician-Gynecologist, COPD Chronic Obstructive Pulmonary Disease, CRF Chronic Renal Failure

The study population is depicted in Figure S1. Patients that were treated with antibiotics during the three days prior to treatment were excluded from the study, and we refer to this period as "washout period". Use of antibiotics that are not associated with future cephalosporin resistance during the washout period was disregarded (see list in Supplementary Material). Outcomes were considered censored in three cases: (a) the patient did not have a relevant bacterial susceptibility test result during the year post-treatment; (b) another antibiotic was prescribed post-treatment and before a susceptibility test was performed; and (c) the patient died post-treatment and before a relevant susceptibility test was performed (see Discussion). Only 5.95% of patients in MMC and 2.70% of patients in SAAH had outcomes that were censored due to death (see Table 1). Patients with censored outcomes were accounted for using IPCW.

Statistical analysis

Let denote the antibiotic treatment, where 0 stands for cefazolin, 1 stands for cefuroxime, 2 stands for ceftriaxone, and 3 for any of the following three antibiotics from the penicillins group: amoxicillin, ampicillin and penicillin; and let be the binary indicator of treatment level A=a. Let be the outcome of the first susceptibility test for 3rd generation cephalosporin resistance, performed on any Gram-negative bacteria isolated from clinical cultures collected from patients within one-year post-treatment. In each such culture, a susceptibility test to at least one of the following 3rd generation cephalosporins was performed: ceftazidime, ceftriaxone, cefdinir, cefixime, cefpodoxime, cefditoren, ceftibuten, cefotaxime, cefoperazone and ceftizoxime. The outcome Y was set to 0 if all bacteria in a culture were susceptible to all tested 3rd generation cephalosporins; if any bacteria in the culture were resistant, then Y was set to 1. Using the potential outcomes framework [1, 32], let be the potential resistance outcome had a patient received treatment level A=a. Let be the hospital in which the treatment was assigned, where 0 stands for MMC and 1 stands for SAAH; let be a vector of the measured confounders detailed in Data; and let be a vector of unmeasured variables. These variables might include, for example, outpatient antibiotic use, or community-level AMR. Our primary estimand was the ATE of ceftriaxone use (A=2) compared to cefuroxime use (A=1), on resistance to 3rd generation cephalosporin within one-year post-treatment on the RD scale, denoted as . Additional estimands were the conditional Odds Ratios (OR) of cefuroxime and ceftriaxone treatments compared to the baseline treatment, cefazolin, adjusted for the confounders; and the conditional OR of ceftriaxone treatment compared to cefuroxime treatment, adjusted for the confounders.

We assumed no-interference, consistency, and conditional exchangeability [1, 38]. No-interference means that the resistance status for each patient is not affected by the antibiotic treatments that other patients have received. Consistency means that if A=a then . Figure 1 depicts our structural assumptions using a Directed Acyclic Graph (DAG) [38]. The conditional exchangeability assumption is with respect to and H, namely, . In this figure, the U→ A arrow does not exist, so all non-causal paths from A to Y go through and H [38, 39]. In our dataset, this assumption is arguably plausible since includes a comprehensive set of demographic and clinical confounders (see Table 1 and Table S5).

Fig. 1.

A DAG conceptually formulating the causal assumptions. The outcome Y is the observed 3rd generation cephalosporin bacterial resistance; A is the antibiotic treatment; H is the hospital in which the treatment was assigned; X is measured confounder(s); and U is unmeasured variable(s). The key assumption in this DAG is the lack of a direct arrow from U to A

To provide additional support to the validity of the conditional exchangeability assumption, we utilized susceptibility test results for aminoglycosides (gentamicin, tobramycin, neomycin and amikacin) as an NCO [40–42] for falsification testing. The NCO falsification test rests on two main assumptions. First, the NCO is not affected by cephalosporin use. Aminoglycoside resistance test results were chosen as an NCO due to their low cross-resistance with cephalosporins, stemming from distinct resistance mechanisms, and their widespread inclusion in susceptibility testing [43, 44]. In 97.4% of cases where a 3rd generation cephalosporin resistance outcome was available, an aminoglycoside resistance outcome was available too. Second, it is reasonable to assume that 3rd generation cephalosporin resistance and aminoglycoside resistance share the same unmeasured confounding structure; e.g. by exposure to resistant bacteria in the patient’s community. In the NCO analysis, the main analysis was repeated after cephalosporin resistance outcomes were replaced by the first aminoglycoside resistance outcomes within one-year post-treatment.

To identify the ATE and the conditional ORs, another causal assumption, the positivity assumption (also known as overlap), is typically needed [1, 32]. Positivity means that for every combination of confounders , within each hospital H, there is a positive probability of being treated with any of the treatment levels. That is, Since positivity did not hold in this study, the standard identification formula for the ATE,

1

cannot be applied to the data. Due to the violation of positivity, the inner expectation in the first term in Eq. (1), cannot be estimated for H=0, and the inner expectation in the second term in Eq. (1) cannot be estimated for H=1.

If positivity was to hold, the inner expectations in Eq. (1) could have been estimated, for example, with a logistic regression model that includes separate interaction terms between cefuroxime, ceftriaxone and the hospital. One such model could have been

2

Under this model, captures the difference on the conditional log(OR) scale with respect to the baseline treatment in SAAH compared to MMC; are the coefficients of the confounders; are the main effects of cefuroxime and ceftriaxone, respectively, compared to the baseline treatment; and are the interaction terms between cefuroxime and ceftriaxone, respectively, and SAAH. The interaction terms alongside in Eq. (2) capture the effect modification in the treatment effect of cefuroxime and ceftriaxone on the conditional log(OR) scale with respect to treatment in SAAH compared to MMC. This effect modification by SAAH compared to MMC could, for example, stem from distinct bacterial environments in SAAH compared to MMC.

However, without positivity, the interaction terms in Eq. (2) are unidentifiable from the data. One solution could have been assuming no hospital-treatment interaction, resembling a previous work that estimated the average treatment effect on the treated in an entirely treated cohort [6], where positivity did not hold as well. A logistic regression model under such an assumption could have been

3

Nonetheless, assuming no hospital-treatment interaction is challenging to defend in the context of this study, as MMC and SAAH differ in their bacterial environments and clinical procedures.

Instead, we formulated a weaker assumption. Consider a logistic regression model in which cefazolin use is the baseline treatment (A=0), and treatment by penicillins (amoxicillin/ampicillin/penicillin) is considered as well This model includes an equi-interaction term, γ, between the cephalosporin treatments (cefuroxime, ceftriaxone and the baseline treatment cefazolin) and SAAH,

4

Under this model, the term () equals 0 for treatment by penicillins and 1 for treatment by any of the three cephalosporins. Consequently, this model implies that the difference, on the conditional log(OR) scale, between treatment by penicillins in SAAH compared to MMC is ; and similarly that the difference on the same scale between treatment by cephalosporins in SAAH compared to MMC is . The equi-interaction term γ expresses the assumption that the hospital modifies the treatment effects of cefuroxime, ceftriaxone and cefazolin compared with penicillins in the same way. Given that these three antibiotics belong to the cephalosporin group and have similar structure and mechanism of action, we found this assumption reasonable. Under this equi-interaction assumption, the parameters of the model in Eq. (4) are identifiable (See Supplementary material, Parameter identification). Due to structural differences, treatment by penicillins induces negligible resistance to 3rd generation cephalosporins [45, 46]. This biological property implies that the difference on the log(OR) scale between treatment by penicillins in SAAH compared to MMC, captured by , reflects hospital factors that are unrelated to the treatment, such as bacterial environment. Given this property, captures the hospital shift common to all treatments, while γ captures the hospital effect modification specific to cephalosporins.

Let the subscript i=1,… ,n denote the instance of antibiotic treatment prescribed to a patient, and we refer to each such instance as an antibiotic prescription. Let denote the type of antibiotic prescribed at the prescription. Let denote the susceptibility test outcome following the prescription. Let denote the set of measured confounders corresponding to the prescription; and let denote the hospital in which the prescription was administered. To mitigate potential bias due to censoring, we implemented IPCW while controlling for the set of confounders [1, 47, 48]. Let denote the censoring status indicator for the prescription, where 0 stands for an uncensored outcome, and 1 stands for a censored outcome. IPCW relies on the conditional exchangeability assumption regarding the joint intervention on treatment and censoring. Antibiotic prescriptions with censored outcomes were accounted for by assigning each prescription with an uncensored outcome the weight , where was the empirical probability of being uncensored among every treatment level (), and was estimated from a logistic regression of censoring status () depending on the treatment level (), and adjusted for the confounders . The form of this logistic regression was identical to Eq. (4), including the same covariates and interaction term, with the outcome being the censoring status indicator instead of resistance result (see Figures S2-S3, Tables S4-S5). Antibiotic prescriptions with a censored outcome were assigned .

The estimators of the parameters in Eq. (4), , and their robust standard errors were estimated using Generalized Estimating Equations (GEE) to account for within-patient correlation due to multiple antibiotic treatments, weighted by the inverse probability of censoring weights. The GEE procedure was used with a working independent correlation structure, as the results did not change materially when an exchangeable correlation structure was considered. Then, the empirical distribution of the covariates and the estimated coefficients was used to estimate the ATE by

5

Additionally, we used the estimated parameters from the model in Eq. (4) to calculate the conditional OR of cefuroxime and ceftriaxone treatment compared to cefazolin. We calculated Confidence Intervals (CI) for the ATE using bootstrap sampling of 1,000 repetitions, with block sampling (at the patient level) and with replacement from 18,271 MMC patients, and 6,698 SAAH patients, preserving the hospitals' proportions in the original data. The block sampling was performed to account for within-patient correlation. Finally, as a falsification test of the conditional exchangeability assumption, we repeated the same estimation procedure with the NCO aminoglycoside resistance as the outcome variable.

Results

We analyzed 39,999 antibiotic prescriptions across 24,969 patients, 18,271 from MMC and 6,698 from SAAH. Of these prescriptions, 5,279 (13.20%) had uncensored outcomes, and 34,720 (86.80%) had censored outcomes due to patient no-return, subsequent antibiotic use, or death, as described in Fig. S1. The characteristics of the study population at the time of first antibiotic prescription are summarized in Table 1 (see Table S2 for the characteristics of the study population weighted for censored outcomes). As expected, the two patient cohorts differed in a few aspects. Patients admitted to SAAH had higher previous antibiotic use and bacterial resistance rates than MMC. SAAH had a younger patient population compared to MMC, in accordance with the population distribution of the city where it is located. MMC had a lower rate of urinary catheter use due to a designated program aimed at urinary catheter use reduction; and as mentioned earlier, both hospitals have distinct antibiotic prescription policies that result in different treatment allocations.

The weighted logistic model was fitted to the data using the estimated stabilized IPCW weights, which ranged between 0.32–4.75 (see Tables S4-S5 and Figures S2-S3 for all IPCW regression coefficient estimates and related plots). Conditional OR estimates of resistance for the treatment levels, the equi-interaction term, and the hospital are presented in Fig. 2 (see Table S6 for all regression coefficient estimates). The conditional OR estimate for ceftriaxone treatment compared to cefazolin (1.97, 95% CI: 1.42, 2.74), was higher than that of cefuroxime treatment compared to cefazolin (1.50, 95% CI: 1.21, 1.87). Notably, the conditional OR for ceftriaxone treatment compared to cefuroxime treatment was 1.31 (95% CI: 0.91, 1.89).

Fig. 2.

Conditional OR estimates of resistance, from the GEE logistic regression model weighted for censored outcomes. Coefficient estimates of five prominent parameters from the model in Eq. (4) are presented on the conditional OR scale. Bars represent 95% confidence intervals (CI). Interaction is the conditional OR of resistance for the cephalosporin equi-interaction term, and Hospital captures the conditional OR of resistance for SAAH compared to MMC

The conditional OR estimate for treatment with any of the penicillins (amoxicillin/ampicillin/penicillin) compared to cefazolin in MMC was 1.46 (95% CI: 0.94, 2.28). The conditional OR estimate for being treated in SAAH compared to MMC for cefazolin use was 1.17 (95% CI: 0.67, 2.05), corresponding with reported higher bacterial resistance rates in SAAH compared to MMC. The conditional OR estimate for the equi-interaction term was 1.23 (95% CI: 0.66, 2.28). While effect modification by the hospitals is reasonable, the standard error of our estimators was too large to accurately determine the effect modification size.

Using the fitted model and following Eq. (5) in the Statistical Analysis section, the ATE, on the RD scale, of ceftriaxone compared to cefuroxime was estimated to be 5.83% (95% CI: −2.06%, 13.39%). The NCO falsification test produced a near-zero estimate for the ATE of 0.30% (95% CI: −5.37%, 7.00%), and did not provide evidence of unmeasured confounding. The regression coefficient estimates from the GEE fitted for the NCO falsification test, and the characteristics of the study population utilized for this test are presented in Tables S7-S8. The sensitivity analysis restricted to prescriptions administered in internal medicine units yielded a similar ATE estimate of 8.80% (95% CI: −10.10%, 29.92%), with the wide confidence interval reflecting the reduced sample size (see Supplementary material, Table S11).

Discussion

In this paper, we proposed a framework based on a parametric model for the conditional expected outcome, which considers interactions between the subgroups where positivity is violated and the treatment. Since the outcome in this study is binary, we assumed a logistic regression model. However, this framework can be naturally extended to other generalized linear models for the conditional expected outcome. Our framework assumes the effect modification in subgroups where positivity is violated equals that among similar groups in which positivity holds, and is therefore identifiable. This assumption is, in our view, clinically justifiable in the context of this study, and should be carefully considered before being applied elsewhere. Alternative assumptions could have been made. For example, we could have assumed the effect modification by the hospital for cefuroxime and ceftriaxone equals the average effect modification across several cephalosporins rather than cefazolin only. However, given the limited sample size of non-cefazolin cephalosporins prescribed in both hospitals, we chose to use cefazolin only. This could be reevaluated in future studies with larger sample sizes.

Using this framework, we estimated the ATE on the RD scale and the conditional OR, between ceftriaxone treatment and cefuroxime treatment. The ATE and the conditional OR for ceftriaxone treatment compared to cefuroxime showed higher point estimates of resistance risk with ceftriaxone, albeit the ATE had a wide confidence interval and a modest absolute effect. This may suggest that ceftriaxone use in inpatient settings bears a moderately higher risk of inducing future 3rd generation cephalosporin resistance among Gram-negative bacteria than cefuroxime, within a year post-treatment. The conditional OR for cefuroxime compared to cefazolin treatment indicated a higher resistance risk for cefuroxime use. This finding aligns with previous research among both hospitalized and outpatient populations, that suggested a higher resistance risk for 2nd generation cephalosporin use compared to 1 st generation cephalosporin use [17, 49]. This might be attributed to the broader spectrum of activity of 2nd generation cephalosporins compared to 1 st generation cephalosporins. Broader spectrum antibiotics exert greater selective pressure, consequently promoting the emergence and proliferation of resistant bacterial populations [50–52].

Our analysis has several limitations. First, despite our comprehensive adjustment for clinical and demographic confounders, conditional exchangeability is a non-testable assumption using the data. Thus, as in most observational studies, it remains a concern. However, the near-zero estimate from the NCO analysis did not reveal presence of unmeasured confounding. Second, our analysis assumes no-interference between patients. This assumption may be violated, as antibiotic use in one patient can affect resistance outcomes in other patients, primarily through the transfer of resistant bacteria between patients. To partially address this concern, we performed a sensitivity analysis in which we expanded our models to adjust for temporal unit-level resistance prevalence, and the results did not change materially. Additionally, patient and healthcare worker movement between the hospitals in this study is uncommon, since the hospitals are relatively distant from one another. Moreover, neither hospital serves as a tertiary referral center that draws patients from the other, and the hospitals belong to different health providers. Third, our framework relies on the key assumption that effect modification of treatment in SAAH compared to MMC is equal across all cephalosporin treatment levels. While biologically plausible given that cefuroxime, ceftriaxone and cefazolin belong to the same antibiotic group, this parametric assumption may not hold. Fourth, the confidence interval of the RD is relatively wide, reflecting limited sample size of patients with uncensored outcomes. Fifth, outcomes censored due to different reasons may indicate the patients had different characteristics, so the censoring might be informative. Nevertheless, we used a comprehensive set of measured covariates to control for such differences, and assigned weights to the uncensored patients accordingly. Sixth, we treated death as a censoring event, yet the potential resistance outcomes are not necessarily well defined among patients who died. So, the IPCW seemingly creates an immortal population in which patients do not die [53]. However, an important subtlety in this research is that while the potential outcomes are defined within patients, they are in fact a property of their bacterial population rather than the patients themselves. Hence, one can conceptually grasp the potential outcome to be defined even among patients who died. Furthermore, censoring by death was rare, and occurred only among 5.10% of patients, and the results did not change materially in a sensitivity analysis that excluded the patients who we treated as censored by death. In future research, alternative estimands can be considered [16, 54]. Finally, while our study utilized data from two different hospitals, both institutions operate within the same healthcare system, potentially limiting generalizability to other healthcare settings with different patient populations, bacterial environments, and clinical practices. Nevertheless, while these empirical results may be hospital-dependent, the proposed statistical framework can be applied to analogous observational studies facing structural violations of positivity.

Conclusion

We developed a statistical framework that enables identification and estimation of the causal ATE, despite structural violations of the positivity assumption among two non-overlapping patient cohorts. By applying our framework, we identified a modest rise in the resistance risk following ceftriaxone treatment compared to cefuroxime treatment, albeit under the aforementioned limitations. These results may provide justification for preferential use of cefuroxime over ceftriaxone, when clinically appropriate. However, further research into this question should be conducted on different populations and larger sample sizes.

Abbreviations

ATE

Average Treatment Effect

DAG

Directed Acyclic Graph

AMR

Antimicrobial Resistance

EMR

Electronic Medical Records

MMC

Meir Medical Center

SAAH

Samson Assuta Ashdod Hospital

RD

Risk Difference

NCO

Negative Control Outcome

IPCW

Inverse Probability of Censoring Weighting

ESBL

Extended Spectrum Beta-Lactamase

COPD

Chronic Obstructive Pulmonary Disease

CRF

Chronic Renal Failure

MRSA

Methicillin-Resistant Staphylococcus Aureus

GEE

Generalized Estimating Equations

CI

Confidence Interval

Authors’ contributions

D.N. and U.O. conceived the study and supervised all analysis; A.B. performed the analyses and simulations; M.C., A.C. and T.B. provided clinical insights; A.B., D.N. and U.O. wrote the paper; all authors critically revised and approved the paper.

Funding

This study was supported in part by a fellowship from the Edmond J. Safra Center for Bioinformatics at Tel-Aviv University, and by the Israel Science Foundation (ISF 1286/21).

Data availability

The data were obtained from Samson Assuta Ashdod University Hospital, Ashdod, Israel, and from Meir Medical Center, Kfar Saba, Israel, and are not publicly available since they are protected under medical confidentiality.

Declarations

Ethics approval and consent to participate

This study was conducted in accordance with the principles of the Declaration of Helsinki. The study was approved by the Institutional Review Board Committees of Meir Medical Center (reference no. MMC-0143–19) and Samson Assuta Ashdod Hospital (reference no. AAA-0071–22). Since this was a retrospective study using anonymized, archived medical records, an exemption from informed consent was granted by the committees.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.