The Ambulatory Diabetes Outreach Program (ADOP): Rigorous Evaluation of a Pharmacist and Nurse-Led Care Model

Abstract

Background

Although several systematic reviews found that ambulatory diabetes mellitus (DM) interventions involving pharmacists generally yielded better outcomes than the ones that did not, existing studies have limitations in rigor and study design.

Objective

To examine the intention-to-treat effects of the Ambulatory Diabetes Outreach Program (ADOP) on participants’ A1c values and healthcare utilization over a 52-month follow-up period.

Design

Difference-in-differences with staggered adoption. Specifically, we employed the Callaway and Sant’Anna’s “group-time average treatment effect” estimator using not-yet treated as controls adjusting for patient’s age, BMI, sex, race, comorbidity, payor, and socio-economic status.

Participants

All patients with at least one ADOP treatment encounter from July 2017 to October 2021, regardless of program completion or length of exposure to the program.

Intervention

ADOP, a collaborative population health program led by pharmacists and nurse specialists to provide individualized type 2 DM management and education within a large and diverse health system.

Main Measures

Patients’ A1c values and healthcare utilization, including inpatient admission, inpatient days, and numbers of visits to the emergency department, urgent care, and primary care in recent 6 months.

Key Results

ADOP participation was associated with an overall average reduction of 1.04 percentage points (95%CI − 1.12, − 0.95) in A1c level. Similar A1c reductions were also observed in the subgroups by sex and race/ethnicity. An average of 2 months were required to reach the overall average effect, which persisted over 4 years. Compared to the respective utilization levels pre-intervention, participants also had average reductions in inpatient admissions by 32.4%, inpatient days by 81.6%, visits to the emergency department by 21.6%, and primary care by 17.9%.

Conclusions

The results suggest that a collaborative model of pharmacist and nurse-led type 2 DM intervention was effective in improving A1c outcomes and reducing healthcare utilization in the long term.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11606-024-08970-w.

INTRODUCTION

The Centers for Disease Control and Prevention (CDC) estimates that 38.1 million US adults (14.7%) live with diabetes mellitus (DM), a chronic disease that imposes a substantial burden on society in terms of reduced productivity and quality of life, increased premature mortality, increased healthcare utilization, and increased medical costs.¹ The economic burden attributed to diabetes in the USA was almost $240 billion in 2017, and is rising.^2,3

The American Diabetes Association (ADA) has developed an active program to disseminate evidence-based care guidelines, but adherence to these guidelines in usual-care settings is suboptimal.^4,5 This fact has led to ADA recommendations for using an interdisciplinary team-based approach to assist with DM management.^6–9 Several systematic reviews and meta-analyses found that DM interventions involving pharmacists generally yielded better outcomes than usual care or the ones that did not.^10–13 However, the existing studies either used simple non-randomized designs that could not prevent contemporaneous trends from confounding the treatment effects of interest, or conducted RCTs with small numbers of patients and pharmacists for only a few months, where the pharmacists might be highly motivated and the patients in the treatment group could have higher adherence to the treatment plans than the average patient population. Therefore, more robust evidence is needed to demonstrate the effectiveness of multidisciplinary team care on type 2 DM management in real-world clinical settings in the long term.

The purpose of this paper is to evaluate the Ambulatory Diabetes Outreach Program (ADOP), a collaborative population health program led by pharmacists and nurse specialists to assist with the care of adults with uncontrolled type 2 DM at a large and diverse health system in southeastern Wisconsin. Using Callaway and Sant’Anna’s recently developed “group-time average treatment effect” estimator, we aimed to examine the intention-to-treat effects of the program on patient’s A1c values and healthcare utilization over a 52-month follow-up period.¹⁴ This analytic method also offered insights into the ramp-up speed and sustainability of the program effects.

METHODS

Intervention

Initiated in August 2016, ADOP consists of 4.5 FTE ambulatory clinical pharmacists and 1.0 FTE nurse certified diabetes care education specialists, who provide individualized diabetes management and education as extenders of the patient’s primary care team. Patients are referred through one of the two pathways: (1) a monthly population health report generated by the electronic health record (EHR) that identifies all potentially eligible patients having their last A1c value ≥ 9%; or (2) a PCP referral, whereby patients are referred due to fluctuating glucose values (can include A1c values < 9% at program start). The program excludes patients with any of the following conditions: type 1 diabetes, pregnancy, use of U500 insulin, or active management by endocrinology. After identifying eligible patients, the ADOP team attempts outreach up to four times (three phone calls and a letter or electronic communication) to schedule the first ADOP appointment. Patients are considered enrolled once they start the first appointment. The interval between the elevated A1c value that identified potential eligibility for the program and program enrollment had a mean (SD) of 30.2 (22.8) days; median of 26 days; IQR of 37 days. This interval included the time until the next monthly report was generated to identify the patient to the program team, as well as time for outreach and time between appointment scheduling and program enrollment. Through a collaborative practice agreement, pharmacists are delegated the authority to independently prescribe and adjust medications, while the nurse specialists can adjust medications via a protocol. As a patient-centric model, most ADOP encounters are provided telephonically with little additional cost to the patient. Patients who enroll in ADOP continue with the program until reaching one of the following end points: (1) program graduation, defined as achievement of patient-specific A1c goals, pre-determined with their PCPs prior to the program start; (2) resource exhaustion, defined as patient unwilling or unable to engage in changes necessary to achieve the A1c goals; (3) patient declines, defined as patients request to end program participation; and (4) lost to follow up, defined as patients unable to be reached after multiple attempts.

Study Population

All patients with at least one ADOP treatment encounter from July 2017 to October 2021, regardless of program completion or length of exposure.

Outcomes

Outcomes were changes in patients’ A1c values (primary) and healthcare utilization (secondary). For A1c values, where the estimation was based on a patient-month longitudinal structure, neighboring data points (if missing) were extrapolated with a “2-month-backward, 2-month-forward” method. Specifically, based on the literature suggesting that A1c values are a weighted average of blood glucose levels of the preceding 120 days, with about 75% of its level determined by the plasma glucose level during a prior 2-month period, the A1c values were extrapolated to their adjacent 2 months if the original monthly data points were missing.^15–17 Healthcare utilization outcomes included inpatient admission, inpatient days, and the number of visits to the emergency department, urgent care, and primary care, with each measured as the moving summation of a patient’s total utilization in recent 6 months.

Covariates

We controlled for potential confounders in the relationship between ADOP program participation and outcomes, including patient demographics (age, sex, race/ethnicity), body mass index (BMI), Elixhauser Comorbidity Index, primary payor (Medicare, Medicaid, commercial insurers, uninsured, others), and socio-economic status.¹⁸ Based on self-reported data, individuals were classified according to race/ethnicity as non-Hispanic Black/African American, Hispanic (any race), non-Hispanic white, and other race/ethnicity. Socio-economic status was proxied by the median household income of patient’s residential zip codes, extracted from the American Community Survey.

Statistical Analysis

Under the framework of difference-in-differences (DiD) with staggered adoption (i.e., once patients were enrolled, they were considered treated in the remaining time), we used the “group-time average treatment effects” estimator, also known as the ATT (g, t) estimator.¹⁴ Specifically, patients’ ambulatory visits were organized longitudinally by patient id and time t, in monthly increments with t ranging from 1 to 52. Enrolled patients were categorized into group g, defined as the month when a patient first enrolled. For each group g, in month t, we exclusively used ADOP participants who were not yet enrolled by month t as the controls (under usual care). Under the assumption of conditional parallel trends, we first point-estimated the treatment effect on the treated (ATT) at every group (g)-time(t) level using the doubly robust approach based on patient’s age, BMI, sex, race, comorbidity, payor, and median household income. We then aggregated all the individual ATT (g, t) estimates into an overall treatment effect for each outcome. For the primary outcome of A1c values, we repeated the estimation for subgroup strata defined by sex and race/ethnicity. We estimated the dynamic treatment effects to reveal the treatment effect heterogeneity by lengths of exposure to the program (across all groups). Patient-level cluster-robust standard errors and 95% simultaneous confidence intervals were produced under a multiplicative bootstrap procedure. Stata 18 (StataCorp LLC, College Station, TX) was used to produce the estimates. This study was approved by the Medical College of Wisconsin Institutional Review Board.

RESULTS

During the study period, 4589 patients enrolled in ADOP. Table 1 presents the summary statistics for the population at program enrollment. The mean age was 59 years, but the range was wide (18–96 years). About 63% were non-Hispanic White, 28% were Black, and 5.2% were Hispanic. The median household income was $81,422 (IQR = 48,444). Almost half the cohort had Medicare insurance, 38% were commercially insured, and 12% had Medicaid insurance. As expected, this cohort had high medical risk, with a mean BMI of 35, and a mean of 3.9 comorbid conditions. At program enrollment, the mean A1c value was 9.2% (SD = 2.0), and the healthcare utilization measures in the recent 6 months are summarized in Table 1. The mean (SD), median, and IQR for the numbers of ADOP touch points per participant were 10.8 (10.2), 8, and 10 times.

Table 1.

Summary Statistics for ADOP Patients at Program Enrollment

	At treatment start (unique patients n = 4589)
Sex, n (%)
Female	2161 (47.1)
Male	2428 (52.9)
Age, mean (SD)	59.2 (13.0)
Race, n (%)
Non-Hispanic White	2880 (62.8)
Non-Hispanic Black	1295 (28.2)
Hispanic	240 (5.2)
Other race/ethnicity	165 (3.6)
Unknown	9 (0.2)
Household income, median (IQR)	$81,422 (48,444)
Insurance, n (%)
Medicare	115,076 (48.2)
Commercial	90,688 (38.0)
Medicaid	27,872 (11.7)
Others	4836 (2.0)
Uninsured	156 (0.1)
Elixhauser comorbidities, n (%)
0–1	300 (6.5)
2–3	1985 (43.3)
4–5	1438 (31.3)
6–7	620 (13.5)
8–15	246 (5.4)
BMI, mean (SD)	35.3 (8.1)
Outcomes
A1c (%), mean (SD)	9.2 (2.0)
Healthcare utilization (recent 6-month total), mean (SD)
Inpatient admission	0.18 (0.7)
Inpatient days	0.68 (3.1)
ED visits	0.25 (0.7)
Urgent care visits	0.11 (0.4)
Primary care visits	2.7 (2.5)

By using the not-yet-treated exclusively as controls, the control group is essentially the same cohort as the treatment group

Table 2 displays the DiD estimates of the overall average treatment effect on the treated (ATT) by outcome. The “baseline” column provides the outcome-specific mean of the treatment group at 1 month prior to the program enrollment. The “rate of change”, with the overall ATT estimates divided by the “baseline”, provides the relative effect of the program when compared to the pre-intervention levels. Using the primary outcome of A1c as an example, ADOP enrollment was associated with an overall average reduction of 1.04 percentage points (95%CI − 1.12, − 0.95) in A1c level over 4 years, a relative reduction of 11.1% when compared to the baseline A1c level of 9.34%. ATT estimates of A1c for subgroup strata defined by sex and race/ethnicity were also statistically significant. Likewise, the program was associated with a reduction of 32.4% in inpatient admissions and a reduction in inpatient days by an average of 0.52 day (95%CI − 0.85, − 0.18) over a 6-month period, a relative reduction of 81.6% compared to the baseline. The number of emergency department visits and primary care visits each declined significantly, by 18–22%. Urgent care visits did not change significantly.

Table 2.

Overall Average Treatment Effects for the Treated (ATT) Estimates by Outcome

Outcome	Overall ATT a	95%CI b	Baseline c	Rate of change d
A1c whole population (%)	− 1.04	− 1.12, − 0.95	9.34	− 11.1%
A1c sub-populations (%)
Male	− 1.00	− 1.11, − 0.89	9.30	− 10.8%
Female	− 1.05	− 1.23, − 0.98	9.39	− 11.2%
Non-Hispanic White	− 1.13	− 1.22, − 1.04	9.15	− 12.3%
Non-Hispanic Black	− 1.04	− 1.24, − 0.84	9.73	− 10.7%
Hispanic and others	− 0.92	− 1.30, − 0.53	9.51	− 9.7%
Healthcare utilization (recent 6-month total)e
Inpatient admission	− 0.05	− 0.09, − 0.01	0.16	− 32.4%
Inpatient days	− 0.52	− 0.85, − 0.18	0.63	− 81.6%
ED visits	− 0.05	− 0.08, − 0.02	0.22	− 21.6%
Urgent care visits	− 0.01	− 0.03, 0.01	0.10	− 10.8%
Primary care visits	− 0.39	− 0.49, − 0.28	2.15	− 17.9%

^aDoubly robust ATT (g, t) estimates adjusting for patient covariates: age, BMI, sex, race, comorbidity, payor, and median household income. Consistent bias-corrected estimator and significance test as proposed in Abadie and Imbens (2004, 2006, 2011). n of patients = 4589; n of total observations = 229,245

^b95% simultaneous confidence intervals with multiplicative bootstrapped standard errors clustered at patient level

^cBaseline is calculated as the outcome-specific mean of the treatment group 1 month prior to the program enrollment

^d Rate of change = overall ATT/baseline. It shows the relative effect by outcome when compared to the pre-intervention level

^e Each type of healthcare utilization was measured as the moving summation of the corresponding outcome in the recent 6 months, e.g., the baseline of inpatient days = 0.63 means at 1 month prior to the program enrollment, the average total inpatient days within the recent 6 months was 0.63 days per patient

Figure 1 plots the dynamic ATT estimates of the program on patients’ A1c values (see Appendix Table 1 for specific estimates). Each dot (with 95% CI) represents the difference in estimates of the A1c values between the treatment and the control groups by X months before (X < 0, blue dots), or after (X > 0, red dots) treatment start (X = 0). The graph demonstrates that the parallel trends assumption held pre-treatment, where the differences in average A1c values between the two groups pre-treatment were constant (close to 0) within 2 years prior to program enrollment. It also reveals the rapidity of the effect and the durability of the improvement in A1c values, as shown by the red dots. It took an average of 2 months for the program to reach its average overall effect, which was subsequently sustained throughout the 4-year average follow-up period. These findings were remarkably robust to different A1c value extrapolation schemes used in the sensitivity analyses (e.g., 2-mo backward 1-mo forward; 3-mo backward 1-mo forward; 3-mo backward only). Interestingly, within 3 months prior to program enrollment, there was a small increase in A1c values among the program participants (Appendix Table 1).

Figure 1.

Estimate plot of dynamic average treatment effect on the treated.

DISCUSSION

We found ADOP program enrollment was associated with a statistically significant reduction in A1c values by an average of 1.04 percentage points (95%CI − 1.12, − 0.95). Importantly, program participation was associated with reductions in the secondary outcomes of inpatient admission, inpatient days, visits to the emergency department, and primary care. The dynamic aggregation of the individual group-time estimates indicated that it took about 2 months for the program to achieve its full effect, which was sustained over the remainder of the study period. The consistency in primary and secondary results and the sustainability of effects support attribution of the improvement to enrollment in the program.

When comparing effects of DM interventions with pharmacists to those without (including usual care), several systematic reviews have favored interventions involving pharmacists.^10–14 However, the evidence underlying these reviews relies on studies involving a small number of patients followed for a short time.^19,20 By examining the 48 + months experience of 4589 patients, our study provides important evidence to suggest that even among less selected patient populations with uncontrolled type 2 DM, the practice model of pharmacist and nurse specialist-led DM intervention produces significant and sustainable improvements in A1c value and reduced healthcare utilization. In addition to increasing access to care to a larger population, a possible explanation for ADOP’s success may be the longer-term active engagement and adoption of therapeutic changes.

Our quasi-experimental design is a strength of this study. Most evaluations of clinical programs rely on pre-post comparisons that cannot address confounding from contemporaneous trends unrelated to the treatment. Our specific setup of the ATT (g, t) estimator, however, offered several advantages. First, the strategy of defining the treatment groups based on months when patients were first treated, and the pairing with ADOP participants who were not-yet-enrolled by then as their contemporary control group, enabled us to adjust for varying treatment timing. It also allowed us to mechanically net out all common confounding effects from contemporaneous trends through the monthly DiDs (such as improvements in medication and/or clinical management, impacts from the COVID-19 pandemic) that might otherwise bias the estimates. Second, the ATT (g, t) estimator was robust to varying treatment effects (across patients or/and over time).²¹ In addition, unlike alternative estimators, which often restrict the usage of “never-treated” individuals as controls, the ATT (g, t) estimator was flexible to allow using “not-yet-treated” patients as controls, overcoming the critical challenge of identifying comparable-but-never-treated patients to use as the comparison group.²² Finally, our study design overcame the problem of regression to the mean, a common source of bias when program enrollees are selected based on high measures of a given factor (in this case, A1c levels). The high A1c values of ADOP participants, for example, might have a greater natural tendency to regress to lower levels over time even without the ADOP program. By taking advantage of the difference in treatment timing and using exclusively the “not-yet treated” as controls, the control group had a very similar patient composition, such that it shared similar amount of regression to the mean over time. Through taking differences in outcomes between the groups, regression to the mean in the treatment group would be mostly cancelled out, greatly mitigating the bias in the estimation.

The temporary increase in average A1c values prior to ADOP enrollment merits comment. The monthly population health reports may have captured patients’ accelerating deterioration in A1c values due, for example, to changes in insurance coverage or natural progression of the disease. However, it may also suggest the presence of the “Ashenfelter’s dip” among the program participants,²³ a phenomenon whereby program participants might temporarily relax their disease management in anticipation of accessing the new program in the near future, leading to worsened A1c values prior to program enrollment. DM interventions with short intervals between patient outreach and program enrollment, such as the average 30-day interval observed between lab value determination and ADOP enrollment, tend to minimize such changes in patient’s behaviors.

The study has limitations. First, a standard concern when evaluating voluntary programs, such as ADOP, is participant self-selection. To the extent that ADOP participants may possess greater risk profiles than those who opted not to enroll, our results could have overestimated the program effects. Second, despite its advantages over other estimation techniques, during the short period when the temporary spike occurs pre-intervention, the staggered DiD provides only an imperfect parallel “Ashenfelter’s dip” to that of the treatment group given the difficulty in accommodating the varying time of the temporary risk changes experienced by each patient in the comparison group. To examine whether our findings are robust to this limitation, we employed a strategy proposed by Card et al. (2011) for “Ashenfelter’s dip” using pre-program outcomes for several periods prior to program entry.²⁴ Specifically, we repeated the same estimations after removing patient’s A1c values for the 3 months prior to their first ADOP appointments. The overall ATT estimate was − 0.75 in A1c level (95%CI − 1.13, − 0.38), confirming the effectiveness of the intervention. Third, using the not-yet-treated as controls inherently sets an upper bound on the analysis’ time horizon as the controls eventually became treated at some later time. In this study with over 4500 participants under continuous enrollment, this limitation mainly led to the widened confidence intervals of the dynamic estimates around both ends of the relative timeline. Despite these qualifications, our application of the ATT (g, t) estimator in the context of a staggered adoption enabled the estimation of causal effects (rather than associations) in a situation where a RCT was not feasible.

In summary, our findings suggest that the collaborative model of pharmacist and nurse-led DM interventions was effective not only in improving A1c outcomes, but also in reducing healthcare utilization. The effects of the program applied to subgroups defined by gender and race/ethnicity and persisted up to 4 years post-program enrollment. Programs such as this one, working in collaboration with the primary care team, offer great opportunity for not only reducing the burden of the disease at the personal level, but also in saving costs to the healthcare system.

Supplementary Information

Below is the link to the electronic supplementary material.

Declarations:

Conflict of Interest:

Drs. Nattinger and Dong, and Ms. Penlesky reported their institution received compensation from Advancing a Healthier Wisconsin (MCW-associated non-profit organization) for a fraction of their salaries related to this work. Dr. Singh reported receiving consultation fees for quality of care for Asthma from Astra Zeneca. No other disclosures were reported by the remaining authors.