Multi-Site Quality Improvement Program Within the Project ECHO Diabetes Remote Network

Abstract

Background:

The tele-mentoring Project ECHO (Extension for Community Healthcare Outcomes) model has been shown to improve disease management in diabetes in many underserved communities. We aim to evaluate if ECHO could also be an effective tool for quality improvement (QI) of diabetes care in these communities.

Methods:

Thirteen clinics in underserved communities in California and Florida participating in Project ECHO Diabetes were recruited for a 12-month QI program. The program provided weekly tele-education sessions, including a didactic presentation and case-based discussion. In addition, clinics chose their own set of quality measures to improve and met remotely to discuss their efforts, successes, and setbacks every quarter with mentorship from QI experts.

Results:

Of the 31 QI initiatives attempted by different clinics, all had either made improvements (25 initiatives, 80.6%) or were in the process of making improvements (6 initiatives, 19.4%) in structural, process, and outcome measures. Examples of these measures include whether clinics have protocols to identify high risk patients (structure), numbers of Continuous Glucose Monitor prescriptions submitted by the clinics (process), and percentage of patients with diabetes whose most recent HbA1c are > 9% (outcome). For one measure, 40.0% of the clinics had achieved a higher percentage of cumulative HbAlc measurement in the third quarter of the year, compared to the fourth quarter in the previous year. The cost of QI implementation varied widely due to different number of personnel involved across sites.

Conclusions:

QI program delivered via Project ECHO Diabetes can facilitate quality improvements in underserved communities.

INTRODUCTION

Problem Description

It has been well documented that cardiometabolic control, including hemoglobin A1C, lipids, and blood pressure, can reduce morbidity and mortality in patients with diabetes.¹ However, adequate control of these clinical factors remains suboptimal in the United States, especially among socioeconomically disadvantaged racial/ethnic minority populations.

Available Knowledge

Most routine diabetes management takes place in primary care, with occasional specialist consultation.² However, primary care clinicians face multiple challenges in managing the complexity of diabetes care. They often struggle to meet evolving treatment targets within limited time and resources and may lack confidence in their knowledge and skills for specific tasks. It can also be difficult to keep up with the use of newer and rapidly evolving diabetes medications and technology. The lack of care coordination among primary care physicians, allied healthcare providers, and subspecialists also often result in gaps in care.

Rationale

The Project ECHO (Extension for Community Healthcare Outcomes) model, developed at the University of New Mexico, employs a hub and spoke design. Experts at a hub provide tele-education learning opportunities and real-time consultative support on specific topics to primary care providers in a group of spoke community clinics. The ECHO model has shown promising results in increasing provider knowledge, confidence, and patient outcomes in a number of areas, including diabetes.^3–6 However, to our knowledge, the ECHO model has not been leveraged for diabetes quality improvement (QI) initiatives. Evidence has shown that many QI strategies targeting health systems, healthcare providers, and patients have helped to improve the efficacy of diabetes care and the health outcomes of patients with diabetes.⁷ Meta-analyses reported that QI implementation facilitates a 0.3% reduction in mean HbA1c, which may translate to 7% fewer deaths at the population level.^3,8–10 As implementing QI initiatives under the ECHO model provides a less burdensome, inexpensive, and individualized approach for each clinic to achieve quality care based on their needs and priorities, we hypothesized that QI under an ECHO model can be beneficial and scalable for improving care quality at clinics that serve resource-limited patient populations.

Specific Aims

We aimed to add a 12-month QI component to the Project ECHO Diabetes program conducted by the University of Florida College of Medicine and the Stanford University School of Medicine.^11–16

METHODS

Context

In underserved communities, many primary clinics do not have in-house endocrinologists and diabetes specialty team members to consult. Through Project ECHO, these clinics will have easy access to specialty consultation and expert guidance in QI to improve patient care.

Intervention(s)

Thirteen health centers ( spokes) in California and Florida providing care for medically underserved communities were strategically recruited from rural and urban areas for Project ECHO Diabetes (Appendix Table 1).¹¹ Targeted clinics were mostly Federally Qualified Health Centers (FQHCs) or community health centers located in low endocrinology provider density and high health risk/poverty areas.¹² Participants in Project ECHO Diabetes received weekly tele-education including a didactic presentation and case-based discussion through the use of Zoom (Zoom Video Communications, San Jose, California, USA), forming a community of practice. These spoke clinics had real-time access to support from multidisciplinary hub teams based at the University of Florida College of Medicine and the Stanford University School of Medicine (endocrinologists, behavioral health specialists, dietitians, and the like), a patient Diabetes Support Coach, and an online repository of recorded tele-education ECHO clinics, among other diabetes resources for primary care providers.¹²

Appendix Table 1:

Cohorts 1 and 2 Spoke Clinics Participating in Project ECHO Diabetes

Clinic Zip Code	Setting	Type of Practice	# of Providers	# of Patients with Diabetes
33756	Metro Area	FQHC	9	3830
33142	Urban City	FQHC	9	1123
33139	Urban City	FQHC	27	3850
32610	Metro Area	Specialty Clinic	10	1100
32680	HRSA Rural Defined Area	FQHC Look Alike	1	155
95350	Urban	Private Practice	1	540
95501	Rural	Independent Physician Association	1	74
95076	Urban	FQHC	2	2,475
95437	Rural	FQHC	1	693
95501	Rural	FQHC	3	271
94590	Urban	FQHC	8	2812
95521	Rural	FQHC	2	892
95901	Rural	FQHC	1	586

To supplement provider tele-mentoring and explicitly address diabetes disparities on the systems level, we adopted the Institute for Healthcare Improvement’s model of improvement (See Figure 1) for a 12-month QI program taking place within Project ECHO Diabetes. (The initiatives ran from September 2020 through August 2021. Accordingly, the first quarter covers September to November 2020, the second quarter December 2020 to February 2021, the third quarter March 2021 through May 2021, and the fourth quarter June 2021 to August 2021.) Thirteen spoke clinics participated in the QI program within Project ECHO Diabetes. Among those who reported on their previous diabetes-related QI work, two reported that they had no previous diabetes-related QI work; seven reported that the Project ECHO Diabetes program supplemented their ongoing work, and two reported that it replaced their previous diabetes-related QI work. In the first 6 months, the hub team and the spokes met weekly; in the second half of the 12 months, the sessions took place every other week. Every session is structured with didactics, case-sharing, and best practice sharing. Under Project ECHO, QI sessions took place quarterly, led by experts from the hub team (diabetes care team specialists and QI experts) who engage with Diabetes Champions (mostly physicians, physician assistants, and nurse practitioners) from the clinics and other clinic staffs interested in QI. The QI sessions were structured similarly to the regular Project ECHO sessions with a focus on QI didactics and cases. The schedule of the didactics and the clinics’ ratings of the didactic topics are provided in Appendix Table 2.

Figure 1.

Shown here is the project driver diagram.

Appendix Table 2:

ECHO Diabetes Session Topics

Date	Didactic Topic	Rating - % of Respondents Making a Commitment to Change Practice
09.17.2020	*Introduction to Quality Improvement Science in ECHO Diabetes	80%
10.01.2020	Provider-Patient Relations and Behavior Change	83%
10.15.2020	Patient Activation	67%
11.05.2020	Patient Panel Addressing Implicit Bias	100%
11.19.2020	How to Teach Healthy Cooking in Your Practice	86%
12.3.2020	Eating Healthy on the Dime	92%
12.17.2020	*Siouxland Community Health Center’s Approach to Quality Improvement	89%
01.07.2021	Brand Versus Generic Analog and Follow On Insulins	82%
01.21.2021	Concentrated Insulin	80%
02.04.2021	Teaching Patients How to Read Their CGMs	90%
02.18.2021	Revisions to the 2021 Abridged Diabetes Standards of Care for PCPs	100%
03.04.2021	Getting the Most Out of Insulin Pump Hybrid Closed Loop Technology	85%
03.18.2021	*Updates to Diabetes QI Initiative in ECHO Diabetes	100%
04.01.2021	How to Interpret and Utilize CGM Trend Arrows	89%
04.15.2021	Tips for Obtaining Transformative Diabetes Devices in Medicare Populations	91%
05.06.2021	Tackling Insurance as a Social Determinant of Health	100%
05.20.2021	Focus Group	N/A - focus group not eligible for CME credit
06.10.2021	The Pillars of Success: Knowledge, Community and Resilience	100%
06.17.2021	*Glycemic Targets and Glucose Monitoring/Quality Improvement and Service Redesign	100%
06.24.2021	Continuous Glucose Monitoring	79%
07.01.2021	T2D Management in Established Atherosclerotic Cardiovascular Disease	82%
07.08.2021	T2D Management in Heart Failure and Chronic Kidney Disease	68%
07.15.2021	Types of Analog Insulin	91%
07.22.2021	T2D Management: Promoting Weight Loss	87%
07.29.2021	T2D Management: When Medication Cost Becomes a Barrier	74%
08.05.2021	Using and Interpreting Data from CGMs	74%
08.12.2021	Initiating Insulins and Dose Calculations for T1D and T2D	68%
08.19.2021	Diabetes and Hypertension Management	90%
08.26.2021	Calculating Insulin Doses for T1D and T2D: A Case Based Approach	60%
09.02.2021	Initiating Insulin Pump Therapy	87%
09.09.2021	Introducing Technology to Patients	95%
09.16.2021	*Screening for Depression and Diabetes Burnout/Updates on Quality Improvement Initiatives	83%

^*Quality improvement (QI sessions)

We followed the steps of the UCLA/RAND Modified Delphi Method in developing the quality measures.^17,18 During the QI sessions, didactic lectures were provided on general QI methodologies. Using a modified-Delphi approach, the hub team and spoke site representatives (Diabetes Champions, sometimes also including QI/data experts) first reviewed commonly used evidence-based quality indicators for diabetes care. The spoke site representatives then rated these quality measures independently (first round) on validity and feasibility on a scale of 1 to 9, where 9 is the best. Validity assesses if the quality measure will likely make a difference in outcomes based on reading of the scientific evidence; feasibility assesses how easy it is to collect data on the quality measure. Those with median ratings below 7 were discussed in a Project ECHO quality improvement video session to clarify, revise, or remove the indicators. The spoke site representatives then re-rated the quality indicators (second round) on validity and feasibility. The difference in the modified Delphi method compared to the traditional Delphi method of surveying experts is in the panel discussion between two rounds of ratings. The discussion allows modifications to quality measures with low ratings or disagreements in ratings resulted from the wording of the measure.

We also added importance ratings in the second round using a 9-point scale; importance assesses how high in priority the quality measure is to the clinics and their patient population. Only those with median ratings of 7 or greater for validity and 4 or greater for feasibility were accepted.^19–22 Importance ratings were used only as a guide for spoke sites to select accepted measures to work on. A typical site would work on two to three measures for quality improvement based on their patient population needs. Each clinic’s work to improve a specific quality measure was called a QI initiative. Quarterly, clinics met remotely to report performance on their QI initiatives and discuss their efforts, successes, and setbacks.

Study of the Intervention(s)

For the QI initiatives, we collected both quantitative and qualitative data from the clinics’ staffs every quarter. Quantitative data include numbers and percentages of patients achieving certain health goals; qualitative data include the challenges and improvements observed and documented by clinic staffs.

To measure the resources used in the QI program by participating ECHO Diabetes spoke clinics, we collected quarterly data on the number of different personnel (physicians, nurse practitioners and/or physician assistants, certified diabetes care and education specialists and/or registered nurses, administrative staff, and others) on each QI team and the time typically spent per month on ECHO QI activities by the different team members.

Measures and Analysis

With the data, we defined progress with the following categories: (1) Improvement: fourth quarter shows improvement (quantitative or qualitative) compared to the first quarter, and either second or third quarter shows improvement compared to the first quarter. Improvements in either the second or third quarters were counted because there were some fluctuations in the number of patient visits and testing accomplished due to surges of COVID-19; (2) In Progress: there was no measurable improvement, but the clinic’s description of its efforts suggested that they were trying different approaches to address the problem; and (3) No Measurable Progress: there was no measurable improvement, and the clinic’s description of their efforts suggested that they were not actively working on the problem. The spokes directly reported their progress status using the REDCap data capture tool. Study data were collected and managed using REDCap tools hosted at the Stanford University School of Medicine.^23,24

The monthly cost of QI activities for each clinic is calculated by multiplying the national wage data for each type of personnel with the number of personnel and the hours each type of personnel spent in ECHO QI activities in a typical month and adding up these costs across all personnel categories. These data were collected for the last two quarters of the QI program and averaged to obtain the mean monthly cost for each clinic. Clinic-specific adult patient enrollment data were used to calculate mean monthly costs per adult patient. Means, medians, and ranges of the monthly data are multiplied by 12 and reported as annual estimates. With these data, we explored the relationships between mean annual costs per patient and the number of initiatives worked on, team size, and number of patients with diabetes.

Ethical Considerations

Project ECHO Diabetes requested a Human Subjects Research Determination from the Stanford University Institutional Review Board (IRB), and based on the QI activities described in the request, the IRB determined that the project did not meet the definition of research as defined in the 45 CFR 46.102(d), nor the definition of clinical investigation as defined in 21 CFR 50.3(c). Therefore, a full panel review was not required for this project (protocol number 58484), and it was approved as a QI activity.

RESULTS

A total of 18 quality measures were selected based on a median validity rating of 7 or greater and feasibility rating of 4 or greater and without significant disagreement in the ratings.^22,25,26 Each clinic used its chosen set of quality measures to track the progress of its QI initiatives over the one-year QI program. The number of initiatives each clinic focused on ranged from 1 to 9, with a median of 3. Altogether there were 31 QI initiatives. The number of clinics working on each quality measure are shown in Table 1; for example, 4 clinics chose to work on quality measure 1. At the end of the one-year QI program, clinics reported widespread improvements across structural, process, and outcome measures (Table 1). Across all improvement initiatives, 25 out of 31 (80.6%) showed improvements, and 6 out of 31 (19.4%) were in progress. There were no initiatives that showed no progress. Specifically, 8 out of 8 (100.0%) initiatives addressing structural measures showed improvements; 13 out of 18 (72.2%) initiatives that worked on process measures showed improvements; 4 out of 5 (80.0%) initiatives showed improvements in outcome measures.

Table 1.

Quality Measures Chosen, Rated, Implemented by Spoke Sites

Quality measure chosen	Validity rating	Feasibility rating	Importance rating	Number of clinics that worked on the measure	Number of clinics with “improvement”	Number of clinics “in progress”
Structural measures
1. Clinic has protocol for identifying “high risk” patients and enrolling them in a special pathway	7	8	8.5	4	4	0
2. Clinic has a sick day management plan	7	8	8	1	1	0
3. Clinic has a workflow for scheduling “lost to follow-up patients”	7	8	8	2	2	0
4. Clinic has an HbA1c point of care machine	9	9	9	1	1	0
Process measures
5. Number of patients who had encounters with an ECHO Diabetes support coach in the last quarter	6	7	7	1	1	0
6. Percentage of patients with HbA1c measurement in current year	9	9	9	5	2	3
7. Percentage of patients with diabetes whose last HbA1c was greater than 9% who had a visit (clinic, phone, video) in past 6 months	8	8	9	1	1	0
8. Number of patients with diabetes whose last HbA1c was greater than 9% who had 3 out of 5 touch points (PCP, Endo, BH, nutrition, peer coach) in the current measurement year	n/a*	n/a*	n/a*	1	1	0
9. Identify and contact patients who have not had HbA1c measurement in the past calendar year, whose last HbA1c was < 9%	8	8	7	1	1	0
10. Identify and contact patients who have not had HbA1c measurement in the past calendar year, whose last HbA1c was > 9%	8	8	9	3	3	0
11. Identify and contact patients who have not had HbA1c measurement in the past calendar year, regardless of the last HbA1c value	8	8	8	1	1	0
12. Number of Continuous Glucose Monitor (CGM) prescriptions submitted during the last quarter	6.5	7	7	2	2	0
13. Percentage of telehealth (phone and video) visits for patients with diabetes during the last quarter	8	7.5	7	1	0	1
14. Percentage of patients who had a visit (in-person, phone, or video) in the current year who did not get an HbA1c	n/a*	n/a*	n/a*	1	1	0
15. Percentage of patients with diabetes for whom microalbuminuria tests have been ordered in the current year	n/a*	n/a*	n/a*	1	0	1
Outcome measures
16. Percentage of patients with diabetes, whose most recent HbA1c > 9%	8	8	9	3	2	1
17. Mean HbA1c of patients with diabetes of most recent HbA1c for the current measurement year. Commercial Insurance	7	7	7	1	1	0
18. Mean HbA1c of patients with diabetes of most recent HbA1c for the current measurement year. Medicaid	7	7	7	1	1	0

(1) Improvement: fourth quarter (August 2021) shows improvement (quantitative or qualitative) compared to the first quarter (November 2020), and either second (February 2021) or third quarter (May 2021) shows improvement compared to the first quarter. Improvements in either the second or third quarter were counted because there were some fluctuations in the number of patient visits and testing accomplished due to surges of COVID-19. (2) In Progress: there was no measurable improvement, but the clinic’s description of its efforts suggested that they were trying different approaches to the problem.

Across all improvement initiatives, 25 out of 31 (80.6%) showed improvements, and 6 out of 31 (19.4%) were in progress. There were no initiatives that showed no progress.

*Measures 8, 14, and 15 are measures that individual spokes are interested in working on in their clinics. They did not go through the modified Delphi approach and were added after the panel ratings.

PCP, primary care provider; BH, behavioral health.

As an example, Figure 2 presents data on one quality measure, the cumulative percentage of patients with an HbAlc measurement in the current calendar year (process measure #6). There is a drop in the second bar because it is the start of a new calendar year. Although we only have data through August 2021, clear improvements were made in 2 out of 5 clinics (Clinics 1 and 3 in Figure 2) because their cumulative rates in August 2021 already exceeded their rates in November 2020. We assumed that the cumulative rates in August 2021 should have been similar to August 2020, and less than achieved by November 2020, if there were no interventions. The other 3 show progress with rates rising consistently during 2021, moving very close to their rate in November 2020.

Figure 2.

The graph shows cumulative percentage by calendar year of patients with HbA1c measurement results from November 2020 to August 2021 (quality measure 6).

Some QI initiatives were labeled “in progress” because we collected data across two calendar years and it was not yet possible to determine whether improvements had been made. For example, in the three initiatives that were in progress addressing measure #6, percentage of patients with HbA1c measurement in current year, the percentages in November 2020, February 2021, May 2021, August 2021, were 89.0%, 32.0%, 79.0%, and 87.0%, respectively, in one clinic; 92.3%, 28.6%, 63.8%, and 85.0%, respectively, in another clinic; and 82.0%, 43.5%, 67.0%, and 77.0%, respectively, in the third clinic. If we observed the QI initiatives for another quarter, it is possible that the percentages in November 2021 would have exceeded those in November 2020. This could also be the case in the initiative addressing measure #15, percentage of patients with diabetes for whom microalbuminuria tests have been ordered in the current year, whose percentages in November 2020, February 2021, May 2021, and August 2021 were 76.0%, 50.4%, 74.3%, and 75.4%, respectively. Moreover, with measure #13 percentage of telehealth (phone and video) visits for patients with diabetes during the last quarter, we observed that the percentage reported in February 2021 (quarterly data including December 2020, January 2021, and February 2021) was 1.5 to 2 times higher than the other quarters, which may reflect the COVID-19 surge in the United States during that time.

The quarterly tele-ECHO QI sessions fostered the sharing of best practices and novel ideas among the clinics (Table 2), the implementation of which helped reach the improvements in Table 1. Strategies included utilizing technology to identify and send reminders to patients in need of HbA1c measurement and advanced care, setting up clinic workflow to ensure HbA1c measurement, increasing appointment frequency, providing additional educational visits for patients with high HbA1c levels; increasing accessibility and willingness of getting HbA1c measurement, and increasing continuous glucose monitoring (CGM) rates among patients by facilitating access to reimbursement.

Table 2.

QI Implementations Reported by Spoke Sites

Identify high-risk patients (QI measure #1)

• Prior to implementation: “We found HbA1c reporting issues for in house testing.” • Highlight abnormal HbA1c data in the most recent 6 months by working with the EHR vendor. • Introduce a data-mining software that helps identify individuals whose most recent HbA1c was above 9%.

Identify patients in need of HbA1c measurement (QI measures #6–11)

• Designate care coordinators to identify patients who have not had an HbA1c within the last 12 months from the patient-centered registries. • Initiate in the workflow that HbA1c measurement information needs to be reviewed prior to patients being seen, and that the medical assistants and wellness coaches should regularly audit patient charts to ensure that patients have had an HbA1c measurement when needed. • Start monthly outreach to patients who require HbA1c measurement by telephone, text, letter, and patient portal.

Identify lost-to-follow-up patients (QI measure #3)

• Prior to implementation: “We have an existing workflow, but it was not efficient due to staff turnovers and workflow changes.” • Incorporate a new software that identifies lost-to-follow-up patients and sends them a message directly to schedule an appointment.

• Identify lost-to-follow-up patients in the past 1 year (previously only within 3 months).

Improve care quality for high-risk patients (QI measure #10)

• Shorten follow-up intervals for high-risk patients: patients with HbA1c > 9% have follow-ups monthly, patients with HbA1c between 8% and 9% have follow-ups every two months, patients with HbA1c between 7% and 8% are seen every 3 months. • Introduce software that sends emails to schedule high risk patients for visits. • Provide additional educational visits and follow-ups with RN, psychologists, and social workers for patients with HbA1c > 9% or who had recurrent DKA events. Develop nurse visit workflows and protocols to support the RN diabetes visit and prepare nurses with training. • Hire a new peer coach to develop referral and tracking process.

Improve HbA1c measurement rates (QI measure #11)

• Prior to implementation: “We appear to have had a drop in HbA1c measurement rates due to visit changes during the COVID pandemic.” • Develop routine order sets to ensure HbA1c measurement for patients with inpatient and virtual appointments. • Create standing order sets to ensure patients who were not measured for HbA1c in the past 3 months get Point-of-Care HbA1c measurement. • Provide curbside HbA1c measurement. • Implement all CDC recommended precautions for COVID-19 to increase patients’ willingness to come for measurement. • Provide HbA1c home measurement kits and educate patients with administration methods in the clinic.

Increase usage of continuous glucose monitoring (CGM) (QI measure #12)

• Prior to implementation: “We are surprised not to find any CGMs ordered in our EHR.” • Document the process for preparing and getting prior authorizations for patients who need CGM and pumps. • Increase partnership with suppliers to get more patients through the process and reimbursement from their insurance companies. • Change clinical process of ordering CGM through EHR instead of contacting suppliers directly to make sure all orders go through. • Obtain some freely available CGM in the clinic. • Connect patients who wanted CGM access with CGM donations. Secure additional funding for CGM donations. • Engage clinical pharmacists in performing phone calls on CGM training.

QI, quality improvement; HER, electronic health record; RN, registered nurse; CDC, Centers for Disease Control and Prevention.

Table 3 shows cost estimates at the spoke clinics for their Project ECHO Diabetes QI initiatives. There was wide variation among the spoke clinics in the number of initiatives, the composition of and number of hours worked by their QI teams, and the number of patients with diabetes. As a result, costs varied widely. There was no association between the number of QI initiatives attempted and total cost or cost per patient. Cost per patient was negatively correlated with the number of patients, although the correlation coefficient (−0.505) was not significant (p = 0.1).

Table 3.

Estimates of Resources Used in the ECHO Diabetes 12-Month QI Program

	Mean	Median	Range
QI team size	8.2 members	6.8 members	4–22.5 members
Cost per Spoke	$ 31,476	$ 20,064	$9,300–$94,176
Cost per adult patient	$ 71	$ 45	$6–$629

QI, quality improvement.

DISCUSSION

Summary

The 12-month QI program within Project ECHO Diabetes resulted in clinics making improvements in structural, process, and outcome measures. With collective knowledge from primary care clinics, professionals specialized in diabetes care, and experts in innovative and practical QI, strategies were developed and implemented in clinics serving under-resourced communities. To our knowledge, this is the first demonstration of how the ECHO model can be leveraged for diabetes QI by aligning with the local priorities of community health centers. Implementing QI initiatives under the ECHO model provides a less burdensome, inexpensive, and individualized approach for each clinic to achieve quality care based on their needs and priorities, while allowing them to learn from other clinics at the same time. Incorporating QI initiatives into the ECHO model widely among primary clinics that care for diabetic patients may optimize disease outcome. In addition, this model of carrying out QI projects through tele-mentoring models can be used in the management of other chronic diseases to improve community healthcare outcomes collectively.

Interpretation

While working on their QI initiatives, many clinics were able to discover shortcomings in their current workflows. The providers actively solved problems, and the quarterly peer clinics and experts at the hub helped to develop strategies to address obstacles in implementation. Clinics introduced new software applications and worked with their vendors to enable more accurate reporting, expanded partnerships with CGM suppliers, and provided more patient education when rates of CGM uptake among patients increased, redesigned clinic guidelines and workflows, and adopted telehealth appointments and drive-through HbA1c measurement for patients who did not want to enter the clinics during the pandemic. Nevertheless, it can be difficult to keep up with the use of newer and rapidly evolving diabetes medications and technology. Clinics encountered some barriers when implementing the initiatives, such as not being able to carry out a written plan due to staff changes, workforce shortage when more high-risk patients were identified and needed referrals, patients not coming to regular visits despite frequent reminders from the clinic, and the challenges of increasing the improvement rate for patients opting to use continuous glucose monitors after all interested patients had received the devices. The supportive environment through the Project ECHO Diabetes QI program when challenges were encountered appeared to facilitate the high rate of improvement among clinics in the program.

There was wide variation among the clinics in the number and type of personnel involved in their QI initiatives, the staff time devoted to their QI initiatives, the total cost, and cost per patient of their QI initiatives. The cost per patient seems to be negatively correlated with clinic size (number of patients with diabetes); although this correlation did not reach statistical significance, it suggests the possibility of achieving economies of scale. These findings are consistent with Sathe et al.’s QI collaborative work that showed that the clinic with the lowest annual per-patient costs had the largest diabetes patient population, while the clinic with the highest per-patient costs had the smallest diabetes patient population.²⁷ Although our sample size was relatively small, the consistency of our findings with those of Sathe et al. may be of interest to policymakers and others who are looking to initiate QI programs focused on diabetes care in primary care clinics. The largely unexplained wide variability in costs suggests that further study is needed to help right size the resources needed for QI collaboratives.

Limitations

It is important to note that the clinics in this study were all self-selected into Project ECHO Diabetes and into the QI program and that they selected the quality measures to work on. We did not ask about the criteria they used to select their quality measures for their QI initiatives. For example, were quality measures selected based on highest priority, ease of implementation, and/or in the areas they had made improvements previously? We did not examine how differences in patient characteristics, such as age, race/ethnicity, insurance, and health status may have affected the initiatives chosen, costs, and rates of success. Also, the QI program took place from September 2020 to August 2021 during the COVID-19 pandemic, which affected the work of the clinics in many ways not quantified in this study. For example, essential lab testing and diabetes outreach activities were paused at the beginning of the pandemic, and patients were hesitant to attend in-person clinic appointments. Since the one-year program was carried out from September 2020 to August 2021, it was also difficult to determine the outcomes of cumulative annual measures such as quality measure 6 (Figure 2), the percentage of patients with an HbA1c measurement in the current year. While this quality measure was the most selected across spoke sites, there were 17 other quality measures selected based on local needs and priorities. Last, the estimates of time committed are self-reported by clinical providers, which might introduce potential bias (e.g., recall bias, social desirability to represent that time is being spent on QI initiatives).

CONCLUSION

Our finding that the clinics made substantial progress on a variety of quality measures supports the inclusion of similar collaborative QI initiatives in subsequent Project ECHO Diabetes programs, as well as Project ECHO programs focused on other health issues.