Determinants, implementation strategies, and mechanisms of the 4T Program

Abstract

The Teamwork, Targets, and Technology for Tight Glycemia (4T) Program has demonstrated effectiveness in early continuous glucose monitor use with remote patient monitoring among three diverse cohorts of youth with new onset type 1 diabetes (T1D, n = 451). Based on promising single center experience, we examine the potential for program scaling to the T1D Exchange Quality Improvement (T1DX-QI) Collaborative using an implementation science approach.

We used the Simplified Implementation Logic Model to identify contextual determinants of implementation success. Then, we described the connections between the implementation support strategies used in 4T to address the contextual determinants and barriers to program reach to youth with T1D and provider adoption. We hypothesized mechanisms linking barriers and strategies to effectiveness and implementation outcomes.

Limited clinician time was identified as a major barrier to program success. Several targeted strategies were used to address this, including the creation of the Timely Interventions for Diabetes Excellence (TIDE) clinical decision support tool early in the program and ongoing refinement of the tool. After clinical effectiveness of the 4T Program was established, the next phase focused on further allocating clinician monitoring to participants who would benefit the most. Additionally, cost to the family participants, cost to the clinic, and interest-holder engagement were other determinants addressed by the program strategies.

This implementation evaluation informed connections between determinants, support strategies, mechanisms, and outcomes of the 4T Program. This enabled a more precise design in preparation to scale the program across a national network of pediatric diabetes clinics in the T1DX-QI.

Highlights

• •Implementation science can bridge the translation gap from evicence to practice.
• •Implementation evaluation linked 4T Program determinants, strategies, and outcomes.
• •Prospective implementation planning will promote successful scaling of 4T Program.

Introduction

Over 30 years ago, the Diabetes Control and Complications Trial firmly established the benefits of intensive diabetes management to reduce vascular complications from type 1 diabetes (T1D) through team-based care and frequent insulin dose adjustments [1], [2]. Translating these research findings into routine clinical practice has been challenging. Implementation science can bridge the translation gap from evidence to practice. Unfortunately, the application of rigorous implementation science approaches to improve care for youth with T1D is sparse [3]. During the past decade, while many countries have improved glycemic outcomes parallel to increased diabetes technology use, the US lagged behind in both technology use and glycemic outcomes [4], [5], [6]. The technology use and glycemic gap is unevenly distributed in the US; families with lower socioeconomic status have lower adoption of diabetes technology and a widening gap in glycemic outcomes compared to those with higher socioeconomic status [7]. A growing shortage of pediatric endocrinologists in the US is anticipated to further negatively impact T1D care [8]. To address this scientific gap, we applied an implementation framework retrospectively to evaluate an innovative pediatric T1D management program.

In 2018, the Stanford Pediatric Diabetes Team developed and launched the Teamwork, Targets, and Technology for Tight Glycemia (4T) Study to synthesize and tailor international initiatives to improve diabetes technology access and increase rates of meeting glycemic targets [7], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]. The 4T Study was a pragmatic clinical research trial that delivered just-in-time personalized care to newly diagnosed pediatric persons with type 1 diabetes (PwT1D) by supporting early initiation and sustained use of continuous glucose monitors (CGMs), providing increased frequency of clinician-guided dose adjustments in between clinical visits, promoting development of healthy self-care habits early, and supporting family education [23]. Patient-reported outcomes (PROs) such as diabetes distress and depression, were also monitored systematically [24]. An innovative component of 4T was the use of remote patient monitoring (RPM) with the Timely Interventions for Diabetes Excellence (TIDE) platform, which is a visual dashboard that facilitates triage of 4T participants using consensus CGM metrics such as CGM wear time, time-in-range, and time-in-hypoglycemia [25], [26], [27], [28]. The diabetes care team used TIDE to target care and education touchpoints to at-risk individuals via asynchronous RPM. In youth with new-onset T1D, the 4T Program reduced hemoglobin A1c (HbA1c) by 1.1% (12.1 mmol/mol), with an odds ratio of 4.5 for meeting target HbA1c <7% (53 mmol/mol) at 1-year post-T1D diagnosis compared to historic controls [37]. The 4T care model was implemented as standard care at Stanford Pediatric Diabetes Clinic in 2024 (New Onset Program).

Other centers in the US have partnered to adopt the 4T Program via a National Institutes of Health (NIH) and National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) U34/U01 funding mechanism, which is a structured two-part process to support high-risk multi-center clinical studies [29], [30]. The U34 phase involves implementation planning and the U01 phase is a cooperative agreement to conduct the clinical study. Specifically, a collaboration between the 4T Program and the T1D Exchange Quality Improvement Collaborative (T1DX-QI) has been proposed to disseminate equitable improvements in new-onset care and address existing care gaps for youth with T1D in the US. T1DX-QI is a quality improvement network of 63 diabetes clinics across 23 states established in 2016 [31], [32], [33]. We hypothesized that successful dissemination and implementation of the 4T Program across T1DX-QI centers is more likely when implementation research tools are applied to clearly define the relationship between determinants of implementation, implementation strategies, the mechanism of action resulting from the strategies, and the outcomes. The objective of this study is to evaluate the 4T implementation at a single center retrospectively. Lessons from this work will inform planning for prospective implementation evaluation and promote more precise program design in preparation for scaling and disseminating the program nationally.

Methods

4T Program and intervention

This was a prospective, pragmatic, single-arm, open-label research study conducted at the Stanford Pediatric Diabetes Clinic. The Pilot 4T Study recruited youth with T1D diagnosed between July 2018 and June 2020. Inclusion criteria included youth with T1D diagnosed within the preceding 30 days, willingness to wear CGM, share CGM data, and receive clinical care at our clinic. Participants started CGM during first month of diabetes diagnosis with glycemic target set at HbA1c <7.5% (58 mmol/mol) based on the 2018 American Diabetes Association guidelines (at time of study initiation) [34]. Additionally, RPM consisting of weekly CGM data review by Certified Diabetes Care and Education Specialists (CDCES) started in May 2019 when the TIDE dashboard was developed. Pilot learnings informed the design for Study 1, which recruited youth with T1D diagnosis between June 2020 to March 2022. Since pilot study participants receiving RPM had lower HbA1c compared to those without RPM, Study 1 was conducted as a single-arm trial with all participants offered CGM initiation within 1 month and weekly RPM. Study 2 recruited youth with T1D diagnosis between March 2022 to May 2024. Study 2 had a stepped-down cadence for RPM consisting of weekly reviews for the initial 3 months followed by monthly reviews until study completion, which translates to a 60% reduction of RPM reviews from 52 to 21 during the first year after diagnosis. Study 2 participants were encouraged to participate in early education on automated insulin delivery (AID) systems (Fig. 1). The studies were approved by the Stanford IRB; informed consent (and assent for participants aged 7–18 years) was obtained from all participants. The detailed 4T Program protocol has been described previously [35], [36], [37].

Fig. 1.

Overview of 4T program timeline. 4T Study started as a pragmatic clinical study in 2018, with the goal of promoting early CGM initiation through RPM to increase the rates of youth with newly diagnosed type 1 diabetes meeting glycemic targets. The protocol was refined in 2020 to lower the HbA1c target to <7% in alignment with updated guidelines. In 2022, protocol was revised with reduced cadence for RPM to evaluate whether a less resource-intensive protocol may still have significant clinical benefits. During this time, early AID education within 3 months of diabetes diagnosis was added, due to increasing evidence for clinical benefits associated with AID use. In 2024, the 4T Program transitioned from research to standard clinical care, using the same protocol as 4T Study 2. In 2025, a workshop was held to plan for 4T Program dissemination to T1DX centers across the US. Subsequently, an NIH U34 Proposal was submitted to further the preparation for scaling 4T. Key implementation strategies during each phase of 4T are noted in red. Abbreviations: Teamwork, Targets, and Technology for Tight Glycemia (4T), automated insulin delivery (AID), continuous glucose monitoring (CGM); hemoglobin A1c (HbA1c); National Institutes of Health (NIH); remote patient monitoring (RPM). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

4T implementation team and key interest-holders

The 4T research team included clinicians, translational scientists, biostatisticians, engineers, and research coordinators. The clinical care team included CDCES, pediatric endocrinologists, psychologists, and fellows. Additional institutional-level interest-holders include clinical informatics, billing, and compliance teams.

Implementation evaluation for 4T Program

The Simplified Implementation Logic Model (ILM), which is an adapted, pragmatic version of the Implementation Research Logic Model, was applied retrospectively to evaluate the 4T Program to describe the relationship between determinants of implementation, implementation strategies, mechanisms, and outcomes [38], [39]. A comprehensive list of implementation determinants was identified on levels of the: 1) intervention characteristics, 2) inner context, 3) outer context, 4) characteristics of the individuals (PwT1D/caregivers), and 5) process. These were organized using the Consolidated Framework for Implementation Research (CFIR) [40]. Implementation strategies from the four iterations of the 4T Program were reviewed for alignment with the identified determinants. Program outcomes were re-examined using the RE-AIM framework, which include 5 key components: Reach, Effectiveness, Adoption, Implementation, and Maintenance [41].

4T planning session with T1DX-QI centers

A 2-day workshop was held in January 2025 to plan for 4T scaling across T1DX-QI centers via the U34/U01 proposal. The workshop covered 3 themes: (1) overview of the 4T Program, including key features, outcomes, and learnings from the implementation at Stanford, (2) identifying differences and similarities in contextual determinants at the T1DX-QI centers, and (3) discussing potential targeted strategies to advance program implementation across the T1DX. The session format included expert presentations, panel discussions, questions and answers with the audience, and small group discussions. Most sessions were led by investigators and clinicians from Stanford 4T Team and the T1DX-QI. Additionally, representatives from Tidepool demonstrated key features of the system for CGM data visualization and interface accessibility. Representatives from a diabetes clinic in Denmark with a robust RPM program provided international perspectives on T1D care and the NIDDK Program Director provided overview and guidance on applying for the U34/U01 funding. Fifteen physician and CDCES dyads from other centers participated in the interactive workshop and shared their perspectives on potentially implementing the 4T program. Detailed workshop meeting notes included a summary of the discussion, questions, feedback, and insights from workshop participants. The notes were reviewed by study investigator (MYL) and organized using the ILM.

Results

Alignment between contextual determinants, implementation strategies, and proposed mechanisms

Contextual determinants for implementation were organized using CFIR according to the levels of intervention, inner context, outer context, individual, and process (Table 1). For each determinant, we identified the implementation strategies used to address it, and the proposed mechanism by which the strategy influenced program outcomes (Table 2). Three determinants were identified as the most critical factors influencing program success: clinical staffing capacity, program cost, and interest-holder engagement. These determinants and the associated support strategies spanned across multiple CFIR levels as described below.

Table 1.

Evaluation of 4T Program determinants from 2018 to 2025.

Level	Determinants	4T Pilot Study 7/2018–6/2020	4T Study 1 6/2020–3/2022	4T Study 2 3/2022–5/2024	New Onset Program 5/2024-current
Intervention	Evidence strength and quality	•Evidence base supporting use of CGM as 1st line option for PwT1D, including during the new onset period, increased over time.
	Cost to clinic	•RPM infrastructure and clinician time supported by research funding.			•Billing to recuperate operational costs for RPM infrastructure and clinician time.
	Cost to families	•No additional financial costs for families to participate in RPM. •Supplies for RPM (including CGMs and iOS devices for data transmission) were provided.	•As of 2022, California Children's Services expanded CGM coverage for youth with T1D by removing the requirement for demonstrating ≥4 blood glucose checks daily for a month.		•Subset of participants received co-payment charges for RPM as a clinical service. •Philanthropic funds and financial counseling were available to support families with RPM co-payments.
Inner context	Cultural, structural, and networks	•Stanford Pediatric Diabetes Clinic benefitted from strong culture of supporting innovation and diabetes technology use. •Collaboration with engineering school to develop TIDE tool to support scaling of RPM. •Clinic connection to philanthropic funds through the hospital.			•Integration of TIDE in Tidepool for turnkey RPM platform
	Implementation climate and readiness	•Pilot phase to identify and develop supports for implementation.	•Supports in place from Pilot Study. •High levels of engagement from leadership and champions within clinical team.
Outer context	Peer pressure	•Stanford Pediatric Diabetes Clinic is part of T1DX-QI, which is a national diabetes registry. Benchmark reporting of diabetes technology use and glycemic metrics serve as a source of peer pressure to promote improvement.
Individuals (PwT1D and caregivers)	Knowledge/beliefs about CGM	•CGM use is increasingly common and accepted as standard care over time. •Families with high health and technology literacy tend to be earlier adopters of CGMs. •Families from marginalized backgrounds may have more misconceptions about the technology and research.
	Program engagement	•Decreasing engagement with RPM with longer duration of T1D diagnosis.
Process	Interest-holder engagement	•Engaging interest-holder early and with interval reflection and evaluation. •Regular meetings between 4T research team and clinical team in planning, execution, and evaluation.
	Planning	•SDRC P&F project provided resources to pilot and plan for 4T.	•Pragmatic trial design allowed equitable access of the program to all youth with new onset T1D without randomization to control group.		•4T study design using existing clinical staff facilitated transition to clinical program as standard care.
	Executing	•Supported by highly competent clinical pediatric diabetes team. •Limitations in clinical staffing. •Turnover of clinical team (mainly CDCES and fellows) requires ongoing training.

Table 2.

List of support strategies used in the 4T Program, matched to the contextual determinants and proposed mechanisms.

Support strategy	Determinant(s) it addresses	Description of proposed mechanisms	4T Pilot	Study 1	Study 2	New Onset Program
Provision of CGM and iOS devices	Program cost	Strategy removed device supply costs to families as potential barrier to participating in RPM, especially for families from lower socioeconomic backgrounds.	x	x	x	x
Regular 4T team meetings	Interest-holder engagement, program planning, program execution	Strategy encouraged ongoing open discussions between the 4T research team and clinical team, which promoted interest-holder engagement and provided structure for program planning and execution.	x	x	x	x
Adding CGM follow up appointment 1 week after CGM initiation	Limited clinical staffing	The number of urgent pages to the endocrinologist for CGM support during new onset period was decreased by the addition of scheduled clinic/telehealth visit with CDCES/NP to provide additional CGM education. This addressed limited clinical staffing of on-call endocrinologists.	x	x	x	x
Developing an algorithm-based clinical support tool (TIDE)	Limited clinical staffing	Facilitate up-scaling workflow to support whole clinic population using limited clinical staffing.	x
Iterative improvements in TIDE interface	Interest-holder engagement, limited clinical staffing	Re-designing TIDE interface with CDCES feedback provided more interest-holder engagement and increased efficiency in RPM execution, which addressed limited clinical staffing. Integration of TIDE into Tidepool for turnkey RPM platform to promote scaling.		x	x	x
Reduce frequency of RPM	Limited clinical staffing	Between Study 1 and Study 2, the RPM frequency decreased from once weekly for a year to once weekly for the first three months followed by monthly for the remainder of the first year. This aligned with limited clinical staffing to perform RPM.			x	x
Creating standard documentation and billing practice	Program cost, interest-holder engagement	Generation of clinical revenue is intended to offset clinical costs to clinic for operating an RPM program. This support strategy was developed with multidisciplinary interest-holder input to increase buy-in from clinical team.				x
Defining criteria for exiting participants from RPM	Limited clinical staffing	Defining program exit criteria addresses the limited clinical staffing to perform RPM while maintaining equity in care				x

Clinical staffing capacity

Addressing the barrier of limited endocrinologist and CDCES staffing was a high priority, since it directly impacts the feasibility of the program. Influenced by the clinic's strong culture of innovation and expertise in using advanced diabetes technologies and clinical informatics tools to support clinical care, the TIDE tool was developed as a support strategy to increase clinical staffing capacity early on. During the 4T Pilot, the initial TIDE tool was developed in collaboration with the engineering school. It helped clinicians prioritize review of CGM data for at-risk participants and generate clinically actionable recommendations. TIDE was iteratively refined during later phases of 4T to further improve efficiency and promote program scaling. Additionally, the 4T team collaborated with Tidepool to integrate TIDE in Tidepool, which is a non-profit diabetes data management platform that already supports over 1000 clinics in the US and provides a turnkey solution for scaling TIDE to other clinics [42].

In addition to the TIDE tool, adjusting RPM review frequency was another strategy used to address clinician capacity. Starting in 4T Study 2, RPM was performed at a stepped down cadence which reduced the number of reviews and preserved clinician resources without worsening glycemic outcomes compared to 4T Study 1. Once 4T became standard clinical care, the program exit criteria for participants were continuously assessed to develop strategies to further address limited CDCES staffing to perform RPM. For example, participants not engaging with clinical care (i.e. not reading RPM messages for >3 months and not seen in clinic for >6 months) were paused from the RPM review while the clinical team attempts to establish contact with the participant to continue clinical care or help with transferring care to another center. Altogether, these strategies triaged limited CDCES resources to support participants who are most likely to benefit from the program.

During the 4T Pilot phase, an unintended consequence of early CGM initiation during new onset period was increased number in urgent pages to the on-call endocrinologist for CGM-related support. This was identified as a critical barrier to address, as it could potentially limit the endocrinologist's ability to manage other more urgent clinical tasks. To address this, an additional CGM follow up visit with the CDCES/NP team was added to the workflow to provide additional CGM education.

Program cost

Program cost was identified as another key determinant. The cost-effectiveness, financial sustainability, and equitable technology access for youth from all socioeconomic statuses were all important considerations for the program's success as a population-health management program.

Determinants related to program cost spanned across CFIR levels and shifted during the phases of the 4T Program. During the research phases (4T Pilot Study, Study 1, and Study 2), the program costs for the clinic to develop and maintain the TIDE dashboard and to pay for CDCES time to conduct RPM were supported by research grant funding. When the program transitioned to standard clinical care in May 2024, clinical billing was initiated as a strategy to recuperate these operational costs. Similarly, costs to families participating in RPM were all covered by research funding during research phases. The study provided iOS devices for all families without compatible smart phone devices for RPM data transmission. During the 4T Pilot Study, iOS device supplies were not tracked systematically. For the 316 participants in the 4T Study 1 and Study 2, 162 iOS devices were provided. Patients' CGM supplies for RPM were provided clinically through insurance and any supply gaps were supplemented by philanthropic funds. The connection to philanthropic network support through the hospital is an example of program facilitator at the inner context level. As of 2022, California Children's Services expanded CGM coverage for youth with T1D by removing the requirement for demonstrating ≥4 blood glucose checks daily for a month. This increased insurance coverage for CGM devices.

During the New Onset Program, philanthropic funds continued to support iOS devices and gaps in CGM supplies. Standard practice on RPM clinical documentation and billing was co-developed with a multidisciplinary team including clinicians, regulatory, and billing specialists at the institution. Additionally, since families were billed for RPM as a clinical service, financial counseling and philanthropic funds were available to support RPM co-payments for families with financial hardship.

Interest-holder engagement

As demonstrated by the “Teamwork” in 4T, multidisciplinary interest-holder engagement in program planning and execution was highly valued. The 4T team included the clinical team, a family advisory council, engineers, hospital leadership, and billing and compliance specialists in the program development process, with early and frequent interval involvement. This process of interest-holder engagement is anticipated to promote the users' adoption, acceptability, and perceived appropriateness of the program. Overall, the program benefitted from high levels of engagement from leadership and champions within the clinical team.

CDCESs are the main clinicians performing CGM initiation and RPM. They participated in regular meetings with the 4T research team, which provided a venue for planning, troubleshooting execution, and ongoing program evaluation. The CDCES team guided the iterative improvements in the TIDE interface, RPM workflow, and billing practices to align with CDCES preferences. A subset of CDCESs served as 4T program champions early in the program, with gradual expansion to involve more CDCESs. Staff turnover posed a barrier, as new staff needed to be on-boarded and engaged in the program. Individual CDCES's knowledge and beliefs about the program intervention generally improved as CGM and RPM use became increasingly common and accepted as standard of care.

The PwT1D and family caregivers are another group of key interest-holders. RPM program engagement from families was very high initially, as demonstrated by 92% RPM messages read by participants within 7 days at the start of the program. This RPM engagement metric was 77% at 1 year [43]. High program engagement by the families represented a facilitator of program success, and the decline in engagement over time represented an opportunity for improvement.

Other determinants

Several determinants were identified as facilitators for program success. At the Intervention Level, the evidence base for using CGM for PwT1D during the new onset period increased over time based on the published data from 4T and other studies. Stanford Children's Diabetes Clinic participates in the T1DX-QI, which has a national diabetes registry and benchmark reporting of diabetes technology use and glycemic metrics. This serves as a source of peer pressure and motivation at the Outer Context Level for ongoing improvement.

Program outcomes

Effectiveness and implementation outcomes were re-examined using the RE-AIM evaluative framework (Table 3).

Table 3.

RE-AIM dimensions with outcomes from 4T Program.

RE-AIM Dimension	Outcomes
Reach	•High rates of enrollment •Equitable participant enrollment
Effectiveness	•Favorable glycemic outcomes at 1 year •No negative impact on diabetes behavioral health outcomes (PROs outcomes)
Adoption	•Workflow implementation by care teams •CDCES participation rate in RPM
Implementation	•CDCES time commitments associated with intervention
Maintenance	•Persistence of glycemic improvement •RPM billing practices

Reach

The combined 4T Studies enrolled 451 out of 511 (88%) youth with new onset T1D assessed for eligibility. A subset of screened youth did not meet inclusion criteria (n = 28). Common reasons for exclusion from the study was family's choice (n = 24) due to not wanting to participate in research, not wanting to start CGM in the specified timeframe, or preference for another CGM device that was not compatible with the study. Twenty-one participants exited the study due to transferring care to another clinic (n = 11) and participant withdrawal (n = 10). Reasons for participant withdrawal included discontinuing CGM use, switching to CGMs other than Dexcom (thus incompatible with RPM workflow at time of the study), no longer wanting to be part of the study, or joining different study. The cohort had 148 (33%) Hispanic, 151 (33%) public insurance users, and 64 (14%) non-English speakers. This suggests that the program has appropriate reach representative of clinic population.

Effectiveness

Glycemic outcomes were the primary clinical effectiveness outcomes. In Study 1, the odds ratio for meeting glycemic target at 1-year post-diagnosis was 4.50 (95% CI: 2.44, 8.30) compared to historic controls [37]. At 1 year, the population HbA1c reduced from 7.7% (61 mmol/mol) in the historic cohort to 6.6% (49 mmol/mol) in Study 1. PROs demonstrated that youth and caregivers had low stress throughout the study, and youth had increased technology acceptance over time [24]. Though highly effective in improving glycemic outcomes, the sustained weekly RPM in 4T Study 1 is resource intensive. Study 2 was designed to preserve early intensive support with a stepped down cadence for RPM three months after diagnosis (weekly during the first 3 months followed by monthly). The stepped down cadence had non-inferior glycemic outcomes compared to the weekly RPM model in Study 1 [44].

Adoption

Adoption of the 4T Program was high. Clinical providers consistently communicate with the 4T Program coordinators when eligible new onset participants are establishing care, facilitating timely enrollment. Early discussions with the care team served as education for the family to promote high and equitable adoption. A subset of clinical providers were early adopters and champions for the program. Incorporating feedback from these providers into the program likely contributed to high levels of satisfaction and increased adoption [23]. Factors influencing adoption timing are not yet defined but likely multifactorial. For example, providers with higher familiarity with CGM technology and the evidence base on diabetes technology use may be more likely to adopt the program. Providers who spend little time in diabetes clinic (providers who practice part time or practice mostly in general endocrinology) may be less likely to promote the program.

Implementation

CDCES time commitment to RPM is an important implementation outcome. With iterative improvements in the TIDE dashboard, CDCES spent less time on RPM review of CGM data and contacting patients (3.2 ± 0.20 to 1.3 ± 0.24 min per patient per week). [25] This corresponded to an estimated 147% increase in weekly clinic capacity for RPM given fixed CDCES time resources.

Maintenance

Persistence of glycemic benefits from the Pilot 4T continues up to 3 years; compared with historic control cohort, the study cohort had HbA1c reduction of 0.5% (6 mmol/mol) at year 1, 0.8% (9 mmol/mol) at year 2, and 1.0% (11 mmol/mol) at year 3 [45]. Program sustainment from a cost-perspective was supported by the standardized RPM billing practices [46]. This clinical documentation and billing workflow can generate revenue to fund RPM as a clinical practice.

Planning for dissemination

The 4T workshop with T1DX-QI centers was an important step in planning for 4T Program dissemination through the U34/U01 mechanism. Contextual determinants, support strategies, and outcome measures were discussed among centers interested in adopting the 4T Program and summarized using the ILM (Fig. 2). Program facilitators and barriers were organized at the levels of the Care Team, Organization, Leadership, and System and mapped to implementation strategies and outcome measures using colour codes. As anticipated, T1DX-QI centers reported variable perceived contextual facilitators and barriers to 4T implementation during the workshop. Key differences identified were the levels of buy-in from clinic staff and leadership, CDCES staffing availability to conduct RPM, clinics' current use of Tidepool for rapid deployment of TIDE, and established relationship with the institution's billing specialist.

Fig. 2.

Simplified implementation logic model for planning 4T dissemination. This figure summarizes the 4T dissemination planning across T1DX-QI centers using information gathered during the 4T T1DX workshop. The facilitators and barriers to implementation are listed on the left, proposed implementation strategies in the center, and proposed outcome measures are on the right. Each column is organized by the CFIR levels of Care Team Level (pink), Organizational Level (blue), Leadership Level (green), and System Level (yellow). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

At the Care Team Level, key determinants included assessing barriers and facilitators at each center using standardized scales, exploring provider confidence as an implementation output, and assessing center readiness and sustainability potential. Planned support strategies included initial and ongoing care team education and training on the 4T Program with support from T1DX and the Stanford 4T Team. Centers were encouraged to use the ILM model for program implementation planning and evaluation, guided by implementation coaches through the learning collaborative. These strategies were hypothesized to increase program alignment with provider perceptions of feasibility, acceptability, and appropriateness, and improve the providers' sense of fulfillment. Key outcome metrics included provider surveys and tracking TIDE tool use.

At the Organizational Level, key determinants were data sharing infrastructure and technical support to implement TIDE. The main planned support strategy was to leverage the T1DX-QI learning collaborative to bring together care teams across centers for peer learning and tracking, as this was a strategy that has worked well in other prior T1DX-QI projects [33], [47], [48], [49], [50]. Key outcome metrics included organizational interest-holder surveys on program feasibility, acceptability, and appropriateness of the 4T implementation, learning collaborative attendance, proportion of eligible patients enrolled (a marker of reach), the number of CDCES RPM reviews, and the number of RPM messages sent per patient.

At the Leadership Level, which included hospital administration, insurance, and industry partners, the key determinants included whether interest-holders perceived value in the program and concern about logistics of setting up clinical billing for financial sustainability. Support strategies included ongoing consultation with hospital system leaders on business practice with systematic tracking and optimizing methods to promote long-term sustainment. These strategies were anticipated to promote leadership engagement. Outcome measures included tracking whether hospital leadership and billing specialist partners are established and tracking billing metrics such as rejected claims and mean reimbursement.

At the Systems Level, key determinants included the development of a systematic approach to engage PwT1D and families and a study infrastructure to collect and analyze data. Support strategies included offering personalized care at the population level and incorporating PROs to support the “whole child”. The clinical effectiveness outcomes included hemoglobin A1c, standard CGM metrics, and patient reported outcomes. The implementation outcomes included surveys to each center on the feasibility and factors that may affect successful implementation and sustainment, reported using the RE-AIM framework.

Discussion

We applied the ILM retrospectively to document the experience of implementing the 4T Program and generated a guide for scaling the program. By sharing our experience in implementing 4T, we aim to encourage adoption of this evidence-based program to increase early CGM use and improve glycemia in youth with T1D on a population level.

Overall, this retrospective evaluation process was informative and enables a more precise design in preparation to scale the program across a national network of pediatric diabetes clinics in the T1DX-QI. This foundational implementation science approach will be instrumental to success in scaling the 4T Program to other diabetes centers.

This study is limited by the retrospective evaluation of implementation. The explanatory implementation frameworks would have been more usefully applied a priori during planning and execution phase of the 4T Study. However, the study was designed as a pragmatic trial with quality improvement framework applied throughout the study to identify areas for improvement. In scaling up 4T, the inner context, outer context, and individual level determinants are anticipated to differ across T1DX-QI centers, as we have identified during the workshop. Therefore, different and more targeted implementation strategies will need to be developed in response to each individual center's determinant factors. By prospectively applying ILM in implementation planning and ongoing program evaluation, the T1DX-QI centers will have higher success to preserve fidelity of essential elements of the 4T Program, while striking a balance with adaptation for less-essential elements to fit local contexts.

From the beginning, we have intentionally designed 4T to be adaptable and to integrate with new advances in care, such as AID technology. During 2018–2022, the program emphasized early initiation of CGMs during first month of diagnosis. As evidence supporting benefits of AID technology continue to increase, the 4T protocol was revised in March 2022 to incorporate early AID education within 3 months of diagnosis to support AID adoption. TIDE currently supports CGM-based RPM and does not yet integrate insulin pump data. The ability to add new features and incorporate different support strategies as technology advances is a strength of the 4T Program design.

We have demonstrated 4T Program's glycemic effectiveness and successfully translated it from a research program to clinical standard care at a single center. To further scale up the program impact, we have generated a plan for program dissemination and applied for the NIH U34/U01 funding. During the U34 preparation phase, T1DX-QI centers seeking to adopt the 4T Program will use the ILM prospectively to evaluate local determinants. Subsequently, the centers could use this information to map more precise implementation strategies tailored to these determinant factors. This will then prepare the study team and protocol for the subsequent Cooperative Agreement (U01) phase to execute a multi-center, randomized, stepped-wedge, clinical trial to evaluate the implementation, effectiveness, and sustainability of 4T across T1DX. Using rigorous implementation methodology to evaluate 4T implementation and effectiveness across multiple centers is anticipated to maximize benefits of diabetes technology by improving glycemic outcomes, improving PROs, supporting consistent access to care, and developing sustainable, scalable strategies to improve outcomes in pediatric T1D care.

CRediT authorship contribution statement

Ming Yeh Lee: Writing – review & editing, Writing – original draft, Project administration, Investigation, Formal analysis, Conceptualization. Nicole Rioles: Writing – review & editing, Conceptualization. Franziska K. Bishop: Writing – review & editing, Project administration, Investigation, Conceptualization. Susan Thapa: Writing – review & editing, Project administration. Victor Ritter: Writing – review & editing, Resources, Investigation, Formal analysis, Data curation. Manisha Desai: Writing – review & editing, Supervision, Funding acquisition. Priya Prahalad: Writing – review & editing, Supervision, Project administration, Formal analysis, Conceptualization. David M. Maahs: Writing – review & editing, Supervision, Investigation, Funding acquisition, Conceptualization. Mark P. McGovern: Writing – review & editing, Supervision, Funding acquisition, Conceptualization.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: David M. Maahs reports a relationship with Abbott that includes: consulting or advisory. David M. Maahs reports a relationship with Sanofi that includes: consulting or advisory. David M. Maahs reports a relationship with Eli Lilly that includes: consulting or advisory. David M. Maahs reports a relationship with Medtronic Inc. that includes: consulting or advisory. David M. Maahs reports a relationship with Biospex that includes: consulting or advisory. David M. Maahs reports a relationship with Kriya Therapeutics that includes: consulting or advisory. David M. Maahs reports a relationship with Enable Biosciences that includes: consulting or advisory. Priya Prahalad reports a relationship with Sanofi that includes: consulting or advisory. Priya Prahalad reports a relationship with Insulet Corporation that includes: consulting or advisory. Priya Prahalad reports a relationship with Dexcom Inc. that includes: funding grants. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.