Evaluation of a New Clinical Tool to Enhance Clinical Care of Control-IQ Users

Abstract

Background:

The purpose of this study was to develop and test a new Clinic Tool to assist health care professionals with clinical care of persons with diabetes using the Control-IQ system.

Methods:

A Clinic Tool was iteratively developed with input from diabetes clinicians, which outlined a systematic process for assessing data, reviewing insulin settings, providing education, and documenting the encounter. Diabetes clinicians were recruited to trial the Clinical Tool in up to five clinical encounters (in-person, telehealth, or telephone). Quantitative surveys and free-text responses, including a knowledge quiz and the System Usability Scale (SUS), were administered to determine clinician satisfaction, confidence, knowledge, and implications for practice.

Results:

Twenty-nine clinicians (43% endocrinologists, mean 10.7 years in practice) enrolled in the study and completed 89 encounters using the Control-IQ Clinic Tool. Participants spent an average of 10 minutes using the Tool and reported excellent SUS scores within the 90%-95% percentile for usability. Knowledge quiz scores increased in 42% of participants. Both familiarity with Control-IQ and confidence providing clinical care to Control-IQ users significantly improved (P = .009 and P < .001 respectively). Ninety percent of participants agreed that the Tool will change their clinical care going forward.

Conclusion:

The Control-IQ Clinical Tool is highly usable and impacted clinical care delivery to Control-IQ users. Tools that serve to improve clinician confidence in delivery of care to diabetes device users should be expanded, leveraged, and studied to assess the impact on adherence and glycemic control for persons with diabetes.

Introduction

In recent years, new automated insulin delivery (AID) systems have become available for persons with diabetes. The AID systems consist of an insulin pump that is integrated with a continuous glucose monitor (CGM) and contain a software algorithm that changes insulin delivery based on glucose levels and other factors, aiming for a target glucose value or range. ¹ To date, the Food and Drug Administration (FDA) has approved three AID systems for use in people with type 1 diabetes (T1D), the Minimed 670G and 770G systems, as well as the Tandem t:slim X2 with Control-IQ Technology (referred to here as Control-IQ).^2-5 Other AID systems are available in other regions of the world as well. ⁶

Although new devices may improve overall care for individuals with T1D, they can be intimidating for clinicians who work with individuals using these systems. In a recent survey of providers in the T1Dx Clinic Network, when clustered by technology attitudes and perceived competency, only 20% of providers were categorized as “ready” for diabetes technology with positive technology attitudes and low barriers. ⁷ In addition, a recent stakeholder workshop sponsored by the Helmsley Trust and JDRF International revealed that health care professionals (HCPs) lack confidence in working with diabetes devices overall, and a variety of recommendations to improve HCP education were generated based on clinician feedback. ⁸ It was emphasized that new resources for HCPs were needed that could be integrated into a clinician’s workflow and provide practical guidance for clinicians during a clinical encounter.

Considering the identified need for practical, workflow-conscious resources, our PANTHER (Practical Advanced THERapies for Diabetes) team developed a Clinic Tool to help HCPs with the Control-IQ system. Control-IQ was recently FDA approved in 2019, with commercial availability in 2020. The Tool was designed to “hold-the-hand” of the HCP through a routine clinical care visit and support the clinician in providing optimal clinical assessment, device education, and insulin dose adjustments, without requiring in-depth knowledge of the device. The purpose of this article is to describe the PANTHER Clinical Tool for Control-IQ and the feasibility testing of the tool in a sample of HCPs.

Methods

PANTHER Clinical Tool for Control-IQ

Initial versions of the Clinic Tool were drafted based on extensive clinical experience with Control-IQ in clinical trials and piloted its use with HCPs at our Center, making iterative changes in response to the feedback. The Tool is designed to work in parallel to the commercially available Control-IQ software (t:connect), as a printed or online resource to use as a step-by-step guide to report interpretation and essential education (Figure 1). The tool instructed clinicians to configure the t:connect Control-IQ reports to certain specifications (eg, target glycemic range, 70-180 mg/dL) and review specific data reports in t:connect (eg, dashboard summary), in addition to providing guidance on behavioral recommendations and insulin dosing adjustments for Control-IQ users. The final Tool includes the following components:

Figure 1.

Control-IQ Clinic Tool, available at http://BDCPantherDiabetes.org.

1. Instructions: How to use the Clinic Tool, brief review of Control-IQ features, and the four “keys” to success with Control-IQ: Wear CGM consistently, set a sleep schedule every night, bolus before meals, and give hyperglycemia correction doses if recommended by bolus calculator.

2. Dashboard Report Assessment: To asses glycemic targets (Time-in-Range [TIR, 70-180 mg/dL], time above and below range, mean glucose, and American Diabetes Association goals for therapy^9,10), system use metrics (Time in Use, CGM active), and insulin delivery and bolus behavior.

3. CGM Hourly and Timeline Assessment: To assess patterns of variability and bolus behavior, and educate on bolus behavior, hypoglycemia management, and infusion set changes.

4. Device Settings and Education: Guidance on which insulin dose settings can be adjusted and suggestions for optimizing other Control-IQ settings.

5. Electronic Health Record (EHR) Note: Template language that auto-populates information from the other components when used online to copy and paste into EHR.

6. After Visit Summary: Summary of key education for Control-IQ users to take home after clinical care visit.

Clinic Tool Feasibility Study Design

To test the feasibility of using the Clinic Tool as part of routine diabetes care, we recruited HCPs to use the tool during clinical visits with Control-IQ users. Recruitment occurred from April to August 2021, and the only exclusion criterion was the inability to read and write in English (as Tool and surveys were in English). Health care professionals from our clinic were recruited in person and by email, and HCPs from other clinics in the United States were recruited by sending emails to HCP colleagues, posting on social media, and soliciting participation during academic presentations. The goal was to recruit individuals in different practice settings and from different disciplines (physicians, advanced practice professionals, diabetes care, and education specialists). The Colorado Multiple Institutional Review Board approved the study, and informed consent was obtained from all participants.

Upon enrollment, baseline data were collected, including demographic and practice-specific information. Health care professionals rated their familiarity with the Control-IQ system and confidence providing clinical care to Control-IQ users on a 1-10 scale and completed a Control-IQ knowledge quiz consisting of 10 multiple choice questions (“pre-test”). The authors (C.B. and L.M.) created the knowledge quiz based on extensive experience with the Control-IQ system. The questions were reviewed by a group of 20 persons with diabetes using Control-IQ and HCPs familiar with Control-IQ to provide content validity and feedback to improve the clarity of the knowledge test. After completing baseline measures, the authors (L.M. or C.B.) trained participants in-person or by videoconference on how to access the Clinic Tool (www.pantheridabetes.org) and use it during a clinical visit. All data and surveys were completed online using Research Electronic Data Capture (REDCap) database.^11,12

Participants were asked to use the Clinic Tool in up to five clinical encounters with Control-IQ users, for in-person, telehealth, or telephone visits. After each use of the Tool, participants completed a brief “Encounter Survey” to report how long it took them to work through the Clinic Tool and which parts of the Tool were used.

After completing five encounters using the Clinic Tool (or at the end of the data collection period), participants completed in-depth exit surveys, including the 10-item knowledge quiz (“post-test”) and post-assessment of confidence on a 1-10 scale. They also answered questions about whether the Control-IQ Clinic Tool was satisfactory, whether it enhanced their knowledge, whether they would recommend to co-workers, whether it was relevant to their practice, whether they plan to use the Tool, and whether they thought they would not need the Tool after a while.

The System Usability Scale (SUS) was used to measure usability and acceptance of the Clinic Tool. ¹³ The SUS is a 10-question validated survey with five responses for each item (strongly agree to strongly disagree) that is scored by adding item responses and multiplying by 2.5 to convert to a 0-100 score. Scores are then normalized to percentile rankings to determine overall usability in comparison with other system usability results from other studies. ¹⁴ Finally, participants provided feedback via free-text responses on how to improve the tool as well as feedback on how the tool would change their clinical care and diabetes education with Control-IQ users going forward.

Statistical Analysis

Descriptive statistics were calculated for all sample characteristics, tool components, and questionnaires. Counts and percentages were calculated for categorical variables, and means, standard deviations, and min-max were calculated for continuous variables. The SUS scale was converted from raw scores and then translated to percentile ranges. ¹⁴ Paired t tests were used to compare pre- and post-assessments of familiarity and confidence separately. A Bonferroni-corrected significance level of .25 was used to determine significant results. All descriptive and inferential statistics were completed using SAS Version 9.4.

Results

In total, 28 participants enrolled in the study from nine different clinical settings in the United States. Most participants were endocrinologists who worked in pediatric clinical care settings (Table 1). Participants reported caring for a mean of 18.3 Control-IQ users per month.

Table 1.

Participant Characteristics.

Variable	Measure
Medical personnel type
Endocrinologist	12 (42.9%)
Primary Care Physician (Peds, Internal, or Family Practice)	0 (0%)
Nurse Practitioner	7 (25.0%)
Physician’s Assistant	2 (7.1%)
Registered Nurse	4 (14.3%)
Registered Dietician	2 (7.1%)
Other	1 (3.6%)
Years in practice
	10.7 (SD = 10.5) (Min-Max = 0-40)
Do you work with pediatrics, adults, or both?
Pediatrics	8 (53.3%)
Adults	5 (33.3%)
Both	2 (13.3%)
How many patients do you see in a month on diabetes devices (CGMs, insulin pumps, hybrid closed loop, etc.)?
	48.6 (SD = 33.5) (Min-Max = 2-150)
How many patients do you see in a month using Control-IQ?
	18.3 (SD = 15.1) (Min-Max = 0-50)

Abbreviation: GGM, continuous glucose monitor.

Use of Tool

There were 89 encounters recorded using the Clinic Tool, with 23 participants using the Tool at least once and 16 participants using the Tool four times or more. Five of the 28 participants who enrolled in the study did not complete any encounter surveys: two never encountered a Control-IQ user and three were lost to follow-up. Of the 89 Clinic Tool uses, 57% were in-person visits (51 encounters), 21% telehealth visits (19 encounters), 18% phone call visits (16 encounters), and 4% other (email and during hospitalization, 3 encounters). When the Clinic Tool was used during in-person visits, the majority used the paper-based version of the Tool, whereas for telemedicine and telephone calls, the Tool was primarily used online (Table 2).

Table 2.

Clinical Encounters Using the Control-IQ Tool and Use of Tool Components.

Tool/component use	Clinic visit: in-person (n = 51)	Clinic visit: telemedicine (n = 19)	Telephone call (n = 16)	Total (n = 89) a
On Paper (%) b	34 (66.7)	1 (5.3)	3 (18.8)	39 (43.8)
Minutes Spent (mean (SD) (min-max))	10.1 (6.7) (1-30)	7.1 (4.7) (2-20)	13.5 (5.6) (5-20)	10.11 (6.4) (1-30)
Instructions (%)	49.0	73.7	87.5	61.8
Dashboard (%)	86.3	84.2	93.8	87.6
CGM Hourly and Timeline (%)	86.3	68.4	100	85.4
Device Settings and Education (%)	78.4	78.9	87.5	79.8
EHR (%)	33.3	15.8	43.8	32.6
AVS(%)	70.6	36.8	31.3	55.1

Abbreviation: AVS, After-Visit summary; CGM, continuous glucose monitoring; EHR, electronic health record.

^aData are not shown for the three “other” type of encounters.

^bPercentages are calculated using the denominator that is presented in the Visit Type.

Participants used the components of the Control-IQ Clinic Tool at different frequencies depending on the type of visit (Figure 2). The Dashboard summary was the most universally used component of the Tool, used during 88% of clinical encounters, followed by the CGM Hourly/Timeline and Device Settings/Education sections, which were used in 85% and 80% of all encounters, respectively. The EHR template was the least used component of the Tool, used in only 33% of encounters overall. The After Visit Summary was used in 70% of in-person visits, but only in about one-third of telemedicine and telephone call visits, likely because the provider could not easily pass along the handout to the Control-IQ users when not meeting in person. There were no discernible trends across multiple encounters with increased or decreased use of different components of the Clinic Tool within participant. Additional free-text comments about Tool use are found in the Supplement.

Figure 2.

Number of encounters (out of total 89) and reasons for not using individual tool components.

Reasons for Not Using Clinic Tool Components

Participants reported that the most common reason for not using individual components of the Clinic Tool was that they did not feel like they needed it for that encounter. Very few participants reported not having enough time or not understanding how to use the tool component as a reason for not using a tool component (Figure 2). The EHR Note was the least used component, and 32% of those who did not use it cited “other” as the reason. Free-text responses indicate that this largely related to clinicians already having their own note templates to work from, thus not needing this component. The After Visit Summary also had a high proportion of “other” reasons for not using it (41%), with most reporting they forgot to print it for the Control-IQ user or could not easily give it to the user during telemedicine or phone call visits.

Clinic Tool Usability

The SUS composite raw score was mean 82.3 ± 14.2 out of 100. When normalized through percentile range conversion, ¹⁴ this score falls into the 90%-95% percentile when compared with other published SUS scores. This is considered an “excellent” score, demonstrating high acceptability ¹⁵ of the Clinic Tool.

Three-quarters of participants reported overall satisfaction with using the Control-IQ Clinic Tool (75%, Table 3). More than 95% of respondents agreed or strongly agreed that the Tool was relevant to their clinical practice, and 42% of participants thought they may not need the Tool after using it for a period of time.

Table 3.

Provider Satisfaction With Control-IQ Clinic Tool Based on 5-Point Scale (strongly disagree → strongly agree).

Question	% responded “agree or strongly agree” (n = 21)	Mean score (out of 5)	Standard deviation
I was satisfied using the Clinic Tool as part of my clinical care visit.	75	3.95	0.94
The Clinic Tool enhanced my knowledge on Control IQ.	85.7	4.14	0.65
I would recommend using this Clinic Tool to my co-workers.	90.4	4.48	0.68
The Clinic Tool is relevant to my clinical practice.	95.2	4.57	0.60
I plan to use the Clinic Tool in my clinical practice.	66.7	4.05	0.97
I think that after a while I will no longer need to use the Clinic Tool.	42.8	3.43	1.08

When asked about use-case for the Tool, 90.5% of participants endorsed the Tool to be used to teach new clinicians how to provide care for control-IQ users, and 87.5% endorsed using during the clinical care encounter with the Control-IQ user (Table 3). Only 61.0% would use the Clinic Tool to prepare ahead of time for a clinical visit, and slightly more than half (52.4%) would give the entire Tool to the Control-IQ user at the end of the visit. Only 23.8% of participants would scan the Tool into the EHR after use.

Knowledge and Confidence

The overall knowledge quiz scores improved slightly from baseline to study end from mean 87.1% to 93.3% correct answers, with 43% of participants improving their score in the post-test. On a scale of 1 to 10, participants’ familiarity with Control-IQ increased from 7.96 (95% confidence interval [CI] = 7.27-8.64) to 8.72 (8.41-9.03) (P = .009) and participants’ confidence providing clinical care to Control-IQ users increased from 7.57 (6.69-8.45) to 9.00 (8.59-9.22) (P < .001).

Impact of Control-IQ Tool on Clinical Practice

Ninety percent (19/21) of participants reported that using the Clinic Tool will change their clinical practice, with Control-IQ users going forward as well as their approach to educating Control-IQ users. In free text, many participants noted that the main benefit of the Clinic Tool was the systematic approach to efficiently analyze data, which increased their confidence in providing care to Control-IQ users. Many appreciated the guidance on how to assess system use and interpret the t:connect reports, and several others found the educational tips useful. Finally, several reported that they were unaware of all the information that could be found in the settings report and found the pump prompts and tips for optimizing the pump settings to be very helpful. Many reported being unaware of the “auto off” setting in the pump and reported identifying several instances where the auto off setting was causing undesired pump suspensions after using the Tool and understanding what that setting was.

Suggestions for Tool Improvement (Free Text)

Many participants reported that the Clinic Tool could be improved by providing more opportunity for customization (Supplement). While many participants reported using all components of the tool and did not think the tool was too long, others desired an option to shorten the tool and pick and choose components.

Several suggestions were provided on incorporating the Tool more efficiently into the workflow. This included suggestions to integrate the Tool more fully into the EHR, as opposed to having text that can be copied and pasted into an EHR note and creating access to the After-Visit Summary via a QR code so that patients could access it easily after the visit.

Discussion

The Control-IQ Clinic Tool was able to significantly increase clinician confidence with the Control-IQ system and took about 10 minutes to use in a variety of settings. In addition, clinicians reported high satisfaction and usability of the tool. To our knowledge, this is the first report of a practical Tool that is able to boost clinician confidence with a new diabetes technology in clinical practice.

Evaluation of new educational and training resources is measured by four hierarchical levels of results, including satisfaction, learning, behavior change, and results. ¹⁶ This study was able to measure the first three levels, demonstrating high satisfaction, and self-reported and objective increase in knowledge. Most participants reported that their clinical practice changed by systematizing their approach to data review, equipping them with salient educational topics, and directing their attention to Control-IQ settings. Future prospective studies will be required to assess whether these practice changes improve knowledge, adherence, or glycemic control for individuals with diabetes using Control-IQ.

Tools that can be used in both telehealth and in-person visits will be important to the future of clinical care. A limitation of the Control-IQ Clinic Tool is that some components worked better on paper (ie, giving the After-Visit Summary to the person with diabetes), and some components worked better in the online version (ie, copying and pasting the EHR note into the medical record). The latter was not used by the majority of clinicians, who reported having their own EHR templates and methods of documentation. However, true EHR integration (beyond template text that requires cutting and pasting) has been identified as a major request from clinicians to incorporate and streamline tools into their workflow^8,17 and was also suggested by the participants of this study to improve the Control-IQ Tool.

Therapeutic technologies for type 1 diabetes are advancing at a rapid rate. In the past four years, the United States has seen commercial approval of two different AID systems with two additional designs currently under FDA review. Each of these designs works differently with changes to different system settings having different impacts on the function of each system. For example, the traditional “insulin sensitivity” setting in the Medtronic 670G/770G system will have no impact on system performance, while in the Tandem Control-IQ system this setting will be used for both user-administered corrections and system-delivered Auto Corrections. For the emerging Omnipod 5 system from Insulet, the insulin sensitivity will impact the user, but not system-delivered insulin, while for the Beta Bionics Islet, there will not even be an insulin sensitivity setting. Such complexity in a rapidly changing care system is challenging for all HCPs, and especially for providers whose practice extends beyond exclusive diabetes care. Without tools such as those developed through this work, providers may be reluctant to prescribe the full range of available systems or may provide ineffective dosing recommendations for systems they do not fully understand.

This work has several limitations. The sample size at this stage of development was relatively small. Many providers were pediatric-focused, and none were primary care providers, so the findings may not extend to adult endocrinologists or primary care providers. As there was no control group, it is unknown whether the results can be attributed to the Tool or simply increased exposure to Control-IQ. Furthermore, many participants had fairly high baseline diabetes technology knowledge, which may also limit generalizability. However, despite the high baseline knowledge of Control-IQ in this sample, participants reported an increase in their confidence in providing clinical care after using the tool, suggesting that knowledge about a device is not sufficient to ensure confidence for clinicians. Clinical tools that provide guidance in clinical assessment and decision-making, like the Control IQ Clinic Tool tested in this study, may benefit clinicians, regardless of the baseline knowledge.

In conclusion, the Control-IQ Clinic Tool performed well in early testing and was successful in improving provider knowledge and confidence in using this system. As the diabetes technology landscape continues to advance in complexity, with new devices becoming available each year, such tools will be needed by a growing array of providers to maintain knowledge and confidence in using emerging technologies to benefit people with diabetes.