Description and Preliminary Evaluation of a Diabetes Technology Simulation Course

Abstract

Background

We aim to provide data on a diabetes technology simulation course (DTSC) that instructs internal medicine residents in the use of continuous subcutaneous insulin infusion (CSII) and continuous glucose monitoring system (CGMS) devices.

Methods

The DTSC was implemented during calendar year 2012 and conducted in the institution’s simulation center. It consisted of a set of prerequisites, a practicum, and completion of a web-based inpatient CSII-ordering simulation. DTSC participants included only those residents in the outpatient endocrinology rotation. Questionnaires were used to determine whether course objectives were met and to assess the satisfaction of residents with the course. Questionnaires were also administered before and after the endocrine rotation to gauge improvement in familiarity with CSII and CGMS technologies.

Results

During the first year, 12 of 12 residents in the outpatient endocrinology rotation completed the DTSC. Residents reported that the course objectives were fully met. The mean satisfaction score with the course ranged from 4.0 to 4.9 (maximum, 5), with most variables rated above 4.5. Self-reported familiarity with the operation of CSII and CGMS devices increased significantly in the postrotation survey compared with that on the prerotation survey (both p < .01).

Conclusions

In this pilot program, simulation-based education increased the perceived familiarity of residents with CSII and CGMS technologies. In light of these preliminary findings, the course will continue to be offered, with further data accrual. Future work will involve piloting the DTSC approach among other types of providers, such as residents in other specialties or inpatient nursing staff.

Introduction

The prevalence of diabetes mellitus is rapidly rising.¹ Contemporary diabetes management increasingly involves the use of complex technologies.^2–5 For instance, an estimated 400,000 patients in the United States with diabetes are now treated with insulin pump therapy [also known as continuous subcutaneous insulin infusion (CSII)].⁶ Among the newest technologies in diabetes care are continuous glucose monitoring systems (CGMSs).^5,7–9 While CSII and CGMS are intended for outpatient use, these devices are also being encountered by inpatient clinical care teams.^10,11

Technology-enhanced simulation can improve learning outcomes.¹² Simulations of clinical cases can be delivered through various modalities, including computers or mobile devices (screen-based patient simulation) or in person (immersive patient simulation). Both types of approach have been used to teach diabetes management to health care professionals and laypersons.^13–17

We previously reported on an online simulation course for resident physicians on inpatient diabetes management.¹⁸A simulation-based education course that focuses on CSII and CGMS technologies has now been developed and piloted. The need grew from the experiences of faculty who believed that trainees were not proficient in basic history-taking skills and that they also lacked familiarity with diabetes-related technology when encountering patients with diabetes. The objectives were (1) enhance knowledge of contemporary technologies used in diabetes management and (2) develop expertise with the institutional computerized order entering system with regard to inpatient CSII. We report on course design and on trainee satisfaction with the program after 1 year of operation.

Methods

Description of Facility

The multidisciplinary simulation center consists of 3300 square feet of space. The space contains a learning center, four rooms set up to mimic patient care areas, and a control room. Two medical codirectors, an operations manager, a technical engineer, two educators, and an administrative coordinator staff the center.

Overview of Diabetes Technology Simulation Course

Experts in endocrinology, diabetes education, family medicine, information technology, and medical simulation designed the diabetes technology simulation course (DTSC). Direct learning objectives (Table 1) were developed. Residents began participating in the DTSC in January 2012 as part of their month-long endocrinology rotation. All residents were in their second year of training.

Table 1.

Direct Learning Objectives of the Diabetes Technology Simulation Course

Objective	Description
1.0	Define the elements of a patient history when interviewing a diabetes patient who is using CSII and/or CGMS
2.0	Become familiar with operation of CSII devices and associated informatics
2.1	Review the various CSII models that are commercially available
2.2	Demonstrate basic CSII operational skillsa
2.3	Describe how stored CSII data can be downloaded and analyzed
3.0	Become familiar with the operation of CGMSs and associated informatics
3.1	Review the various CGMS models that are commercially available
3.2	Demonstrate basic CGMS management skillsb
3.3	Demonstrate basic knowledge of CGMS informatics and data modeling
4.0	Practice computerized entry of CSII orders
5.0	Complete a patient history on a mock patient using CSII and CGMS devices

^aIncluding, but not limited to, insertion/disconnection of pump infusion set, filling reservoir/priming pump, programming, accessing the status screen, locating the basal review screen, delivering insulin bolus, locating daily total insulin and history screen, locating current amount of insulin in reservoir, and locating the toll-free customer service number on the pump.

^bIncluding identifying appropriate CGMS candidates or clinical applications, inserting the glucose sensor, programming the glucose sensor parameters into the system receiver, and explaining interstitial fluid lag time.

Residents were provided with the direct learning objectives and the course expectations. The DTSC has three components: a set of prerequisites, a practicum, and completion of a web-based inpatient CSII-ordering simulation. The prerequisites consisted of assigned readings on CSII and CGMS^2–5 and completion of a manufacturer’s web-based course covering basic CSII and CGMS device operation. Residents were required to complete these prerequisites during the first week of their endocrinology rotation.

The practicum was a hands-on, 3 h session conducted by a certified diabetes educator. The course was limited to one or two residents per class, thus allowing for individualized learning. Commonly used CSII and CGMS devices were selected for the practicum. The most frequently used skills were demonstrated by the trainer, with a return demonstration. Checklists were used to ensure that all the necessary teaching points had been addressed. Additionally, residents were presented with different tabletop scenarios and asked to demonstrate how they would reprogram the CSII device with new settings for each scenario. Insertion of the CSII infusion set and insertion of the glucose sensor were practiced using a partial task trainer. The in-room computer was used to demonstrate how to download information for analysis.

Skills were further reinforced in a simulation utilizing a standardized patient, with residents taking a focused history from the patient that was supplemented by obtaining data from the patient’s insulin pump. A checklist was developed to document whether all the necessary historical information had been obtained by the resident, and the instructor provided feedback after completion. All hands-on course activities were videotaped for future review.

Lastly, a computer-based simulation was constructed on how to place CSII orders for inpatients. The rationale for this component in the DTSC stemmed from prior studies indicating that residents were unfamiliar with inpatient diabetes-related technology.^{19, 20} Moreover, our hospital has a long-standing policy allowing individual patients on outpatient CSII therapy to transition that technology to the inpatient setting.¹⁰ Part of that process involves completion of an electronic order set (for current format, see Figure 1). The DTSC simulation walked the resident stepwise through the ordering process. The simulation was then reinforced by a certified diabetes educator who observed and provided feedback as the resident placed orders on test patients.

Figure 1.

View of the electronic CSII orders for inpatients. An electronic simulation was constructed to walk trainees through the various steps of the ordering process, which was then reinforced by a tutorial on test patients. SQ, subcutaneous.

Diabetes Technology Simulation Course Assessment

The primary outcome assessed during this pilot phase of the DTSC was whether residents perceived any improvement in their familiarity with the diabetes technologies and electronic ordering. A questionnaire was administered to residents before the DTSC, immediately after the DTSC, and then upon completion of the endocrinology rotation. Response choices were “not at all familiar,” “somewhat familiar,” and “very familiar.” Responses were assigned a numeric value of 0, 1, and 2, respectively.

Additionally, the trainees were asked to rate the course and to indicate whether it met the stated objectives. Residents were asked whether the stated objectives were met and to indicate “not met,” “partially met,” or “met” with respect to the different course goals. The DTSC ratings were graded from 1 (needs improvement) to 5 [top 10% (one of the best)]. Finally, after completion of the endocrinology rotation, trainees were surveyed to determine if they had encountered any patients using CSII or CGMS and, if so, if that had given them an opportunity to input electronic orders.

Data Analysis

Responses to the DTSC surveys were tabulated by averaging precourse and postcourse survey data and comparing the data using the Student’s t test to determine whether residents expressed improved familiarity with the CSII and CGMS technology.

Results

Course Assessment

To date, 12 of 12 residents have completed the course. Eleven participants rated the course objectives as being fully met (Table 2). Mean course satisfaction scores ranged from 4.0 to 4.9, with most questions rated above 4.5. The lowest satisfaction score was related to the quality of the scenarios (Table 3). Most residents reported that they thought the course was relevant to their practice.

Table 2.

Assessment of Diabetes Technology Simulation Course Objectives

Objectivea	Fully met	Partially met	Not met
Operation of insulin pumps (CSII)	12	0	0
Operation of CGMS	12	0	0
Obtain and interpret data from CSIIs and CGMSs	11	1	0
Obtain a focused history from a patient with a CSII	12	0	0

^aFrequency of response to “Were the following learning objectives met?”

Table 3.

Rating of Diabetes Technology Simulation Course

Indicator	Mean (standard deviation) ratinga
The course met my learning needs	4.92 (0.28)
Faculty knowledge of subject matter	4.92 (0.28)
Quality of the scenarios	4.00 (0.71)
Opportunity to reflect on my performance during the debriefng session	4.42 (0.67)
Faculty provided constructive feedback in a nonthreatening manner	4.83 (0.39)
Overall course rating	4.92 (0.28)

^aOn a scale of 1 to 5 (1, strongly disagree disagree nor agree; 4, agree; 5, strongly; 2, disagree; 3, neither agree).

Changes in Familiarity

All 12 residents returned the postrotation survey. There was a significant increase in self-reported familiarity with the operation of CSII and CGMS devices (Table 4). Familiarity was reported as not improving with respect to entering orders on inpatient CSII users. In terms of postrotation experiences, 9 residents had taken one to three patient histories, 7 residents had encountered one to three patients with CSII devices, and 3 residents had encountered one to three patients with CGMSs. One resident reported feeling “very comfortable with pump patients when I saw them later that month.”

Table 4.

Rating of Familiarity with Targeted Diabetes Technologies^a

Variable	Precourse	Postrotation	P value
Operation of CSII	0.42 (0.51)	1.25 (0.62)	<0.01
Operation CGMS	0.25 (0.45)	1.00 (0.43)	<0.01
Entering inpatient CSII orders	1.08 (0.67)	1.50 (.067)	0.14

^aOn a scale of 0 to 2 (0, not familiar; 1, somewhat familiar; 2, very familiar). Values are mean (standard deviation) unless indicated otherwise.

Discussion

Both CSII and CGMS devices can be encountered across a span of medical specialties and in various settings. Although considerable resources can been invested in training the patient in their use, little attention has focused on methods to train providers in the operation of devices such as insulin pumps and CGMSs. In settings other than an endocrinology practice, these technologies will be encountered infrequently. Nonetheless, resident physicians may encounter patients who use such devices and thus would benefit from having some basic knowledge of how they function.

Various simulation modalities were used during the course, including web-based training, clinical case scenarios, and immersive human patient simulation. Specific objectives were developed and outlined. In this preliminary assessment, most residents indicated that the course objectives had been met, with maximum or nearly maximum favorable ratings.

Additionally, the overall rating of the course was high. Nearly perfect scores were received on whether the course met learning needs, on faculty knowledge, on opportunity for self-reflection, and on the feedback provided. The lowest score, albeit still high overall, was on the clinical scenarios, which has since prompted some revisions; these revisions have consisted mostly of making the patient simulation more complex (e.g., having the patient be less knowledgeable about the technology, thus forcing the trainee to operate the pump to obtain the required data) and developing more varied and complex tabletop scenarios. In relation to the scenarios, the residents requested additional scenarios that address the management of patients with poor glycemic control and that involve troubleshooting related to insulin pumps.

One of the main objectives of the DTSC was to improve the familiarity of residents with CSII and CGMS—to remove the sense of uncertainty that they may feel when encountering these devices. On average, the course participants reported an increased familiarity with the operation of CSII and CGMS devices. Familiarity with electronic ordering did not improve, however, possibly because the residents did not have an opportunity to encounter patients on CSII in the inpatient setting, which the simulation was designed to represent.

This pilot study included only internal medicine resident trainees rotating through an outpatient endocrinology elective. Nonetheless, the results were encouraging and showed that simulation-based education increased the perceptions of residents about their familiarity with diabetes technology that may not have happened without this educational intervention. On the basis of these preliminary findings, the course will continue to be offered, with further data accrual and analysis.

Future work will involve piloting the DTSC approach among other types of providers, such as residents in medical specialties other than internal medicine. Of particular interest would be tailoring the DTSC to train inpatient nursing staff. These technologies are a clinically low frequency in the hospital, yet they represent a potentially high risk when encountered in that setting. A course for inpatient nurses would have the potential to remedy a latent safety threat to the patient entering the hospital with such a device by training the first-line providers who interact with them and care for them. Ongoing experience will allow refinement of the simulation-based educational process and permit expansion beyond endocrinology.

Glossary

(CGMS)

continuous glucose monitoring system

(CSII)

continuous subcutaneous insulin infusion

(DTSC)

diabetes technology simulation course