Impact of Human Factors Testing on Medical Device Design: Validation of an Automated CGM Sensor Applicator

Abstract

Improving user engagement with continuous glucose monitoring (CGM) is considered a major requirement for achieving optimal treatment efficacy. Human factors testing is needed to ensure that CGM product designs and requisite training simplify the user experience and enhance usability and patient safety. Dexcom, Inc, recently introduced a novel, “one-button” automatic sensor applicator (ASA) for use with the Dexcom G6 (rtCGM) system. The device was developed utilizing all phases of the human factors testing process. We recruited eight certified diabetes educators from independent health care institutions to conduct a comparative ease-of-use analysis to confirm the usability of the ASA. Participants judged the instructions and device to be easier to use than the previous sensor applicator.

Maintaining near-normal glucose levels without debilitating hypoglycemia remains an ongoing challenge for people with diabetes who are treated with intensive insulin therapy. Advances in continuous glucose monitoring technologies have significantly reduced the burden of diabetes for many of these patients.^1-5 Although clinical trials play an important role in demonstrating the effectiveness of these new technologies, usability testing is needed to ensure that the product design and requisite training simplify the user experience and enhances usability and patient safety.

Human factors testing involves a formal, three-phase process to ensure that it matches the cognitive and physical capabilities and limitations of the user and to eliminate any product characteristics that could result in human error leading to an adverse health care outcome. A key aspect of human factors testing is that it is based almost entirely on observation during simulated use of the device.

During the first two phases of testing, developers assess the match between the product’s design and user’s needs, user limitations and effects of the environment of use (analysis phase). This is accomplished by observing representative user interaction with initial product prototypes (formative phase). Data collected during these observations are then used to inform subsequent iterations of the design. The final phase is validation testing also called the summative testing phase.

The US Food and Drug Administration (FDA) describes a model for analyzing user requirements when interacting with a device.⁶ The model assesses three aspects of human requirements when interacting with a device: perception—what users must see, feel, or hear during performance of each step; cognition—what needs to be understood and remembered when performing each step; and action—what users need to do to physically manipulate the device to perform the various steps involved in successful operation. In this study, these three points of interaction with the previous and redesigned automatic sensor applicator (perception, cognition, and action) were compared on the dimension of ease of use. That is, participants were asked to subjectively rate the perceptual, cognitive, and physical action requirements of each step involved in sensor application.

Dexcom, Inc (San Diego, CA) recently introduced a novel, “one-button” automatic sensor applicator (ASA) for use with Dexcom G6 (rtCGM) System. The ASA device was developed utilizing all phases of the human factors testing process, extensive formative testing was performed to refine the ASA designs resulting in various design modifications. In this article, we report a study that was conducted to confirm the ease of use of the ASA relative to the previous Dexcom manual sensor applicator (MSA).

Research Design

The goal of this comparative ease-of-use analysis was to confirm that the new ASA was easier to use compared to the previous MSA. A third applicator, Medtronic Enlite applicator (Medtronic, Inc, Northridge, CA), was also assessed as a second comparator applicator; however, it was excluded from the current analysis due to design changes that occurred subsequent to this report.

The study was conducted on December 7, 2016, in Denver, CO. Study participants included eight certified diabetes educators (CDEs) from independent health care institutions. CDEs were chosen to represent the real user population due to their accumulated knowledge and experience with patient needs and use related problems with devices for managing diabetes. An independent human factors engineer designed the study and facilitated the research session.

Methodology

Procedures

Prior to starting the evaluation, the study facilitator presented an introduction to concepts and components of human factors testing and discussed the perception-cognition-action (PCA) task analysis model that would be the basis for ease of use evaluation during the study. Participants were divided into small groups of two or three and given a workbook that contained the step-by-step instructions from the official user manual for each device (Figure 1). The images and text included in the workbook were taken directly from the respective user manuals. No additional text, markings or instructions were included.

Figure 1.

Example page from participant workbook.

The groups were then assigned to begin interaction with the ASA at one of the stations situated in the corners of the conference room. Each station contained one of the test devices, with a sufficient number of applicators and sensors for each participant to perform the steps involved in inserting the sensor and attaching the transmitter at least one time. Participants were asked to perform each step according to the instructions. In most cases, participants completed the sensor insertion and transmitter insertion several times. Each participant interacted with and assessed ease of use for both sensor applicators. The sensors were applied to a “dummy tummy” in simulated use. No actual sensors were deployed. Each participant rotated to each station and assessed all of the systems.

Participants evaluated each sensor’s installation steps on their perception, cognition, and action requirements, using a 5-point Likert-type scale (1 = difficult, 5 = easy) to evaluate users’ ability to perceive important inputs from the device or instructions, understand information, and perform the appropriate actions. Table 1 lists the steps require for each sensor. Each step was given an ease-of-use rating and recorded in the workbook.

Table 1.

Comparison of Sensor Application and Transmitter Attachment Steps.

Manual Sensor Applicator (MSA)	Automatic Sensor Applicator (ASA)
Sensor Insertion	Sensor Insertion
1. Clean skin with alcohol wipe	1. Clean skin with alcohol wipe
2. Remove adhesive backing	2. Peel off the adhesive backing
3. Place the sensor pod horizontally on the skin	3. Place the applicator horizontally on the skin
4. Move fingers and apply pressure around the adhesive patch to secure the adhesive	4. Firmly press down, sticking the adhesive patch to your skin
5. Hold the applicator and pull the safety lock straight out	5. Fold down and break off the safety guard
6. Place hand on the adhesive at the edge of the sensor pod and place two fingers above the collar	6. Push down the button to insert the sensor. When the button is released, the applicator comes off
7. Depress the plunger	7. Move fingers and apply pressure around the adhesive patch to secure the adhesive
8. Move your fingers below the collar	Transmitter Attachment
9. Retract the collar until you hear 2 clicks	8. Insert the transmitter into the slot at the narrow end of the transmitter holder
10. Squeeze the rib tabs at the sides of the sensor pod	9. Firmly press down on the round end of the transmitter until it clicks into place
11. Rock the applicator forward away from your body
Transmitter Attachment
12. Place the transmitter in the sensor pod
13. Place one hand on the transmitter to keep it in place while pulling up and forward on the transmitter latch until you here 2 clicks
14. Hold the sides of the sensor pod
15. Remove the transmitter latch with the other hand by quickly twisting off the latch away from the body

Participants documented comments and observations regarding use difficulties based on their experience with patient users interacting with CGMs. In addition, participants were encouraged to “think aloud” during their evaluation; however, participants provided their own independent rating for each step of the instructions for each product.

After completing all instructions for the setup and use of each applicator, participants independently provided two overall system rankings for ease of use, one for inserting the sensor and one for attaching the transmitter. Thus, these “system rankings” were aggregates of the assessments within each system’s required steps. Both system rankings used a 10-point Likert-type scale (1 = difficult, 10 = easy). The study facilitator conducted a 1-hour debriefing of the full group to discuss participants’ experiences with the systems.

Analysis

The mean overall ease-of-use ratings for sensor insertion and transmitter attachment rated by each participant were compared. A correlation between the combined PCA ratings across devices and the overall rating documented at the end of the session was assessed. Correlation was considered significant at the .01 level (2-tailed). Mean PCA scores for each applicator system were compared. A combined score was calculated by averaging the individual PCA scores for each device. A paired t-test was conducted to compare the mean scores of each parameter between the ASA and MSA; P value < .05 was considered significant for each test. Individual PCA scores for each device were compared; sensor insertion and transmitter attachment were assessed separately.

Results

The overall ease-of-use rating scores for both sensor insertion and transmitter attachment were significantly higher for the ASA device than the MSA device (both P < .05) (Figure 2).

Figure 2.

Comparison of overall ease-of-use ratings between systems.

The combined sensor/transmitter insertion PCA rating for the ASA device was significantly higher than for the MSA device (P = .01) (Figure 3A). The advantage of the ASA was apparent in all three domains of the PCA score (perception, P = .006; cognition, P = .02; action, P = .02) (Figure 3B).

Figure 3.

Comparison of total mean combined scores for sensor insertion/transmitter insertion (A) and mean scores by individual metric (B). P values are shown for within-participant paired sample t-tests comparing the mean device scores.

The paired t-test for sensor insertion showed that participants found the ASA instructions and device to be significantly easier in steps requiring perception (P = .02) and cognition (P < .05) than the MSA; however, the difference in ratings for steps requiring action of inserting the sensor was not rated significantly different than the MSA device (P = .07) (Figure 4A). For transmitter insertion, the ASA device system was found to be significantly easier than the MSA device in all three interaction domains—perception (P = .02), cognition (P = .02), and action (P = .009) (Figure 4B).

Figure 4.

Comparison of mean PCA scores for sensor insertion (A) and transmitter insertion (B). P values are shown for within-participant paired sample t-tests comparing the mean device scores.

A statistically significant correlation (Pearson) was found between the combined PCA ratings and overall ease-of-use rating (.76), indicating a relationship between the individual task perception/execution and the respondents’ overall preference rating for the device.

Discussion

Advances in sensor technologies have led to the development of rtCGM systems that are effective in helping these individuals safely achieve optimal glycemic control.^1-5 Because of the close link between the benefits of rtCGM use and frequency of sensor wear,^1,2,7,8 the usability of these devices and accompanying instructional materials is fundamental in the optimization of use for this technology. Our analysis showed that the ASA instructions and device were found to be easier to use than the previous MSA.

Some limitations should be noted. Significance testing offers only limited insights because of the small sample size; a larger sample size would have yielded a more definitive assessment. In addition, the study included only CDEs; thus, patient perceptions of ease and usability are not known. However, because CDEs are have knowledge of and experience with patient needs and challenges, we feel that their responses likely reflect those of their of patients.

Improving user engagement to rtCGM is considered a major requirement for achieving optimal treatment efficacy.⁹ Human factors is particularly relevant to clinicians because a product that is easy to use should be easier to teach and easier to learn. Patients’ perceptions of the benefits derived from their rtCGM devices and ease of use are key drivers of adherence.¹⁰ Frequent failure to properly insert the sensor can negatively impact patients’ persistence in using their devices. Recent evidence suggests that the perceived benefits of CGM and continued CGM use is related to the usability of the device and patients’ trust in the accuracy and reliability of their glucose data.^11-13

Conclusion

Employing a human factors engineering approach in the design of rtCGM systems increases patient safety, supports adherence in device use and enhances clinician effectiveness and efficiency in patient training. Our study demonstrates that effective end-user analysis and early identification of design issues in the formative development phase and iterative design changes resulted in a sensor application device that is easy to use and that may enhance frequent and persistent rtCGM use in a broad population of potential users.