Comparison of the Usability, Accuracy, Preference, and Satisfaction of Three Once-Weekly Glucagon-Like Peptide 1 Receptor Agonist Pen Devices in People With Type 2 Diabetes

Abstract

Aim

This study’s aim was to compare the time and accuracy of use and participants’ satisfaction and preferences with pen devices for the once-weekly glucagon-like peptide 1 (GLP-1) receptor agonists dulaglutide, exenatide XR BCise, and semaglutide.

Materials and methods

In this triple crossover, open-label, simulated injection study, GLP-1 receptor agonist pen devices were compared, with time and accuracy of use and participants’ satisfaction and preferences as primary outcomes. Participants had type 2 diabetes and were naive to GLP-1 receptor agonist therapy. Participants watched instructional videos for each device, demonstrated administration, and then provided feedback after each demonstration. Investigators tracked errors and omissions of demonstration steps for accuracy and time. Differences across devices were compared using univariate mixed models, adjusting for multiple comparisons.

Results

Of the 60 participants, 50% were male, a majority (65%) were Caucasian, and most (65%) had adequate health literacy. Participants rated the dulaglutide device easier to use than those of exenatide XR BCise or semaglutide (P <0.001 for each). Participants expressed greater satisfaction with the dulaglutide device compared with those of exenatide XR BCise or semaglutide (P <0.01 for each). Most participants (75%) preferred the dulaglutide device overall; however, many participants (61%) preferred the size and portability of the semaglutide device. The dulaglutide device took less time to use than the exenatide XR BCise or semaglutide devices (69 vs. 126 and 146 seconds, respectively; P <0.001 for each). Participants were less accurate when using the dulaglutide device.

Conclusion

Most participants preferred the dulaglutide device. The dulaglutide device took the least amount of time to demonstrate; however, demonstration accuracy was lower.

Glucagon-like peptide 1 (GLP-1) receptor agonists comprise one of several medication classes available to treat type 2 diabetes. GLP-1 receptor agonists are a preferred treatment option for many different individuals with type 2 diabetes (1). They are attractive options because they target multiple pathophysiologic defects and reduce glucose and weight with minimal risk of hypoglycemia. Some GLP-1 receptor agonists have also demonstrated cardiovascular and renal benefits (1). Based on clinical evidence, these drugs are preferred agents for patients with established atherosclerotic cardiovascular disease (ASCVD) and those with high ASCVD risk, as well as those with a compelling need to minimize hypoglycemia or a compelling need to minimize weight gain or promote weight loss (1). They are also preferred over insulin as the first injectable agent for type 2 diabetes treatment (1). However, disadvantages such as gastrointestinal adverse effects, subcutaneous administration requirements (except for oral semaglutide), and cost can affect adherence and persistence and may limit use (1).

Seven GLP-1 receptor agonists are available for use in the United States: exenatide (2), liraglutide (3), exenatide XR (4), dulaglutide (5), lixisenatide (6), semaglutide (7), and oral semaglutide (8). GLP-1 receptor agonists are available in injectable pen devices and administered subcutaneously into the abdomen, thigh, or upper arm, except for oral semaglutide, which is the first and only orally administered option in the class. At the time of this study, exenatide XR was available in its original pen delivery system, and an updated pen device was sold under the name BCise. Many differences exist within the class, including efficacy and safety profiles, dosing schedules, and preparation and administration requirements of the individual agents, making product selection and use potentially confusing for both health care professionals and patients (9–11).

In addition, utilization, adherence, and persistence remain particularly challenging with the GLP-1 receptor agonists. Even with well-established cardiovascular benefit, few people with diabetes are prescribed these agents. Recent data from the National Health and Nutrition Examination Survey show that use of GLP-1 receptor agonists and sodium–glucose cotransporter 2 inhibitors (another relatively new drug class for the treatment of diabetes) has increased only marginally from 0.6% in 2003–2006 to 7.1% in 2015–2018 (12). Several barriers exist to initiation and consistent use of GLP-1 receptor agonists, including treatment and prescribing complexity, clinical inertia, cost, gastrointestinal adverse effects, and injection concerns (13,14).

Dosing frequency is one important consideration that could affect adherence with GLP-1 receptor agonist therapy. Whereas three of the injectable agents are given either twice-daily (exenatide) or once-daily (liraglutide and lixisenatide), the other three (exenatide XR, dulaglutide, and semaglutide) are once-weekly formulations. Once-weekly formulations may offer the advantage of improved adherence and persistence compared with once- or twice-daily medications in some people. In some clinical trials, adherence and persistence to treatment and treatment satisfaction significantly increased with once-weekly GLP-1 receptor agonists compared with once- or twice-daily agents in the class (15–19). In addition, patient surveys indicate greater preference for once-weekly dosing than for once-daily dosing (20,21).

Of note, one study showed better persistence with dulaglutide compared with liraglutide and exenatide XR, with better persistence with liraglutide compared with exenatide XR, which suggests the persistence is related to more than dosing frequency alone (19). In addition, evidence from real-world studies shows significant variation in adherence and persistence among the GLP-1 receptor agonists. Not all of the data in these studies favor the once-weekly agents, and there have been generally poor persistence rates overall (22).

Some studies have surveyed participants to identify preferences for specific treatment attributes such as dosing frequency, delivery system, efficacy, and side effects, with varying results. One study found that the key drivers were side effects, efficacy, dosing frequency, and required preparation (23). Another showed side effects, efficacy, and dosing frequency to be most important (24). A third showed that, when differences in efficacy between medications were deemed small, dosing frequency and delivery system were more important to participants (25).

Ease of use is another particularly important consideration when comparing the GLP-1 receptor agonists because complexity of medication administration requirements has been linked to reduced adherence, lower user satisfaction, and inaccurate dosing (26). Preparation and administration requirements vary significantly among the GLP-1 receptor agonists. Some pen devices are single-use, whereas others are multiuse. Some require attachment of a pen needle, whereas others have a built-in needle. Some require reconstitution; others do not.

Only a few studies have evaluated the impact of product differences on accuracy of use and user satisfaction. One study compared the usability of and user preference for the once- or twice-daily injectable agents (lixisenatide, exenatide, and liraglutide) in people with type 2 diabetes (27). The study results indicated that lixisenatide allowed faster task completion and had more successful user performance. Our previous research compared the usability and accuracy of the three once-weekly GLP-1 receptor agonists that were available for use in 2016 (albiglutide, dulaglutide, and exenatide XR), when used by health care professionals. Dulaglutide was associated with the highest user satisfaction and fastest demonstration, with fewest errors compared with exenatide XR and albiglutide (28). Since then, the exenatide XR pen device was modified with the approval of the newer BCise pen device in 2017, albiglutide has been discontinued because of a steady decline in sales, and semaglutide has been approved. Most recently, user preferences for the dulaglutide and semaglutide pen devices were compared in a randomized, crossover trial among injection-naive people with type 2 diabetes. The dulaglutide device was associated with greater user preference and shorter training time compared with the semaglutide device (29).

To date, there has been no direct comparison of user accuracy, satisfaction, and preference of all three currently available once-weekly GLP-1 receptor agonists. Therefore, the primary objective of this study was to perform such an evaluation to provide information regarding the complexity and usability of these agents and aid clinicians in the decision-making process when selecting a specific medication within the GLP-1 receptor agonist class. Secondary objectives were to explore various factors that affect usability, satisfaction, accuracy, and preference scores.

Research Design and Methods

General Design and Materials

This was a multicenter, prospective cohort study. To evaluate study outcomes, participants completed an individual interview with a study investigator to model pen device demonstrations and complete usability and preference surveys. Demonstration pen devices and injection pillow foams were used. Participants were aware that they would not receive any subcutaneous injection nor any active drug component. Study participants were provided the remuneration of a $50 gift card for participating. A sample size calculation was performed based on our previous study (28), indicating that at least 40 participants were needed to show significance in the accuracy rating. We determined that 60 participants would allow for additional subgroup analysis regarding the secondary outcome. This study was approved by the Colorado Multiple Institutional Review Board.

Participants

The study population included individuals aged 18–89 years with type 2 diabetes who were naive to GLP-1 receptor agonist therapy. People with type 1 diabetes; non-English speakers; individuals with self-reported impairment of cognition, hearing, vision, or dexterity; and those employed in a position with a role in treating or educating people with diabetes (e.g., health care workers) were excluded.

Study participants were recruited between June 2019 and May 2020 from four diverse outpatient clinics throughout Colorado, including an endocrinology specialty clinic, a primary care patient-centered medical home, a women’s health primary care clinic, and a family medicine federally qualified health center. Participants were recruited primarily from populations receiving clinical care from the research investigators or under the care of a practice colleague at the investigators’ respective sites.

Potential participants were screened for eligibility at the time they presented for a routine clinic visit or were contacted via telephone in advance of a visit. Study recruits received notification and assurance that participation in the study had no effect on their access to or the quality of clinical care.

Interview Procedures

Three investigators performed the interviews. Study participants were individually interviewed by one of the investigators in a private room (e.g., clinic office space, exam room, or conference room) for ∼60–120 minutes. Interviews were audio-recorded to allow for review of participant responses at a later date, if needed, and participants were informed that they would be recorded before the interviews started. Each participant was given blank paper and a pencil to take notes throughout the interview, if desired.

To start the interview, study participants first signed an informed consent form and then completed a self-administered, written intake questionnaire that documented their eligibility for participation and demographic and background health information and assessed their baseline willingness to use injectable medications. For individuals deemed otherwise eligible for study inclusion, a health literacy assessment was verbally administered by the investigator using a widely used, validated screening tool (Newest Vital Sign [NVS]) (30).

Participants completed three cycles of pen device demonstrations that included 1) preparing for demonstration by watching the manufacturer’s instructional video of the product up to two times and reviewing printed instructions for use, 2) demonstration of the pen device without support from the observer or use of printed instructions, and 3) completion of written satisfaction and preference surveys. Participants were instructed to demonstrate and/or verbalize each step of the demonstration; for example, participants could choose to state that they would wash their hands instead of physically completing this task. The order of the pen device demonstrations was randomized using 3 × 3 Latin squares. The time taken to demonstrate the use of each device was tracked using a digital stopwatch.

Assessments

Task Evaluation Form

The study investigators developed an accuracy checklist for each device based on the manufacturer’s instructions for use (Supplementary Appendix S1). Based on this checklist, investigators determined that dulaglutide and exenatide XR BCise each had nine required steps, and semaglutide had 11 required steps. Investigators tracked whether participants physically demonstrated or verbalized each step correctly or incorrectly or omitted the step. To ensure consistency in using the task evaluation form and to standardize ratings, investigators met at baseline and after one-third of participants (n = 20) had completed the study procedures.

User Satisfaction and Usability Survey

After each device demonstration, participants completed a three-item satisfaction questionnaire and a six-item usability questionnaire. A seven-point Likert scale (with 1 being “very unsatisfied” and 7 being “very satisfied”) was used to assess satisfaction with each device and included questions regarding comfort of using the device, time needed to prepare the injection, and overall experience with the device. For usability, a five-point Likert scale (with 1 being “very difficult” and 5 being “very easy”) was used to assess ease or difficulty of various preparation and injection steps, understanding of device instructions for use, and overall rating.

User Preference Survey

For overall preferences, participants completed an eight-item questionnaire using a three-point ranking system with 1 indicating “most preferred” and 3 indicating “least preferred.” The ranking questionnaire was followed by an additional three questions designed to ascertain how likely the participants were to use injectable medications, how likely they were to use these three specific pen devices, and how likely they were to recommend those same three medications to other people. The latter three questions were ranked on a scale from 0 to 10, with 0 indicating “not at all” and 10 indicating “very likely.”

Statistical Analysis

Differences in outcomes across pen devices were compared using univariate mixed models, adjusting for multiple comparisons using Tukey’s adjustment. The Wilcoxon signed rank test was used to look at changes within individuals for nonnormal data. To compare usability, ease, and satisfaction among the devices, we used generalized linear mixed models with random effects. Independent variables that were tested between groups included baseline willingness to use injectable medications, previous use of a non–GLP-1 receptor agonist injectable medication, and health literacy. Descriptive statistics were used for other analyses. A P value <0.05 was deemed significant. All analyses and plots for this project were generated using SAS, v. 9.4, software (SAS Institute, Cary, NC).

Results

Participant Demographics

Sixty participants were included in the study, with a mean age of 61 years. Half of the participants were male, 65% were Caucasian, and 43.3% had previously used a non–GLP-1 receptor agonist injectable medication (Table 1). A majority of study participants (63.3%) had adequate health literacy based on the NVS screening.

Table 1.

Participant Demographics (N = 60)

Category	Responses
Mean age, years	61
Male sex	30 (50)
Race/ethnicity White/Caucasian Hispanic/Latino Native American/American Indian Black/African American Asian/Pacific Islander Other Preferred not to answer	39 (65) 11 (18.3) 2 (3.3) 2 (3.3) 1 (1.7) 2 (3.3) 3 (5)
No previous use of a non–GLP-1 receptor agonist injectable agent	34 (56.7)
Education College, graduate level College, undergraduate level High school Less than high school	22 (36.7) 19 (31.7) 15 (25) 4 (6.7)
English as first language	56 (93.3)
Diabetes duration, years >10 6–10 1–5 <1 year or unknown	21 (35) 12 (20) 21 (35) 6 (10)
NVS health literacy score 0–1 (limited) 2–3 (possibly limited) 4–6 (adequate)	10 (16.7) 12 (20) 38 (63.3)

Data are n (%) unless otherwise noted.

User Preference Survey Outcomes

Overall, a majority of participants preferred dulaglutide over semaglutide or exenatide XR BCise (74.6 vs. 13.6 and 11.9%, respectively), as shown in Figure 1. A majority of participants indicated that dulaglutide was the easiest pen device to learn how to use, yielded the best confidence of correct and complete dose delivery, had the best exterior design and styling, and was the easiest on which to set a dose. However, the semaglutide pen device was noted to have the best size and portability. Within this survey, two questions were negatively framed. Based on these negatively framed questions, exenatide XR BCise was noted as the pen device requiring the most effort to inject and was least likely to be recommended to others with type 2 diabetes.

FIGURE 1.

User preference survey responses.

User Satisfaction and Usability Survey Outcomes

Overall, user satisfaction with all three pen devices was high on average, with mean ratings of 5 or above on a seven-point scale, correlating to ratings of “somewhat satisfied” or above (Figure 2). Participants indicated greater satisfaction with the dulaglutide pen device compared with the exenatide XR BCise or semaglutide pen devices (P <0.01 for each), including comfort of use (P <0.01 for each) and time to prepare the injection (P <0.001 for each).

FIGURE 2.

User satisfaction survey responses.

Participants rated the dulaglutide pen device easier to use than the exenatide XR BCise or semaglutide pen devices (P <0.001 for each), as shown in Figure 3. This trend was consistent for other aspects of use, including mixing the medication, removing the needle cap or cover, and knowing when the injection is complete. Injecting the medication was found to be easiest with the dulaglutide pen device (P <0.001 for each comparison) and was rated the same between the semaglutide and exenatide XR BCise devices. Understanding the instructions for use was easiest with dulaglutide (P <0.001 for each comparison) and was deemed similar between semaglutide and exenatide XR BCise (P = 0.89).

FIGURE 3.

Pen device usability survey responses.

Willingness to Inject Medications

The mean rating for willingness to inject medications, on a scale of 0–10, was 7.35 (SD 2.98) at baseline and 7.68 (SD 3.24) at the completion of the study (P = 0.2251). Willingness to inject medications at both baseline and completion of the study did not differ between those who had previously used an injectable medication and those who were injection-naive.

Participants also rated on a 0–10 scale how likely they would be to use one of the once-weekly GLP-1 receptor agonists for management of their type 2 diabetes, with a mean rating of 7.93 (SD 3.05). The mean rating of likeliness of recommending one of the once-weekly GLP-1 receptor agonists to someone else with type 2 diabetes was slightly lower at 7.86 (SD 3.18).

Accuracy and Time of Demonstration

Accuracy at demonstrating injection technique was lower when using the dulaglutide pen device compared with the exenatide XR BCise and semaglutide devices (P <0.001 for each), as shown in Table 2. Handwashing was the most commonly omitted step for all pen devices (73% with dulaglutide, 77% with exenatide XR BCise, and 75% with semaglutide). Other commonly omitted steps included explanation of proper storage such as taking the medication out of the refrigerator (77% with dulaglutide, 45% with exenatide XR BCise, and 75% with semaglutide) and ensuring that the medication does not appear cloudy (61% with dulaglutide, 85% with exenatide, and 58% with semaglutide). The most common errors in demonstrations of dulaglutide injection were related to making sure the pen device was locked before removing the gray base cap (12% incorrect, 7% omitted) and waiting for the second click after completing the injection (15% omitted). With the semaglutide pen device, the most common error was related to completing a flow check (28% incorrect, 18% omitted). Multiple errors were seen with demonstration of exenatide XR BCise injection, including the steps of shaking the auto-injector hard for at least 15 seconds until the medicine was mixed (10% incorrect, 3% omitted) and checking to see if the medication was mixed (17% omitted). Many participants did not wait 15 seconds before releasing the auto-injector (13% incorrect, 13% omitted). Three exenatide XR BCise pen demonstrations resulted in a broken pen device.

Table 2.

Comparison of Accuracy and Time to Complete Simulated Injection Demonstration

	Dulaglutide (9 Steps)	Exenatide XR BCise (9 Steps)	Semaglutide (11 Steps)	P
				Dulaglutide Versus Exenatide XR BCise	Exenatide XR BCise Versus Semaglutide	Dulaglutide Versus Semaglutide
Mean accuracy, % of steps completed	62.7	74.4	73.1	<0.001	0.826	<0.001
Mean time to complete, seconds	69	126	146	<0.001*	0.080*	<0.001*

P values adjusted for multiple comparisons using Tukey’s adjustment and univariate mixed models.

^*P value result from univariate generalized linear mixed models.

The demonstration time varied for each device. The dulaglutide device took less time to demonstrate compared with the exenatide XR BCise and semaglutide devices (P <0.001), as noted in Table 2. There was no significant difference in time to inject between exenatide XR BCise and semaglutide.

Influence of Health Literacy on Outcomes

As a secondary analysis, the potential influence of health literacy on outcomes related to satisfaction, usability, preferences, accuracy, and timing of demonstration was assessed. Based on the NVS health literacy screening, 36.7% of participants had limited health literacy. Limited health literacy did not affect satisfaction and usability ratings; these ratings were not significantly different between those with adequate and limited health literacy for any pen device.

Health literacy also did not significantly affect the accuracy of injection demonstration. Participants with low health literacy took longer to demonstrate injection technique compared with those with adequate health literacy, but this difference was not significant.

Discussion

GLP-1 receptor agonists are a preferred therapy in the treatment of type 2 diabetes and have multiple potential benefits. Limitations to their use often include prescriber concerns about a person’s willingness to use injectable therapies and the usability of the pen devices. However, we found that willingness to use an injectable, once-weekly GLP-1 receptor agonist was high, regardless of prior experience with non–GLP-1 receptor agonist injectable therapies. Participants found the pen devices easy to use overall and were satisfied with their features. Although all pen devices were generally accepted, most participants preferred the dulaglutide pen over the semaglutide and exenatide XR BCise pens. It is noteworthy that preference and accuracy were not significantly affected by prior experience with non–GLP-1 receptor agonist injectable therapy use or health literacy. Although not seen as an influencing factor in the results, participants with previous injection use frequently verbalized that the semaglutide pen device was more comfortable to use.

Accuracy was overall near 70%, and the most commonly missed steps were less clinically impactful (e.g., handwashing). There were erroneous demonstrations with each pen device that were more clinically impactful, and these examples could potentially lead to injection failure, inaccuracy of dose delivered, or injection of improperly mixed medication. This finding highlights the importance of direct education about pen device use and injection technique before initiation of a GLP-1 receptor agonist, as accuracy concerns remained after self-study of the product instructions and instructional video.

In consideration of educational methods, study investigators noted that most participants did not use the printed instructions for use and instead relied on education provided in the manufacturers’ instructional videos. There were differences in the training videos, including length (ranging from 3 to 8 minutes), key points emphasized, and format. Some participants noted that the training videos were distracting because they offered extraneous content, which could have limited participants’ ability to comprehend instructions. Additionally, most manufacturers only offered accessible training videos in one or two languages, which may limit their usability in diverse populations. This finding further emphasizes the need for individualized education and training, a need that could be filled by various health care providers, including community pharmacists at the time of dispensing a GLP-1 receptor agonist.

Preferences and perceptions play a key role in adherence, which in turn affects the real-world effectiveness of GLP-1 receptor agonists. Although clinical studies of these agents demonstrate their robust glucose-lowering efficacy, large claims database studies suggest less effect in real-world populations, with medication-taking behaviors accounting for much of the observed difference in A1C reduction in clinical practice versus randomized controlled trials (31,32). Thus, the American Diabetes Association’s Standards of Medical Care in Diabetes (33) recommend that treatment decisions be made collaboratively with people with diabetes based on their preferences. Shared decision-making is an important element in therapy selection. Based on this study, most participants preferred dulaglutide over semaglutide or exenatide XR BCise injections. However, participants commented on other factors that could have influenced the overall decision that were not individually assessed, such as fear related to ability to see the semaglutide needle and consideration of the multidose nature of semaglutide to limit plastic waste.

One of this study’s strengths was recruitment from four distinct practice settings across two different regions in the state of Colorado, including primary care, specialty, and underserved-focused clinics. This strategy allowed for a diverse group of participants from varying age, racial/ethnic, and socioeconomic groups. In addition, consistency was ensured among sites in a number of ways. Randomization of pen demonstration order limited learning bias. Additionally, the pen device training provided to each participant was consistent, as all subjects watched the same instructional videos and had access to the same printed instructions for use.

Limitations

A few limitations must be acknowledged. The number of steps required for demonstration differed among the pen devices. Thus, accuracy ratings may be slightly skewed for pen devices that had fewer overall steps, where each step counted for a higher percentage in the accuracy score. In addition, this study used demonstration pen devices instead of actual injectable medications. Thus, one variable that could affect practical patient injection preference includes comfort of the injection itself. For example, needle size varies among the devices. The dulaglutide pen device contains a 29-gauge, 5-mm needle; the exenatide XR BCise pen uses a 23-gauge, ∼7-mm needle; and the semaglutide device uses a 32-gauge, 4-mm needle, which could technically be changed to a different size if a user so desired. Our study did not assess these differences that could be seen in real-world practice. Additionally, participants commented on a potential for biased responses based on the frequency of seeing certain products on television commercials.

Conclusion

Overall, the dulaglutide pen device was preferred over the semaglutide and exenatide XR BCise devices by a majority of participants, based on usability and satisfaction. Although the dulaglutide pen took the least amount of time to demonstrate, its associated demonstration accuracy was lower. This was the first study to compare participant preferences for the three available once-weekly GLP-1 receptor agonist injectable pen devices, and the results provide important information to contribute to shared decision-making when prescribing a GLP-1 receptor agonist. Additional studies may be needed if new GLP-1 receptor agonists and pen devices become available.