Abstract
Background:
With the rapid development of new insulin delivery technology, measuring patient experience has become especially pertinent. The current study reports on item development, psychometric validation, and intended use of the newly developed Diabetes Impact and Device Satisfaction (DIDS) Scale.
Method:
The DIDS Scale was informed by a comprehensive literature review, and field tested as part of two focus groups. The finalized measure was used at baseline and 6 months post-assessment with a large US cohort. Exploratory factor analyses (EFAs) were conducted to determine and confirm factor structure and item selection. Internal reliability, test–retest reliability, and convergent/divergent validity of the emerged factors were tested with demographics, diabetes-specific information, and diabetes behavioral and satisfaction measures.
Results:
In all, 778 participants with type 1 diabetes (66% female, mean age 47.13 ± 17.76 years, 74% insulin pump users) completed surveys at both baseline and post-assessment. EFA highlighted two factors—Device Satisfaction (seven items, Cronbach’s α = 0.85-0.90) and Diabetes Impact (four items, Cronbach’s α = 0.71-0.75). DIDS Scale demonstrated good concurrent validity and test–retest reliability.
Conclusion:
The DIDS Scale is a novel and a brief assessment tool with robust psychometric properties. It is recommended for use across all insulin delivery devices and is considered appropriate for use in longitudinal studies. Future studies are recommended to evaluate the performance of DIDS Scale in diverse populations with diabetes.
Keywords: automated insulin delivery, device impact, diabetes technology, insulin pumps, patient-reported outcomes, psychosocial, type 1 diabetes
Introduction
Rapid innovation of insulin delivery technologies has transpired over the past decade. Patients using insulin therapy have the option to consider a variety of insulin delivery devices including multiple daily injections (MDI), insulin pumps with or without continuous glucose monitors (CGM), and insulin pumps with automated insulin delivery (AID), such as predictive low glucose suspend and hybrid closed-loop insulin delivery.1,2 The availability of these devices and the increased rate of adoption of new device technologies3-5 have dramatically influenced patients’ experience managing their diabetes.6 Previous research has shown significant clinical and psychosocial benefits from new insulin delivery device technologies.7-15 However, such benefits are heavily dependent on successful uptake and continued use of these devices.16,17
To best understand factors related to the adoption and ongoing use of diabetes devices, it is critical to assess patient experience and psychosocial outcomes through use of validated measures.18 Despite the growing number of diabetes-specific psychosocial assessment tools, there is a dearth of measures evaluating the human factors (HF) aspect of diabetes-related technology in combination with the impact of device use on daily diabetes management.18
HF is the study of the interaction of people and machines to ensure the safety and effectiveness of that interaction.19 In the context of a chronic condition like diabetes, a greater understanding of HF in a real-world setting is critical in order to understand whether patient’s interaction experience with a device fits the demands of living with diabetes while also improving outcomes.20 Specifically, the assessment of HF during patient onboarding and adherence to new diabetes device technology remain important to adherence overtime.20 Device ease of use, perceived effectiveness, and perceived decrease in any burden related to managing diabetes are all essential factors for successful implementation and adherence to new diabetes technology.21 Researchers have pointed to the need to measure such device experience and attitudes both independently and in supplement to clinical and psychosocial outcomes.22,23 Past research has highlighted a positive impact of diabetes technology including AID systems, in that the devices helped improve both clinical and psychosocial outcomes.18,23,24 However, other studies have shown that patients could experience some undesirable outcomes as well, such as interrupted or poor-quality sleep when using certain types of technology.25,26 This inconsistency highlights the importance of assessing the individualized “impact” (instead of “burden”) of real-world use of technology can have on diabetes management.
The aim of this study was twofold: (1) to develop a brief measure for the evaluation of patient experience related to insulin delivery device interaction (specifically device-related satisfaction and the impact of diabetes management on an individual’s life at the time of device use), and (2) to assess the psychometric properties of the newly developed measure in a sample of users with diabetes. In this study, the term “insulin delivery device” refers both to insulin pumps, (ie, sensor augmented pumps [SAP], non-SAP, AID systems, and non-AID systems) and to devices used for MDI (eg, pens, smart pens, syringes).
Methods
Item Development
The initial questionnaire content was informed primarily by literature review of published measures that assessed diabetes-specific impact and satisfaction related to diabetes devices (supplemental Table 1). Scale items were reviewed by authors MM, KS, and SH and then coded under two primary themes—those relating to device-related satisfaction and others that evaluated the impact of diabetes management on a person’s life. Any recurring subthemes were quantified for frequency across measures. These subthemes subsequently informed the development of 12 items for the new scale. The emerged subthemes are consistent with previous research that has connected device satisfaction (DS) and psychosocial outcomes with successful use and adherence to new diabetes technology.16-17,20-26
The 12-item pool was field tested as part of 2 focus groups involving 47 participants, in all, with type 1 diabetes. The aim of the first focus group was to explore the overall applicability and appropriateness of items selected. Participants were encouraged to highlight their concerns, if any, as well as express their thoughts around any important aspect(s) missing from the questionnaire. Feedback from the first focus group helped to further refine the questionnaire. The new draft measure was subsequently administered to the second focus group as part of the Children with Diabetes meeting held in Florida (USA). This resulted in slight modifications and the final draft of the Diabetes Impact and Device Satisfaction (DIDS) measure.
Study Procedures
The DIDS Scale was used as part of a larger set of surveys administered by a market research company, dQ&A (San Francisco, CA, USA). Survey participants were members of dQ&A’s proprietary, US-based, opted-in panel that provided informed consent to receiving invitations to diabetes-related survey studies. The survey was included both in the baseline assessment for study participants (N = 1200) and then repeated at post-assessment (six months from baseline, N = 1039). Participants received a $5 incentive for completing each of the baseline and post-assessment surveys (total of $10 each participant). This study was deemed exempt from Institutional Review Board review by the Western Institutional Review Board.
General demographic and diabetes-specific information (eg, insulin delivery device used, CGM use, and self-reported glycated hemoglobin [A1c] value) were collected from participants. A1c values older than six months were excluded from the analysis.
Study Measures
The DIDS Scale included 12 items with response options on a 10-point Likert Scale. Seven of these 12 items focused on satisfaction related to the insulin delivery device (eg, trust and ease of use). The remaining five items were intended to assess frequency of commonly reported implications of diabetes (diabetes-related impact on daily life, such as worry around hypoglycemia, sleep interruptions).
Net Promoter Score (NPS)
NPS is a single-item measure used to gauge overall satisfaction and loyalty to a product or brand.27 NPS is an index ranging from −100 to 100, defined as the percentage answering 9-10 on a 10-point willingness-to-recommend scale, minus the percentage of respondents answering 0-6.
The Hypoglycemia Attitudes and Behavior Scale (HABS)
The HABS subfactor, Hypoglycemia Anxiety (HA), was used to measure hypoglycemia-related anxiety, five items rated on a five-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree).28 Internal consistency was acceptable (α = 0.72). Overall HABS and other subfactor scores were not relevant for construct validity assessment.
Data Analysis
De-identified data were obtained from dQ&A and retrospectively analyzed by the study team at Tandem Diabetes Care, Inc. (San Diego, CA, USA). Participants who reported a diagnosis of type 1 diabetes, fluency with the English language, and had completed measures both at baseline and post-assessment were included in the analysis. Participants younger than 18 years old were required to have a parent or guardian complete the survey on their behalf. The baseline survey was deployed in September 2018 (N = 1200) and the second survey (post-assessment) in March 2019 (N = 1039). The final sample size analysis included 778 participants (Table 1).
Table 1.
Patient Demographics for Baseline and Post-Assessment.
| Characteristic | Baseline (N = 778) | Post-assessment (N = 778) |
|---|---|---|
| Age | ||
| Mean (SD) | 47.13 (±17.76) | 48.13 (±17.75) |
| Gender | ||
| Female | 516 (66%) | 516 (66%) |
| Male | 262 (33%) | 262 (33%) |
| Duration of diabetes | ||
| Mean (SD) | 26.88 (±16.79) | 27.97 (±16.78) |
| Diabetes type | ||
| Type 1 diabetes | 778 (100%) | 778 (100%) |
| HbA1c (self-reported) | ||
| Mean (SD) | 6.93 (±1.05) | 6.95 (±.99) |
| CGM use | ||
| CGM user | 81.87% (637) | 85.73% (667) |
| No CGM | 18.13% (141) | 14.33% (111) |
| Insulin delivery method | ||
| Multiple daily injections | 26.0% (202) | 26.1% (203) |
| Insulin pump | 74.0% (576) | 73.9% (575) |
| Insulin pump brand/model | ||
| Animas insulin pump | 10.0% (78) | 6.4% (50) |
| Insulet OmniPod insulin pump | 18.1% (141) | 17.9% (139) |
| Medtronic MiniMed older models | < 1% (4) | < 1% (4) |
| Medtronic MiniMed 630G | 6.0% (43) | 6.0% (43) |
| Medtronic MiniMed 670G | 10.5% (82) | 11.7% (91) |
| Tandem t:slim X2 (no Basal-IQ)* | 14.8% (115) | 7.7% (60) |
| Tandem t:slim X2 with Basal-IQ* | 2.7% (21) | 13.6% (106) |
| Tandem t:slim, t:slim G4, t:flex | 11.8% (92) | 10.5% (82) |
Abbreviation: CGM, continuous glucose monitoring.
*p < .01.
Exploratory factor analysis (EFA) was performed to uncover the underlying factor structure of the newly developed scale (Figure 1).29,30 Initial EFA was performed on the data collected at post-assessment, as these sample demographics better represented a population using most current diabetes device technology (AID systems). Equivalence of construct measurement was tested over time (baseline vs post-assessment) and across device type subgroups (MDI vs insulin pump users).29 EFA was performed with both orthogonal (varimax) and oblique (direct oblim) rotations.30,31 Items were removed if they did not similarly load onto the same factors at the cutoff value of |.40| across device type subgroups.32
Figure 1.
Exploratory factor analysis procedure flow chart.
aPrinciple axis factoring extraction method was used since it is free of distributional assumptions and is less likely to produce an improper solution.31
bUsing Pedhazur and Schmelkin’s (1991) strategy, EFA was performed with both orthogonal (varimax) and oblique (direct oblim) rotations, to ensure orthogonal solution remains consistent when factors are no longer forced to correlate.31,33 If differences are negligible between the two (eg, consistent factor loadings on both rotated factor matrix from orthogonal rotation and pattern matrix from oblique rotation), the orthogonal rotation solution should be interpreted.33
Internal reliability of factors was assessed with Cronbach’s α.33,34 Factor correlations between the two time points (baseline vs post-assessment) were conducted to evaluate test–retest reliability. Construct validity was assessed using correlations and multiple regression analyses, as appropriate. IBM SPSS Statistics Version 26 (Armonk, NY, USA) was used for all statistical analyses.
Results
Descriptive Statistics
Participant descriptives are presented in Table 1. Majority of participants at both baseline and post-assessment were using an insulin pump (74%, 73.9%) and a CGM (81.87%, 85.73%). Chi-squared results indicated that CGM use significantly increased (+6.7%) and a significant change in insulin delivery device model usage from baseline to post-assessment, p < .001. Specifically, t:slim X2 pump with Basal-IQ technology (Tandem Diabetes Care Inc., San Diego, CA, USA) usage significantly increased (2.7%-13.6%), while use of t:slim X2 pump without Basal-IQ significantly decreased (14.8%-7.7%), likely due to the ability to add Basal-IQ technology to a t:slim X2 pump during the course of the study. No other significant changes in insulin pump model were detected.
Zero-order correlations were performed on the 12 measure items (Table 2). Multivariate analysis of variance test, with the 12 measure items as dependent variables and participants group (MDI vs pump) as the between-subjects factor, showed a significant difference between MDI and pump groups (F(12) = 6.171, p < .001) (Table 3). Participants using insulin pumps reported higher satisfaction, trust, sense of control in diabetes, and ease of device use and scores for hassle and worry about going low.
Table 2.
Inter-item Correlations of Initial 12-Item Pool for the DIDS Scale.
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1. Satisfaction | – | ||||||||||
| 2. Trust | .67a | – | |||||||||
| 3. Helps me feel more in control of my diabetes | .62a | .63a | – | ||||||||
| 4. Helps me have good BG control | .64a | .69a | .74a | – | |||||||
| 5. Easy to use | .59a | .57a | .51a | .58a | – | ||||||
| 6. Is a hassle to use | −.59a | −.46a | −.42a | −.48a | −.63a | – | |||||
| 7. Is too complicated | −.42a | −.37a | −.35a | −.37a | −.61a | .64a | – | ||||
| 8. Miss work/school/etc due to diabetes | −.13a | −.15a | −.16a | −.15a | −.13a | .18a | .15a | – | |||
| 9. Wake up at night to treat low BG | −.13a | −.13a | −.16a | −.14a | −.06 | .09b | .05 | .29a | – | ||
| 10. Worry about going low | −.13a | −.15a | −.17a | −.176a | −.09b | .136a | .08b | .38a | .42a | – | |
| 11. Have a bad night’s sleep due to diabetes | −.23a | −.23a | −.28a | −.27a | −.18a | .23a | .16a | .41a | .54a | .43a | – |
| 12. Treat low BG by eating snacks | −.09a | −.08b | −.13a | −.11a | −.04 | .09b | .09a | .15a | .29a | .40a | .24a |
Abbreviations: BG, blood glucose; DIDS, Diabetes Impact and Device Satisfaction.
Correlation was statistically significant at p < .01 (two-tailed).
Correlation was statistically significant at p < .05 (two-tailed).
Table 3.
Scale Items, Means, and SDs by Insulin Delivery Device Group.
| MDI |
Pump |
||||
|---|---|---|---|---|---|
| Item | M | SD | M | SD | p |
| Satisfaction | 7.59 | 1.96 | 8.25 | 1.6 | < .001 |
| Trust | 8.31 | 1.6 | 8.59 | 1.52 | .02 |
| Helps me feel more in control of my diabetes | 7.59 | 1.97 | 8.24 | 1.78 | < .001 |
| Helps me have good BG control | 7.98 | 1.76 | 8.44 | 1.66 | < .001 |
| Easy to use | 8.56 | 1.52 | 8.61 | 1.73 | .70 |
| Is a hassle to use | 3.95 | 2.59 | 3.15 | 2.5 | < .001 |
| Is too complicated | 2.35 | 1.69 | 2.42 | 2.01 | .65 |
| Miss work/school/etc due to diabetes | 2.45 | 1.92 | 2.31 | 1.81 | .35 |
| Wake up at night to treat low BG | 4.7 | 2.06 | 4.39 | 2 | .06 |
| Worry about going low | 5.66 | 2.65 | 5.07 | 2.57 | .01 |
| Have a bad night’s sleep due to diabetes | 4.83 | 2.27 | 4.85 | 2.24 | .95 |
| Treat low BG by eating snacks | 7.04 | 2.52 | 6.66 | 2.47 | .06 |
Abbreviations: BG, blood glucose; M, mean; MDI, multiple daily injections.
Note: M and SD of each measure item by insulin delivery type. p-values were derived from the between-subjects effects (MDI vs pump) of the multivariate analysis of variance performed on these data.
Exploratory Factor Analysis
All EFAs performed, both preliminary and final model analyses, had a Kaiser–Meyer–Olkin measure of sampling adequacy greater than 0.80 (above criterion of 0.60) (Figure 1). Bartlett’s test of sphericity indicated an analyzable correlation matrix.
Preliminary EFAs
PAF with varimax rotation results interpreted using Kaiser criterion indicated a three-factor solution (three eigen factors had eigenvalues >1.00), explaining 65.23% of variance.36 Conservative interpretation of the scree plot based on Catell’s rockpile criterion (Figure 2) and parallel analysis results (supplemental Figure 1) suggested a two-factor solution.37 Moreover, rotated factor matrix showed two items cross-loaded onto Factors 1 and 3, all leading to our choice to retain two factors.30,31,38-40
Figure 2.
Scree plot.
PAF with two-factor extraction explained 56.71% of variance, with all factor eigenvalues >2.13. Seven items loaded onto Factor 1 and five items loaded onto Factor 2 above |.40|. There was a significant small correlation between factors, r = −0.26, p < .001.
Separate PAF analysis for the MDI and pump group supported the two-factor solution explaining 52.88% (MDI) and 61.35% (pump) of variance. Both groups had seven items loaded on Factor 1 and five items on Factor 2. However, for the MDI group, one item loaded below the cutoff value |.40| (“Treat low blood glucose by eating snacks”), and was subsequently dropped from Factor 2. The original item pool of 12 items was reduced to 11 items, with 7 items on Factor 1 and 4 items on Factor 2. Interfactor correlations were significant for the MDI, r = −0.28, and pump group, r = −0.25, p < .01.
Final Model EFA
EFA with PAF analysis on the 11 items and results confirmed the two-factor solution, explaining 59.95% of variance. Both the rotated factor matrix and pattern matrix uniformly showed seven items loaded on Factor 1 and four items loaded on Factor 2 (Table 4). Results for MDI and pump groups also confirmed the two-factor solution, explaining 56.59% (MDI) and 61.35% (pump) of variance. Both groups had seven items loaded on Factor 1 and four items on Factor 2. Interfactor correlations were significant, r = −0.24 to −0.31, p < .01.
Table 4.
Post-Assessment Survey Factor Loadings.
| Factor 1 |
Factor 2 |
|||||
|---|---|---|---|---|---|---|
| All | MDI | Pump | All | MDI | Pump | |
| Satisfaction | 0.80 | 0.71 | 0.84 | −0.12 | −0.19 | −0.08 |
| Trust | 0.76 | 0.72 | 0.77 | −0.16 | −0.13 | −0.16 |
| Helps me feel more in control of my diabetes | 0.72 | 0.70 | 0.72 | −0.20 | −0.21 | −0.19 |
| Helps me have good BG control | 0.78 | 0.82 | 0.77 | −0.18 | −0.15 | −0.18 |
| Easy to use | 0.79 | 0.69 | 0.83 | −0.04 | −0.10 | −0.02 |
| Is a hassle to use | −0.70 | −0.60 | −0.73 | 0.11 | 0.16 | 0.09 |
| Is too complicated | −0.60 | −0.50 | −0.63 | 0.05 | 0.00 | 0.07 |
| Miss work/school/etc due to diabetes | −0.11 | −0.17 | −0.10 | 0.51 | 0.48 | 0.52 |
| Wake up at night to treat low BG | −0.04 | 0.01 | −0.05 | 0.67 | 0.70 | 0.66 |
| Worry about going low | −0.08 | −0.12 | −0.05 | 0.62 | 0.64 | 0.60 |
| Have a bad night’s sleep due to diabetes | −0.19 | −0.21 | −0.19 | 0.74 | 0.78 | 0.73 |
Abbreviations: BG, blood glucose; MDI, multiple daily injections.
Note: Factor loadings are derived from exploratory factor analysis using principle axis factoring with orthogonal (varimax) rotation, two-factor extraction.
Evaluation of longitudinal measurement invariance was tested by comparing the post-assessment and baseline EFA results. Using the baseline survey data, EFA using PAF was performed on the 11 items, which confirmed the two-factor solution, explaining 55.54% of variance. Rotated factor matrix and pattern matrix uniformly showed seven items loaded on Factor 1 and four items on Factor 2 (supplemental Table 2). Interfactor correlation was significant, r = −0.27, p < .001.
Factor 1 was labeled DS as the seven items represented user satisfaction with the insulin delivery device. Factor 2 was labeled Diabetes Impact (DI) highlighting the impact of diabetes on the user’s life at the time of device use. Descriptive statistics for the DS and DI are in Table 5. The final DIDS Scale can be found in supplemental Table 3.
Table 5.
Descriptive Statistics for Insulin Delivery Device Satisfaction and Diabetes Impact.
| Range |
|||||
|---|---|---|---|---|---|
| Factor | M | SD | Potential | Actual | Skewness |
| Device satisfaction | |||||
| Baseline | 8.18 | 1.41 | 1-10 | 2.29-10.00 | −0.89 |
| Post-assessment | 8.26 | 1.45 | 1-10 | 2.00-10.00 | −1.13 |
| Diabetes impact | |||||
| Baseline | 4.32 | 1.65 | 1-10 | 1.00-9.75 | 0.45 |
| Post-assessment | 4.22 | 1.63 | 1-10 | 1.00-10.00 | 0.43 |
Abbreviations: M, mean; SD, standard deviation.
Note: M and SD for each factor at baseline and post-assessment. Sample size was 778.
Reliability
DS showed good internal consistency, when measured at baseline (overall α = 0.86, MDI α = 0.85, pump α = 0.86) and at post-assessment (overall α = 0.89, MDI α = 0.85, pump α = 0.90). DI showed adequate internal consistency at baseline (overall α = 0.71, MDI α = 0.72, pump α = 0.71) and at post-assessment (overall α = 0.73, MDI α = 0.75, pump α = 0.73).
Zero-order correlations were significant (p < .001) between baseline and post-assessment DS (r = 0.69) and DI (r = 0.70), demonstrating good test–retest reliability.
Concurrent and Divergent Validity
No significant correlations were detected for gender, body mass index, and household income in relation to DS or DI. Participants’ age did not significantly correlate with DS but had a small correlation with DI (r = −0.130, p < .01). Self-reported A1c had a significant positive correlation with both DS (r = 0.122) and a negative correlation with DI (r = −0.188), p < .01.
Two multiple regression analyses were performed to demonstrate construct validity. Exploratory data analysis for both models did not reveal outliers and predictors showed acceptable independence, all variance inflation factors (VIFs) were within acceptable range (VIFs ≤1.08). Multiple regression analysis on device-related NPS being predicted from DS and DI showed a significant overall effect of the model, F(2,692) = 258.159, R2 = 0.427. DS was positively correlated with NPS (b* = 0.661, t(692) = 22.24, p < .001, ra(b.c) = 0.64), with higher DS scores associated with higher NPS. DI did not significantly predict NPS, b* = 0.033, t(692) = 1.11, p = .268, ra(b.c) = 0.03.
Multiple regression analysis on HA being predicted from DS and DI showed a significant overall effect of the model, F(2,676) = 106.90, R2 = 0.490. DI was positively correlated with HA (b* = 0.480, t(676) = 13.86, p < .001, ra(c.b) = 0.47), with higher DI scores associated with higher HA. DS did not significantly predict HA, b* = −0.036, t(692) = −1.05, p = .296, ra(c.b) = −0.04.
Longitudinal Assessment Use
Eighty-six participants reported using Basal-IQ technology at post-assessment and had not been using Basal-IQ technology at baseline. At post-assessment these new Basal-IQ users reported a significant increase in DS (Mdiff = 0.34, SDdiff = 0.96, p = .002) and decreased in DI (Mdiff = −0.56, SDdiff = 1.32, p < .001) compared to previous device model used at baseline.
Discussion
This study demonstrated that the DIDS is an acceptable, valid, and reliable measure of satisfaction with use of an insulin delivery device (DS) and impact of diabetes (DI) for individuals with type 1 diabetes. The DIDS Scale is insulin delivery device agnostic and appropriate for longitudinal use. This study showed the ability for DIDS Scale to detect the impact of insulin delivery device change (ie, from a traditional pump or MDI to an AID system) on DS and DI.
The psychometric analyses of DIDS Scale were robust, showing consistency across two time points (baseline and post-assessment) and insulin delivery devices (MDI and insulin pump). The two factors DS and DI demonstrated acceptable concurrent and divergent validity. Specifically, even though DS and DI were correlated, they were shown to be independently valuable, in that DS was associated with NPS (general satisfaction metric) but not HA. DI was related to HA but not with NPS. These findings support our scale validity in that our DS factor is appropriately measuring satisfaction while DI items assess diabetes-related attitudes and feelings, reflecting the impact of diabetes on a person’s life.
The ability to measure DS and DI with a single, brief tool that can be used across various insulin delivery devices and is appropriate for longitudinal use is valuable for clinical, research, and commercial settings. DIDS Scale can allow for immediate and reliable assessment of users’ experiences in relation to new device technology. This can engender important insights to aid in design improvements and future patient-centered product development. This information can prove valuable to further develop appropriate interventions to decrease likelihood of attrition in device-related use and possibly overall engagement in diabetes management. Patient perceptions of whether a device helps reduce the burden of living with diabetes while improving outcomes effectively and conveniently are essential factors when determining acceptability and long-term use of new insulin delivery device technology.
Previously published measures such as the Insulin Delivery Satisfaction Survey41 and the INSPIRE measures42 aim to specifically assess either the psychosocial impact of diabetes on daily life or device-related satisfaction. In the current study, the DIDS Scale was validated for use with all insulin delivery devices, including AID systems and can assist with evaluation of both (a) users’ experience interacting with the device, and (b) the impact of diabetes on their life. It is believed that in addition to assessing patient experience, it is important to assess whether devices “offer patients a sense of night time security, prevent headaches that result from low blood sugars, decrease the burden of the daily regimen demands, and improve sleep quality.”43
The strengths of this current study include the large sample size representative of all types of insulin delivery devices and the ability to analyze the psychometrics of DIDS Scale at two separate time points. The use of a third-party company, dQ&A, to collect the data decreased the likelihood of response bias. Moreover, the extensive literature review, focus groups and interviews, and the subject matter experts review provided strong support for the selection of items included in the initial item pool.
Limitations to this study and the DIDS scale include that the DS subscale yielded higher mean scores, which may introduce a ceiling effect while assessing satisfaction with device use. Higher mean scores may have occurred since the current study sample was homogenous (English-speaking individuals with long-standing type 1 diabetes reporting satisfactory glycemic control). Therefore, use of this scale in a more heterogenous setting is encouraged for confirmation of current findings or further scale refinement. The scope of the current study did not allow for other validated measures of diabetes DS or diabetes-related impact to be included to assess for convergent and divergent validity of the DIDS Scale. Additionally, no objective assessment of glycemic control in study participants was available. Future studies involving clinically and demographically diverse population samples are recommended to add to the current understanding and use of the DIDS scale.
Conclusion
The DIDS is a brief, reliable, and a validated scale to ascertain device-specific satisfaction as well as impact of diabetes management in individuals with type 1 diabetes. Future research is recommended to continue with the validation process by testing DIDS Scale’s concurrent validity with more diabetes-related measures and in more diverse populations with diabetes.
Supplemental Material
Supplemental material, DST897976-SUPP for The Development and Psychometric Validation of the Diabetes Impact and Device Satisfaction Scale for Individuals with Type 1 Diabetes by Michelle L. Manning, Harsimran Singh, Keaton Stoner and Steph Habif in Journal of Diabetes Science and Technology
Supplemental material, MANNING_Supplement_Figure_1 for The Development and Psychometric Validation of the Diabetes Impact and Device Satisfaction Scale for Individuals with Type 1 Diabetes by Michelle L. Manning, Harsimran Singh, Keaton Stoner and Steph Habif in Journal of Diabetes Science and Technology
Acknowledgments
Thanks to all the participants who completed survey data, and all the staff at dQ&A. We also acknowledge Laurel Messer, RN, MPH, CDE at Barbara Davis Center for helping edit the manuscript and Haidee Sanchez, BS, a behavioral researcher at Tandem Diabetes Care.
Footnotes
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Michelle Manning, Harsimran Singh, and Steph Habif are full-time employees of Tandem Diabetes Care, Inc.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by Tandem Diabetes Care, Inc.
ORCID iDs: Michelle L. Manning 
https://orcid.org/0000-0002-4731-6788
Keaton Stoner
https://orcid.org/0000-0003-3444-5123
Supplemental Material: Supplemental material for this article is available online.
References
- 1. Kropff J, DeVries JH. Continuous glucose monitoring, future products, and update on worldwide artificial pancreas projects. Diabetes Technol Ther. 2016;18(suppl 2):S253-S263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Tauschmann M, Hovorka R. Technology in the management of type 1 diabetes mellitus - current status and future prospects. Nat Rev Endocrinol. 2018;14(8):464-475. [DOI] [PubMed] [Google Scholar]
- 3. Liberman A, Buckingham B. Diabetes technology and the human factor. Diabetes Technol Ther. 2016;18(suppl 1):83-90. [DOI] [PubMed] [Google Scholar]
- 4. Bode BW, Sabbah HT, Gross TM, Fredrickson LP, Davidson PC. Diabetes management in the new millennium using insulin pump therapy. Diabetes Metab Res Rev. 2002;18(suppl 1):S14-S20. [DOI] [PubMed] [Google Scholar]
- 5. McAdams BH, Rizvi AA. An overview of insulin pumps and glucose sensors for the generalist. J Clin Med. 2016;5(1):5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Tanenbaum ML, Iturralde E, Hanes SJ, et al. Trust in hybrid closed loop among people with diabetes: perspectives of experienced system users [published online ahead of print July 1, 2017]. J Health Psychol. doi: 10.1177/1359105317718615. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Liberman A, Buckingham B. Diabetes technology and the Human Factor. Diabetes Technol Ther. 2016;18(suppl 1):S101-S111. [DOI] [PubMed] [Google Scholar]
- 8. Misso ML, Egberts KJ, Page M, O’Connor D, Shaw J. Continuous subcutaneous insulin infusion (CSII) versus multiple insulin injections for type 1 diabetes mellitus. Cochrane Database Syst Rev. 2010;1:CD005103. [DOI] [PubMed] [Google Scholar]
- 9. Shalitin S, Gil M, Nimri R, de Vries L, Gavan MY, Phillip M. Predictors of glycaemic control in patients with type 1 diabetes commencing continuous subcutaneous insulin infusion therapy. Diabet Med. 2010;27(3):339-347. [DOI] [PubMed] [Google Scholar]
- 10. Miller KM, Foster NC, Beck RW, et al. Current state of type 1 diabetes treatment in the U.S.: updated data from the T1D Exchange clinic registry. Diabetes Care. 2015;38(6):971-978. [DOI] [PubMed] [Google Scholar]
- 11. Giménez M, Lara M, Conget I. Sustained efficacy of continuous subcutaneous insulin infusion in type 1 diabetes subjects with recurrent non-severe and severe hypoglycemia and hypoglycemia unawareness: a pilot study. Diabetes Technol Ther. 2010;12(7):517-521. [DOI] [PubMed] [Google Scholar]
- 12. Schmidt S, Nørgaard K. Sensor-augmented pump therapy at 36 months. Diabetes Technol Ther. 2012;14(12):1174-1177. [DOI] [PubMed] [Google Scholar]
- 13. Bergenstal RM, Tamborlane WV, Ahmann A, et al. Sensor-augmented pump therapy for A1C reduction (STAR 3) study: results from the 6-month continuation phase. Diabetes Care. 2011;34(11):2403-205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Cummins E, Royle P, Snaith A, et al. Clinical effectiveness and cost effectiveness of continuous subcutaneous insulin infusion for diabetes: systematic review and economic evaluation. Health Technol Assess. 2010;14(11):iii-iv, xi-xvi, 1-181. [DOI] [PubMed] [Google Scholar]
- 15. Weissberg-Benchell J, Hood K, Laffel L, et al. Toward development of psychosocial measures for automated insulin delivery. J Diabetes Sci Technol. 2016;10(3):799-801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Tanenbaum ML, Hanes SJ, Miller KM, Naranjo D, Bensen R, Hood KK. Diabetes device use in adults with type 1 diabetes: barries to uptake and potential intervention targets. Diabetes Care. 2017;40(2):181-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Tanenbaum ML, Adams RN, Hanes SJ, et al. Optimal use of diabetes devices: clinician perspectives on barriers and adherence to device use. J Diabetes Sci Technol. 2017;11(3):484-492. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Barnard KD, Hood KK, Weissberg-Benchell J, Aldred C, Oliver N, Laffel L. Psychosocial assessment of artificial pancreas (AP): commentary and review of existing measures and their applicability in AP research. Diabetes Technol Ther. 2015;17(4):295-300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. U.S. Food and Drug Administration [Internet]. Office of Device Evaluation; [about 1 screen]. Human Factors. Silver Spring: CDRH Human Factors Team; 2018. https://www.fda.gov/medical-devices/device-advice-investigational-device-exemption-ide/ide-related-topics#human. Accessed September 27, 2018. [Google Scholar]
- 20. Schaeffer NE. The role of human factors in the design and development of an insulin pump. J Diabetes Sci Technol. 2012;6(2):260-264. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Schaeffer NE. Human factors research applied: the development of a personal touch screen insulin pump and users’ perceptions of actual use. Diabetes Technol Ther. 2013;15(10):845-854. [DOI] [PubMed] [Google Scholar]
- 22. Naranjo D, Tanenbaum ML, Iturralde E, Hood KK. Diabetes technology: uptake, outcomes, barriers, and the intersection with distress. J Diabetes Sci Technol. 2016;10(4):852-858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Liberman A, Buckingham B, Phillip M. Diabetes technology and the human factor. Diabetes Technol Ther. 2014;16(suppl 1):S110-S118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Barnard KD, Wysocki T, Ully V, et al. Closing the loop in adults, children and adolescents with suboptimally controlled type 1 diabetes under free living conditions: a psychosocial substudy. J Diabetes Sci Technol. 2017;11(6):1080-1088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Kropff J, Dejong J, del Favero S, et al. Psychological outcomes of evening and night closed-loop insulin delivery under free living conditions in people with type 1 diabetes: a 2-month randomized crossover trial. Diabet Med. 2017;34(2):262-271. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. van Bon AC, Hermanides J, Koops R, Hoekstra JB, DeVries JH. Postprandial glycemic excursions with the use of a closed-loop platform in subjects with type 1 diabetes: a pilot study. J Diabetes Sci Technol. 2010;4(4):923-928. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81(12):46-54. [PubMed] [Google Scholar]
- 28. Polonsky WH, Fisher L, Hessler D, Edelman SV. Identifying the worries and concerns about hypoglycemia in adults with type 2 diabetes. J Diabetes Complications. 2015;29(8):1171-1176. [DOI] [PubMed] [Google Scholar]
- 29. Brown TA. Confirmatory Factor Analysis for Applied Research. 2nd ed. New York: The Guilford Press; 2015. [Google Scholar]
- 30. Field A. Discovering Statistics Using IBM SPSS Statistics. 4th ed. Los Angeles: Sage; 2013. [Google Scholar]
- 31. Pedhazur EJ, Schmelkin LP. Measurement, Design, and Analysis: An Integrated Approach. Student ed. New Jersey: Lawrence Erlbaum Associates, Inc; 1991. [Google Scholar]
- 32. Stevens JP. Applied Multivariate Statistics for the Social Sciences. 4th ed. New Jersey: Lawrence Erlbaum Associates Publishers; 2002. [Google Scholar]
- 33. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297-334. [Google Scholar]
- 34. Kline P. Handbook of Psychological Testing. 2nd ed. London: Routledge; 2013. [Google Scholar]
- 35. Brod M, Hammer M, Christensen T, Lessard S, Bushnell DM. Understanding and assessing the impact of treatment in diabetes: the Treatment-Related Impact Measures for Diabetes and Devices (TRIM-Diabetes and TRIM-Diabetes Device). Health Qual Life Outcomes. 2009;7(1):83. doi: 10.1186/1477-7525-7-83 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Kaiser HF. The application of electronic computers to factor analysis. Educ Psychol Meas. 1960;20(1):141-151. [Google Scholar]
- 37. Cattell RB. The scree test for the number of factors. Multivariate Behav Res. 1966;1(2):245-276. [DOI] [PubMed] [Google Scholar]
- 38. Nunnally J, Bernstein IH. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994. [Google Scholar]
- 39. Zwick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychol Bull. 1986;99(3):432-442. [Google Scholar]
- 40. O’Connor BP. SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behav Res Methods Instrum Comput. 2000;32(3):396-402. [DOI] [PubMed] [Google Scholar]
- 41. Polonsky WH, Fisher L, Hessler D, Edelman SV. Development of a new measure for assessing insulin delivery device satisfaction in patients with type 1 and type 2 diabetes. Diabetes Technol Ther. 2015;17(11):773-779. [DOI] [PubMed] [Google Scholar]
- 42. Weissberg-Benchell J, Shapiro JB, Hood K, et al. Assessing patient-reported outcomes for automated insulin delivery systems: the psychometric properties of the INSPIRE measures. Diabet Med. 2019;36(5):644-652. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Barnard K, Weissberg-Benchell J. Psychosocial aspects and diabetes technology - head to head or hand in hand? Eur Endocrinol. 2016;12(1):35-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, DST897976-SUPP for The Development and Psychometric Validation of the Diabetes Impact and Device Satisfaction Scale for Individuals with Type 1 Diabetes by Michelle L. Manning, Harsimran Singh, Keaton Stoner and Steph Habif in Journal of Diabetes Science and Technology
Supplemental material, MANNING_Supplement_Figure_1 for The Development and Psychometric Validation of the Diabetes Impact and Device Satisfaction Scale for Individuals with Type 1 Diabetes by Michelle L. Manning, Harsimran Singh, Keaton Stoner and Steph Habif in Journal of Diabetes Science and Technology