Psychosocial Assessment of Artificial Pancreas (AP): Commentary and Review of Existing Measures and Their Applicability in AP Research

Abstract

Aim: This study aimed to systematically review the evidence base for the use of existing psychological and psychosocial measures suitable for use in artificial pancreas (AP) research.

Materials and Methods: This systematic review of published literature, gray literature, previous systematic reviews, and qualitative and economic studies was conducted using terms and abbreviations synonymous with diabetes, AP, and quality of life (QoL).

Results: Two hundred ninety-two abstracts were identified that reported psychosocial assessment of diabetes-related technologies. Of these, nine met the inclusion criteria and were included. Only four of 103 ongoing trials evaluated psychosocial aspects as an outcome in the trial. Of these, treatment satisfaction, acceptance and use intention of AP, fear of hypoglycemia episodes, satisfaction with AP, and an unspecified QoL measure were used.

Conclusions: A better understanding of the psychosocial side of AP systems and the extent to which human factors play a role in the uptake and efficient use of these systems will ultimately lead to the most benefit for people with diabetes.

Introduction

Artificial pancreas (AP) research has progressed significantly over recent years, raising expectations of availability of technology within the next 3–5 years.¹ It is particularly important during the development of AP systems to understand and manage these expectations and to understand the psychosocial implications of this technology in order to aid user engagement.² In addition, it is equally important to assess the confidence of a person with diabetes (PWD) in being able to trust the accuracy of the system as well as his or her perceived burden in using the technology, specifically, how much effort will be required of him or her in order to manage it. Although the primary aim of current AP research is to create a fully automated, 24-h closed-loop system with minimal or no input by the user, this is unlikely in the short or medium term, thus necessitating human input for use. Human input will likely include the tasks associated with setting up current pump and continuous glucose monitor use as well as involve additional inputs for meal announcements and exercise adjustments. For PWDs and their families, AP systems will need to engender confidence in the system's ability to perform accurately and consistently for them to achieve their desired goals, albeit with a potential “cost” of loss of personal control. The purpose of this review is to highlight psychosocial research from AP systems or their components and emphasize opportunities for future work in this area.

The concept of the AP is simple; however, its practical application is complex. All AP devices use an insulin pump and a continuous glucose sensor in addition to a feedback-controlled algorithm that automatically adjusts the rate of insulin delivery by the insulin pump based on real-time continuous glucose monitoring (CGM) data. AP systems can be unihormonal—using insulin alone—or bihormonal—where glucagon or pramlintide is also infused to reduce hypoglycemia risk. Some AP systems need to have meals or exercise “announced,” requiring the user to inform the system. Some aim to control to target glucose levels, and others focus on a target range. Finally, some are fully automated, requiring almost no input from the user. Most studies of the AP have been performed in controlled situations, but recently AP research has been taken out of the laboratory and into more normal “free living” situations, where PWDs live and eat their meals in an observed outpatient setting. Some have allowed participants to perform short periods (up to an hour) of unannounced exercise, challenging the AP system to properly adjust insulin doses to prevent hypoglycemia both in the immediate period related to exercise and in the delayed period hours later.^3–7 Others have encouraged even more challenges to the system. For example, in the bionic pancreas study by Russell et al.,⁸ participants were encouraged to eat as much and as often as they liked, drink up to three alcohol beverages a day, and exercise at will as often and as long as they wanted to. For the adolescents in this study, they were at diabetes camps and engaged in hours of daily physical activity.

Although studying the efficacy and reliability of AP systems in the real world setting is exciting, a greater understanding of the “lived experience” is crucial to ensure that technologies develop successfully and are fit to meet the psychosocial demands of living with diabetes, in addition to improving glycemic control. Only people with type 1 diabetes know whether they are able to meet the demands of AP technologies long term.² Wearing and interacting with an AP device may appear a reasonable trade-off for euglycemia and reduced risk of diabetes complications; however, the practical challenges, lack of accuracy of CGM in hypo- and hyperglycemia, the need to maintain one (or two) pumps plus one or two CGM sensors, and variable results in terms of time in target range remain problematic. The visibility of the disease state has already been demonstrated to be a challenge for many insulin pump users, and the addition of a sensor and controller simply adds to this burden.⁹

There are currently no existing validated and reliable measures that specifically assess the psychosocial aspects of AP systems. Similarly, there are no measures assessing the human factors side of AP technology. The term “human factors” has traditionally been used to define individual characteristics such as those related to the design, operation, and use of products for optimizing human performance or health. In the diabetes world, human factors engineering is an integral part of most diabetes technology and device companies. Their goal is to design a product that people with type 1 diabetes will use both immediately and long term, with a special emphasis on the product's safety and reliability. Key human factors have, as of yet, been underevaluated and include a patient's attitudes, beliefs, and emotions related to using the technology. Expectations, perceived burden, and ability to trust the technology have all been demonstrated as potential barriers to engagement with AP devices.¹⁰ Clear evidence-based guidelines on the use of psychosocial measures in AP research are required to complement published guidance aiming to align the biochemical and engineering outcomes. Guidelines to support research teams to identify and select appropriate measures, agreed upon by working group consensus, in collaboration with experts, users, research teams, and regulators, will ensure best-practice psychosocial evaluation alongside medical and engineering evaluation. In addition, this information will be of value to regulators (Food and Drug Administration, National Institute for Health and Care Excellence, and Agency for Healthcare Research and Quality, for example) in determining the target group(s) for access to AP technology.

One such psychosocial outcome is quality of life (QoL), which is a widely recognized as an important health outcome in diabetes, where the burden of self-management is demanding. The concept of QoL can be confusing with terms such as psychosocial functioning, functional health status, and well-being, often used interchangeably with QoL but in reality assessing different things. It is unsurprising therefore that QoL is not routinely included psychosocial assessments in AP studies, perhaps because research teams struggle to identify the most appropriate measures to effectively assess psychosocial aspects alongside the engineering and biomedical outcomes of AP research. Regulatory decisions, however, are increasingly made on all aspects of a device or therapy, including patient-reported outcomes and impact on QoL central to those decisions.

The following definitions of QoL, health-related QoL, and psychosocial functioning perhaps emphasize the subjectivity of the concepts and help us understand why they can be so difficult to measure:

• • QoL

  • ○ “A good quality of life can be said to be present when the hopes of an individual are matched and fulfilled by experience. The opposite is also true: a poor quality of life occurs when the hopes do not meet with the experience.”¹¹

  • ○ “A multi-faceted construct that encompasses the individual's behavioural and cognitive capacities, emotional well-being and abilities requiring the performance of domestic, vocational and social roles.”¹²

  • • Health-related QoL. Health-related QoL can be defined as an individual's subjective experience of his or her illness and the impact that illness and its treatment has on the individual's functioning across a variety of domains.^13–16 The key domains of health-related QoL include physical, psychological, and social functioning^13,16 as well as the impact of illness on the ability to engage in activities of daily living.^13,15

• • Psychosocial functioning. Psychosocial functioning describes the interaction between psychological and social behavior (i.e., the behavior of an individual in relation to his or her social environment). Thus, it can be used to explore and describe the way an individual responds to a given situation in the context of his or her lived experience.

The aim of this article was to systematically review the evidence base for the use of existing QoL and psychosocial measures suitable for use in AP research. This will provide essential information to research teams when assessing the psychosocial impact of AP technology and to regulators when evaluating research outcomes and considering approvals of new devices.

A systematic review of published literature, gray literature, previous systematic reviews, and qualitative and economic studies was conducted using the search terms “artificial pancreas,” “closed loop,” “type 1 diab*,” “psych*,” and “quality of life.” Two hundred ninety-three abstracts that reported psychosocial assessment of diabetes-related technologies were identified and screened for instrument names. Of these, 10 met the inclusion criteria and were included in the review. Only four of 103 current ongoing trials evaluated psychosocial aspects as an outcome in the trial. Of these, treatment satisfaction, acceptance and use intention of an AP, fear of hypoglycemia, satisfaction with the AP, and an unspecified QoL measure were used. These studies are beyond the scope of the current review as they are ongoing and not published.

Summary of Identified Studies

AP/closed-loop-specific studies

Van Bon et al.¹⁷ explored patients' perception and future acceptance of an AP using interviews and treatment satisfaction questionnaires. Interviews were based on the technology acceptance model and were held with 22 adults with type 1 diabetes. Results showed the AP was “perceived as likely to be useful, with specific advantages including stable glucose regulation, reduced need for self-monitoring of blood glucose, relief of daily concerns and time saving.”¹⁷ Despite over half of participants (58%) being reluctant to start insulin pump therapy, “the majority (79%) reported having no barriers to starting to use the artificial pancreas. Trust in the device was related to the quality of glucose control it would provide, with greater glucose control equating to greater perceived trust in the device. Almost all participants expressed an intention to use the new system when it was available even if it would not initially cover a full 24 hour period.”¹⁷

Barnard et al.¹⁰ conducted a mixed-methods psychosocial evaluation alongside the biomedical investigation of the overnight at-home closed-loop research. Qualitative interviews were held with adolescent participants (n=15) in the trial as well as their parents (n=13) to elicit views on the lived experience of using the closed loop and taking part in the study. Quantitative measures included the Diabetes Technology Questionnaire and the Hypoglycemia Fear Survey. Results show mixed feelings about use of the technology, with positives cited as reassurance and peace of mind, confidence, time off from diabetes demands, safety, and improved diabetes control. However, difficulties with calibration, alarms, and the size of the devices were perceived disadvantages. Diabetes Technology Questionnaire scores reflected the qualitative findings, whereas Hypoglycemia Fear Survey scores were mixed. The authors concluded that although closed-loop technology was cutting edge in the treatment of type 1 diabetes, further research from longitudinal studies is required to determine the long-term psychosocial benefit. The psychological and metabolic benefits of the device outweighed the practical challenges; however, as all participants reported technical difficulties to some extent, these cannot be overlooked when designing next-generation closed-loop devices.

Bevier et al.¹⁸ explored the acceptance of future AP technology in clinical trial participants (all using AP for trial purposes) via a novel 34-item questionnaire. The survey assessed current treatment satisfaction, dimensions of clinical trial participant motivation, and variables of the technology acceptance model: “Thirty-six (out of potential forty-seven) participants completed the survey. Of these, 86% were either highly likely or likely to adopt the technology once available. Current treatment satisfaction, satisfaction (motivation), personal health benefit (motivation), perceived ease of use and perceived usefulness were high.”¹⁸ The authors concluded that “individuals with direct artificial pancreas technology experience expressed a high likelihood of future acceptance. Personal benefit, convenience, perceived usefulness and perceived ease of use suggest system adoption in this highly motivated participant population.”¹⁸

Other diabetes technology studies with psychosocial data collection

Several studies have focused on specific components of the AP or diabetes devices that are closely linked to AP systems. Examining research in this area is particular pertinent as there is limited usage today of insulin pump therapy and CGM devices despite their potential benefits on glycemic control and QoL. Thus, understanding the extant research in these areas can help improve efforts to streamline and optimize uptake of AP devices when they become more readily available.

Markowitz et al.¹⁹ examined psychosocial issues in the JDRF CGM Clinical Trial in a subset of 28 youth and their parents and 21 adult patients assessed longitudinally, comparing those randomized to CGM with standard care. Measures included the Children's Depression Inventory, the Center for Epidemiologic Studies–Depression Scale, the State-Trait Anxiety Inventory, the Diabetes Family Conflict Scale, the Problem Areas in Diabetes Scale, and a blood glucose monitoring communications questionnaire. The youth, their parents, and the adult patients reported difference responses to CGM use, highlighting the need to assess perceptions separately across different developmental groups (youth vs. adults with type 1 diabetes) and different stakeholders (patients vs. parents, for example). Although this was a pilot study of CGM psychosocial issues, the findings indicated that CGM use was associated with both positive and negative psychosocial impacts. Parents in both treatment groups reported greater fear of hypoglycemia than youth in the corresponding groups. The youth in the CGM group and their parents reported more negative affect around blood glucose monitoring than those in the control group. On the other hand, the adults in the CGM group reported less diabetes-related burden than adults in the control group. In assessing symptoms of affective disorders, youth in the CGM group reported more anxiety than control youth, whereas adults in the CGM group reported less anxiety than the control adults. Finally, the parents of youth in the CGM group reported greater perception of depressive symptoms in their children than did parents of youth in the control group. The authors concluded that “there is a need for ongoing research to better understand the barriers and burdens of diabetes technologies until such advances are truly seamlessly integrated into one's daily life effortlessly.”¹⁹

Polonsky and Hessler²⁰ investigated the QoL-related benefits and losses associated with real-time CGM via a survey of current users. Eight hundred seventy-seven participants completed an online questionnaire investigating perceived QoL since initiation of real-time CGM and real-time CGM attitudes and behavior: “Major quality of life factors reported included perceived control over diabetes, hypoglycemic safety, and interpersonal support. Quality of life improvements were common for perceived control over diabetes (86%) and hypoglycemic safety (85%) although less common for interpersonal support (37%). Predictors of perceived quality of life benefits were greater confidence in using real-time continuous glucose monitoring data, satisfaction with device accuracy, usability, older age, more frequent receiver screen views, and use of multiple daily injections.”²⁰ The authors concluded that diabetes-specific QoL benefits resulting from real-time CGM were common, with major predictors of such benefit being satisfaction with device accuracy and usability and trust in one's ability to use the device. Thus “perceived efficacy for both device and self are key quality of life determinants.”²⁰ The authors concluded that psycho-educational strategies to boost confidence in using real-time CGM and provide reasonable device expectations might enhance QoL benefits.

Steed et al.²¹ reported the development and piloting of an acceptability questionnaire for CGM devices. Using semistructured interview methodology with six participants with diabetes, issues relating to acceptability of and satisfaction with the devices were explored: “Resulting broad themes were interference with daily activities; reliability and accuracy of the devices; practicality and ease of use; improvements in glycemic control; side effects and self-consciousness and disclosure.”²¹ The authors concluded that “users' preferences and their assessment of acceptability will determine uptake and use of continuous glucose monitoring devices.”²⁰ It is therefore essential to consider and evaluate this alongside clinical efficacy and cost-effectiveness as AP technology develops to a marketable device.

Other diabetes technology studies—reviews

Schaffer²² reported the role of human factors in the design and development of an insulin pump. The definition of a human factor is presented as “any physical, perceptual, cognitive, or behavioral aspect of a human being that impacts a technological system or environment.”²² He described the way that human factors are applied during the development of a medical device, specifically the t:slim insulin pump, to minimize the risk that the user interface design could lead to user errors, adverse events, and product recalls. Schaffer²² described the human factors design process as “being exemplified by three distinct phases: (1) preliminary analysis, (2) formative design evaluation and modification, and (3) design validation. Methodologies used include Web-based surveys, focus groups, usability studies, and finally a validation study to ensure the t:slim insulin pump was safe and effective for human use.” Schaffer²² argued that the most important benefit is increased safety, coming in the form of reduced risk through a design that has been heavily tested with representative users in its intended use environment in order to eliminate design flaws. Another benefit may be increased adherence, although this is not demonstrated in the current article. This process is important in the development of new AP devices.

Gonder-Frederick et al.²³ reviewed the psychological and behavioral considerations of closed-loop glucose control. The article aimed to “review psychological and behavioural factors that have influenced adoption and utilization of past technologies, to examine three theoretical frameworks that may help in contextualizing relevant patient factors in diabetes management, and to propose patient-section factors that will likely affect future closed loop glucose control studies.”²³ Technologies covered were insulin pump therapy, CGM, sensor-augmented insulin pump therapy, and closed-loop glucose control. Theoretical frameworks were the health belief model, the theory of planned behavior, and the diffusion of innovation theory. The authors summarized that research has focused primarily on the clinical objective. They also recognized that “this approach ignores the evidence that when PWD make decisions about adoption of new technology, these are rarely based solely on objective benefits. There are a number of important psychological and behavioural factors affecting decision-making that are often overlooked.”²³ They recommended studies involving psychological and behavioral processes addressing the impact of constructs such as satisfaction and coping styles.

Clarke and Renard²⁴ reviewed some of the clinical requirements for closed-loop systems, based on contemporary safety and efficacy data, and suggested possible ways in which new systems might be evaluated. The review focused on the medical and technical aspects of closed-loop but does include recommendation of inclusion of fear of hypoglycemia and diabetes QoL measures when considering clinical requirements for a closed-loop therapy system. The authors recommended longitudinal studies to track variables of clinical concern, frequencies of severe hypoglycemia, diabetes ketoacidosis, and measures of glycemic control.

Liberman et al.²⁵ presented a review of recently published articles (within the last year) on issues affecting the “human factor,” including decreased adherence, lack of motivation, and low QoL. Eleven articles were included, each summarized and commented on by the review authors. The authors concluded that “diabetes technology in general and continuous glucose meters in particular may become an obstacle for teens rather than a useful instrument.”²⁵ This is, they stated, because adolescents tend to “over-emphasize the demands and personal intrusions” caused by CGM while ignoring the benefits. The authors suggested developers of new technology remember that new technology should reduce the burden imposed on people having to live with it, particularly in the adolescent age group.

Additional data from randomized controlled trials and meta-analyses suggest that CSII use leads to improved glucose control,^26,27 although many individuals still experience hypoglycemia. In spite of an improvement in overall glycemic control using CSII, many individuals choose to discontinue pump use, and their glucose control returns to baseline.²⁸ Factors impacting insulin pump discontinuation include ongoing suboptimal glycemic control, poor body image, frustration at the daily demands of the diabetes regimen, higher diabetes-related emotional distress, and less frequent blood sugar monitoring.^29,30

Data from randomized controlled trials and meta-analyses suggest that CGM use leads to improved glucose control.^26,31 Benefits include the ability to predict glucose trends, the opportunity to correct out-of-range numbers, and the ability to understand the impact of food and exercise on glucose data.³² However, the alarms warning of low blood sugar levels are often not protective against lows as many people sleep through them,^33,34 so hypoglycemia remains an ongoing risk. Families may find that the quantity of information provided by the CGM overwhelming.^32,35 Data suggest that CGM can be perceived as a burden to PWDs and that many tend to lose interest over time.

Discussion

There is scant published literature on the psychosocial and human factors assessment of AP devices, with only four articles identified in the literature that contain specific data on the AP. Furthermore, there are currently large numbers of ongoing clinical trials, but few include psychosocial aspects of AP technologies alongside biomedical and engineering evaluation, and an accepted approach to psychosocial assessment of AP technology is lacking. Multiple tools exist to measure QoL and diabetes-related distress, but none specifically addresses the needs of AP as a novel technology. From a psychosocial perspective the AP could be viewed as a CGM device and insulin pump, and existing measures would be implemented, but the AP is unique in taking responsibility for glucose concentrations, and it is this transfer of trust, along with the burden of multiple devices, novelty, and day-to-day user requirement, that makes it unique and requires an additional component of assessment in a novel tool. No existing measures adequately address these important factors.

Regulatory and commissioning bodies such as the National Institute for Health and Care Excellence and the Food and Drug Administration expect psychosocial evaluation and outcomes to support the implementation of devices into routine care so it is, perhaps, surprising that these outcomes have been afforded little attention thus far. In any complex device development program the technical challenges must be overcome before implementation in subjects with type 1 diabetes affords the opportunity to assess human factors and effects on the lived experience. We are now at that stage in AP development, and it is becoming increasingly important to have psychological expertise and the experience of people with type 1 diabetes embedded in biomedical engineering teams. A lack of engagement with PWDs in research teams as part of a multidisciplinary expert group poses a risk of producing technologies that may be fit for “glycemic” purpose, but are unfit for “lived experience” purpose. Existing interventions for type 1 diabetes provide outcomes in clinical studies that are not replicated in clinical practice, with suboptimal glycemia being common, and it is critically important that barriers to extracting the maximum value from any diabetes therapy are considered throughout its development and design. Recognition of perceived barriers can aid in the design of interventions aimed at overcoming such barriers to encourage greater uptake and continued use of the available technologies, even before they can be perfected. The progress to date, although imperfect, allows for opportunities for patients with type 1 diabetes today to achieve greater time in target glucose range without severe hypoglycemia, which, in turn, will prevent complications and preserve health, thereby readying patients for future improvements.

There also remain marked geographical variations in usage of insulin pump therapies, suggesting different payers and clinicians may have different interpretations of the value and experience of this technology. Similarly, evidence from large registries of people with type 1 diabetes has shown that the use of CGM remains uncommon in general. In addition, there is a significant variation across different age groups, and discontinuation rates remain high.³⁶

It could be argued that, to date, AP clinical trial participants are not representative of the wider diabetes population, choosing to self-select for participation and having an interest in new technologies. This is undoubtedly true, but as studies become larger and are run for longer time periods, trial populations will become more representative, and QoL results will be comparable across studies. It will also be critical to understand which psychological factors and barriers are related to discontinued use of the AP system and/or suboptimal use of the system. If the most engaged participants are part of these early-phase studies, efforts at preventing problems with optimal use of AP technology will be key to promoting more widespread use.

A potential downside of developing new psychosocial measures specifically relevant to AP technology is that, although novel, they are yet to be validated. Validation demonstrating consistency and relevance requires significant numbers, and although existing measures are not wholly fit for purpose, work to develop a sensitive, specific tool to measure outcomes from AP studies must use data from existing tools to balance novelty and lack of validation. It must also consider future use in large studies that may focus on other outcomes such as cardiovascular or microvascular outcomes over a long period.

A better understanding of the psychosocial side of AP systems and the extent to which human factors play a role in the uptake and efficient use of these systems will ultimately lead to the most benefit for PWDs. By understanding this information through assessments, interviews, and focus groups, we can introduce targeted interventions to optimize the performance of the system for individuals and promote optimal health and psychosocial outcomes.

Author Disclosure Statement

No competing financial interests exist.