Participatory surveillance of diabetes device safety: a social media-based complement to traditional FDA reporting

Abstract

Background and objective

Malfunctions or poor usability of devices measuring glucose or delivering insulin are reportable to the FDA. Manufacturers submit 99.9% of these reports. We test online social networks as a complementary source to traditional FDA reporting of device-related adverse events.

Methods

Participatory surveillance of members of a non-profit online social network, TuDiabetes.org, from October 2011 to September 2012. Subjects were volunteers from a group within TuDiabetes, actively engaged online in participatory surveillance. They used the free TuAnalyze app, a privacy-preserving method to report detailed clinical information, available through the network. Network members were polled about finger-stick blood glucose monitors, continuous glucose monitors, and insulin delivery devices, including insulin pumps and insulin pens.

Results

Of 549 participants, 75 reported device-related adverse events, nearly half (48.0%) requiring intervention from another person to manage the event. Only three (4.0%) of these were reported by participants to the FDA. All TuAnalyze reports contained outcome information compared with 22% of reports to the FDA. Hypoglycemia and hyperglycemia were experienced by 48.0% and 49.3% of participants, respectively.

Discussion

Members of an online community readily engaged in participatory surveillance. While polling distributed online populations does not yield generalizable, denominator-based rates, this approach can characterize risk within online communities using a bidirectional communication channel that enables reach-back and intervention.

Conclusions

Engagement of distributed communities in social networks is a viable complementary approach to traditional public health surveillance for adverse events related to medical devices.

Introduction

Eleven per cent of the US population over 20 years old has diabetes,¹ a leading cause of cardiovascular disease, renal failure, and blindness. People with diabetes depend on medical devices, equipment, and supplies for glycemic control. Most people with diabetes use finger-stick devices and chemical test strips to check blood glucose levels and some add subcutaneous continuous glucose monitors to track interstitial fluid glucose levels. Of the 26 million Americans with diabetes, approximately 5% have been diagnosed with type 1 diabetes¹; an estimated 20–25% of this group uses an insulin pump rather than multiple daily injections to deliver insulin² and of those using injections, many use insulin pens rather than syringes and needles. Adverse events (AEs) associated with devices used to monitor and manage diabetes range from minor to severe, and include hyperglycemia and diabetic ketoacidosis. These conditions can be due to pump malfunction, faulty connections between the pump reservoir and tubing, tube kinking, infusion set leakage or extrusion from the skin (interrupting rapid-acting insulin delivery), moderate to severe infections at the pump or monitoring sites, and atrophy, hypertrophy, and bruising of the skin. Insulin, while life-saving, can cause severe harm when blood glucose levels drop below safe limits, including hypoglycemia associated accidents and injuries,³ missed work and lost productivity,⁴ debilitating worry for patients and their family members,⁵ medical complications and death.⁶ Diabetes-related devices either deliver insulin (insulin pumps and pens) or provide the data used to calculate insulin dose and frequency (glucometers and continuous glucose monitors), and errors in use can lead to both hyper- and hypoglycemia.

The FDA's Center for Devices and Radiological Health (CDRH) has been receiving voluntary reports of AEs related to glucose meters, test strips, and insulin pumps from both healthcare professionals and patients through the MedWatch program.^{7 8} Surveillance and reporting of AEs has evolved little since the 1960s, despite the emergence of new electronic channels for collecting patient reports of health problems. As a result, the FDA rarely hears directly from patients, the large majority of whom use their devices on a continuous basis.

Social media may support more rich, frequent, and timely reporting compared with mechanisms ‘filtered’ through health professionals or manufacturers. Hearing directly from patients is especially important for capturing issues related to ‘real world’ use of devices in the home setting.^9–12 Further, tapping into online communities for reporting may reach a more diverse and representative group of device users and provide information across the spectrum of events managed by patients that may potentially be severe yet underreported to manufacturers or the FDA.

To evaluate social media as a complementary source of information on device failures and AEs, we targeted a sample from a large online social network of people with diabetes who have been actively engaged in participatory surveillance.^{13 14}

Methods

Setting, subjects, and enrollment

The subject population were members of an online social networking community of people with diabetes and their caregivers or family members. TuDiabetes and its Spanish-language sister site EsTuDiabetes together have over 25 000 members; 59% are female and 51% have type 1 or 1.5 diabetes (also called latent autoimmune diabetes in adults or LADA). Median member age is 40 years. Study activities were approved by the Boston Children's Hospital Institutional Review Board. Eligible participants had internet access, were able to read and write English and/or Spanish, and were at least 13 years old (individuals aged 13–17 may join the site with consent of a parent/guardian, and from that point may participate in all site activities).

From October 2011 through September 2012, structured self-reports of AEs related to diabetes device use were gathered through a free, non-commercial software application (‘app’) called TuAnalyze that supports privacy-preserving participatory surveillance by members of the TuDiabetes.org international online diabetes social network.^{13 14} Subjects were polled about use of finger-stick blood glucose monitors and continuous glucose monitors as well as insulin delivery devices including insulin pumps and insulin pens. Summary data from member surveys were returned to the community through a TuDiabetes site research blog (the main dissemination vehicle) and through newsletters and postings on the site home page. Online consent for use of member-provided data for research is obtained when a member first accesses the app.

Measures

The survey was made available to all TuDiabetes members through the TuAnalyze app between October 2011 and September 2012. The survey was promoted to the site membership through multiple channels including email newsletters, site blog and forum postings, and a banner on the homepage. Respondents identified the diabetes devices they currently use, if any, and the brand, model, and length of use for each device. For each device, respondents were asked, ‘Using your current [device], have you ever had a severe health problem (such as a severe infection, extreme hypoglycemia or hyperglycemia, or seizure) you think was RELATED to the device?’ Respondents who experienced a severe AE were asked to report on the most recent event. Information collected on device-related AEs included a description of the event, health outcomes, assistance required in treating the event, contributing causes including device features, malfunctions, or user error, possible prevention or amelioration factors, and reporting of the event. Demographic data and familiarity with use of the FDA reporting system were collected from all survey participants.

AE descriptions, captured in free text only, were reviewed and categorized by two independent analysts, using a set of thematic codes developed using a shared, inductively derived scheme (codes were developed from initial joint review of the narrative data). Events could be assigned multiple codes; results were compared across analysts and discrepant codes were resolved or combined to create the final dataset. Most survey questions were multiple-choice response or check-all-that-apply, with an open-text option for ‘other’; a response was required to advance the survey. Free-text responses were reviewed and, when appropriate, assigned a coded value. The coded responses were categorized based on the free-text responses to provide additional insight into the reasons users chose a particular cause of the AE.

The FDA Manufacturer and User Facility Device Experience (MAUDE)¹⁵ AE reporting database was queried for events reported during the survey study period (October 2011 through September 2012) to determine the number of diabetes device-reported events and outcomes logged in a traditional data source.

Analyses

Descriptive statistics characterize the demographics of survey respondents, experiences of AEs, and event reporting. Due to the low number of AEs related to finger-stick blood glucose monitors (n=10 or 2.2% of 463 users) and insulin pens (n=6 or 5% of 119 users), stratified analyses of AE causes and outcomes for these devices were not reported; however, these reports were included in the overall AE summary and comparisons with MAUDE data. All analyses were performed in SAS V.9.3.

Results

A total of 549 users reported on their device use through the survey. Nearly 500 (486) users reported on blood glucose monitors through the survey and 442 on insulin delivery devices; 315 users reported on both. Demographic characteristics for the entire population are shown in table 1.

Table 1.

Demographics

	Total N=549
Characteristics	N (%)
Age in years
Mean (SD)	42.8 (15.9)
Gender*
Male	198 (46.2)
Female	231 (53.8)
Diabetes type
Type 1/LADA	425 (78.6)
Type 2/Pre-diabetes	116 (21.4)
Insulin use*
Yes	376 (87.7)
Time since diabetes diagnosis
<1 year	52 (9.5)
1–2 years	64 (11.7)
3–4 years	58 (10.6)
>5 years	375 (68.3)
HbA1c†
>7%	234 (65.7)
≤7%	122 (34.3)
Educational level
High school and lower	131 (23.9)
Vocational training	70 (12.8)
Bachelor's degree or higher	348 (63.4)
Devices used‡
Finger-stick blood glucose monitor	463 (95.3)
Continuous glucose monitor	170 (35.0)
Insulin pen	119 (26.9)
Insulin pump	276 (62.4)

*Total N=429; gender and insulin use not available for the full set of respondents.

†Total N=356; HbA1c% not available for the full set of respondents.

‡More than one device could be reported; percentages do not add up to 100.

Distribution of device use

Finger-stick blood glucose monitors were the most commonly used device, reported by 95.3% of those who reported about glucose monitor use (table 1). Continuous glucose monitor use was less common, reported by 35.0% of survey respondents. Nearly two-thirds of respondents reporting an AE related to an insulin delivery device used an insulin pump.

Frequency of adverse events

The frequency of AEs was lowest among blood glucose monitors, with only 10 out of 463 users (2.2%) reporting any AE related to their blood glucose monitor. Of those who used a continuous glucose monitor, 24 (14.1%) had experienced an AE. Among survey participants using an insulin pump, 55 out of 276 users reported an AE related to the device; however, in an analysis of the narrative reports of these AEs, over one-third of the events described (36.4%, or N=20 AEs) were hypoglycemia as a result of insulin use, due to the user miscalculating carbohydrates consumed, not having fast-acting carbohydrates on hand to treat mild hypoglycemia, or a low blood sugar that was not recognized and treated due to hypoglycemia unawareness. These events were determined to be associated with insulin use in general and not specific to the use of a device, and were therefore excluded from analyses of AEs, leaving a total of 35 (12.7% of device users) insulin pump or infusion set AEs (table 2).

Table 2.

Adverse events (AEs): total, continuous glucose monitor (CGM), and insulin pumps/infusion sets

	Total, n (%)	CGM, n (%)	Pump/infusion set, n (%)
Device user
Yes	549 (100)	170 (35.0)	276 (62.4)
AEs
Yes	75 (13.7)	24 (14.1)	35 (12.7)
Number of AEs/past 12 months
0–1	26 (34.7)	8 (33.3)	14 (40.0)
2–3	27 (36.0)	9 (37.5)	13 (37.1)
4+	22 (29.3)	7 (29.2)	8 (22.9)
Cause of AE*
Device error	41 (54.7)	20 (83.3)	12 (34.3)
User error	19 (25.3)	5 (20.8)	9 (25.7)
Health problem experienced†
Hypoglycemia	36 (48.0)	18 (75.0)	4 (11.4)
Hyperglycemia	37 (49.3)	7 (29.2)	25 (71.4)
Infection	7 (9.3)	2 (8.3)	4 (11.4)
Diabetic ketoacidosis	9 (12.0)	0 (0)	8 (22.9)
Needed help to manage AE
Yes	36 (48.0)	12 (50.0)	17 (48.6)
Reported adverse event
Yes	40 (53.3)	9 (37.5)	26 (74.3)
Event reported to*
Healthcare provider	24 (32.0)	8 (33.3)	12 (34.3)
Manufacturer	21 (28.0)	6 (25.0)	14 (40.0)

*Continuous glucose monitor.

†Multiple responses allowed.

Causes of adverse events related to the continuous glucose monitor

Of the 24 users reporting a continuous glucose monitor-related AE, 66.7% cited incorrect readings (eg, a reading inconsistent with a finger-stick monitor, or severe hypoglycemia not recognized by the continuous glucose monitor), followed by problems with the sensor or transmitter (20.8%). Nearly one third of continuous glucose monitor users (29.2%) reported that they were unsure as to the cause. Three out of four continuous glucose monitor AEs resulted in hypoglycemia, and infections made up 8.3% of events related to continuous glucose monitor use. Five of the 24 continuous glucose monitor users (20.8%) reported that additional training or information on the device could have prevented or reduced the severity of the event.

Causes of insulin pump related AEs

Of the 35 insulin pump users who experienced an AE, the most common reasons cited for the event were problems with the infusion set or cannula (60.0%), inaccurate delivery of insulin (22.9%), or that the device stopped working (17.1%). Some users (8.6%) experiencing a pump AE were unsure what happened and 8.6% of AEs were infections. Of those experiencing a pump AE, 64.3% reported that they understood ‘all’ of the features of their pump, with the remainder reporting that they understood ‘most’ of the features. Only 14.3% of users experiencing a pump/infusion set AE thought that it could have been prevented or ameliorated by additional training or information on the device.

Remedial actions and reporting

Across all 75 reported device AEs (associated with insulin pumps, continuous glucose monitors, blood glucose monitors, and insulin pens), nearly half of users (48.0%) required help or intervention from another person to manage the event. Slightly over half of AEs (53.3%) were reported to any other individual or entity. Events were most commonly reported to a healthcare provider (32.0% of events) or the device manufacturer (28.0%). Less than half of all survey respondents (37.3%) were aware of the MedWatch system and among those, only 5.7% had ever used it to report a device error. Of the 75 total AEs reported in the survey, only 3 (4.0%) were reported to the FDA.

AEs reported to the FDA

From October 2011 through September 2012, 20 423 diabetes device AE reports were submitted to MedWatch. Nearly all (99.9%) reports were from manufacturers; only 23 reports (0.1%) were voluntarily submitted by users. Event date was available for 8100 reports (39.7% of the total) and patient outcome information was available for 4579 (22.4%). Of these events, 49% required intervention and 24% required hospitalization (table 3).¹⁵

Table 3.

Adverse events reported to FDA* and to TuAnalyze†

	FDA, N (%)	TuAnalyze, N (%)
Total events	20 423	75
Device malfunction	15 795 (77.3)	39 (52.0)
User injury	4408 (21.6)	3 (4.0)
Type of device
Blood glucose monitor	13 108 (64.2)	10 (13.3)
Continuous glucose monitor	1210 (5.9)	24 (32.0)
Insulin pump	7260 (35.5)*	35 (46.7)
Insulin pen	21 (0.1)	6 (8.0)
Interval between event and report available	8100 (39.7)	75 (100)
Less than 4 weeks	6063 (74.9)	32 (42.7)
1–6 months	1906 (23.5)	23 (30.7)
Greater than 6 months	131 (1.6)	20 (26.7)
Events with patient outcome information	4579 (22.4)	75 (100)
Required intervention	2225 (48.6)	40 (53.3)
Required hospitalization	1090 (23.8)	5 (6.7)

*U.S. Food and Drug Administration.

†For FDA data available in the Manufacturer and User Facility Device Surveillance (MAUDE) database, devices that include an insulin pump and integrated continuous glucose monitors are counted as both types of device.

Discussion

The TuDiabetes community actively engaged in participatory surveillance, rapidly defining a range of patient-experienced AEs in their population. Participation is consistent with previous reports showing that patients are willing to share their health data through this electronic medium.^16–19

The TuAnalyze participants consistently reported outcome data only to one in five reports to the traditional FDA system. Further, the participatory surveillance approach has captured a wider range of AEs, including less severe events given the much lower rate of hospitalization compared with reports in the MAUDE database. These milder events, though not resulting in healthcare visits, may contribute to poor glycemic control, decrease quality of life, and increase personal and family distress. Using a participatory online patient surveillance system as an adjunct reporting source offers the potential to track the incidence of more mild events and allows the FDA to monitor trends in reporting and, thereby, the potential to alert manufacturers, clinicians, and patients about risk factors for particular device malfunction.

Prevention of the acute and chronic complications of diabetes depends on patients using devices to achieve normoglycemia. Failure and improper use of these devices, particularly the insulin pump, a higher risk medical device, can result in hyperglycemia and diabetic ketoacidosis or hypoglycemia with potentially life threatening sequelae. MedWatch, the traditional FDA post-marketing device safety surveillance system, does address insulin delivery and glucose monitoring devices but has low levels of reporting.^{9 10 15 20–22} Consequently, the FDA's CDRH is exploring enhancements, including consumer-driven approaches to AE reporting.¹²

Approximately half of the AEs captured through the social network were reported outside the network, most commonly to a medical provider or the device manufacturer, though very few were reported directly by participants to the FDA. Presumably, many of the events reported to manufacturers were reported to the FDA as well, though linking AEs across MAUDE and the TuDiabetes survey is not possible as publically available MAUDE reports are anonymous.

Social network-based reporting systems may help to address the low level of patient awareness of and engagement in AE reporting, both by providing a reporting channel highly visible to engaged patients, and by presenting a more patient-focused reporting mechanism. Further, within current reporting systems, there are no ready channels for contacting the reporters. Social media offers the opportunity to reach out to new populations, reduce barriers to reporting, and engage the FDA and other public health agencies—such as the Centers for Disease Control and Prevention and local departments of health—in bidirectional communication with reporters. Social media creates the opportunity to feedback results to communities and elicit additional information, and potentially intervene and remediate identified problems. With 800 million users of Facebook and large numbers of users of health-oriented social networks such as TuDiabetes, PatientsLikeMe, and others, social media allows feedback of results to users and communities, and also follow-up queries after an initial report.

This bidirectional nature of social media also provides a channel whereby reporters can be engaged to provide further information, and potentially mitigate the safety threat. In its call for development of consumer-driven patient safety reporting systems, the Agency for Healthcare Research and Quality calls for linkage between reporting mechanisms and safety and improvement efforts.²³ Social media-based programs allow development of rich and targeted marketing of AE reporting programs as well as online community or individual-level feedback and interventions. Online, disease-focused social networks like TuDiabetes capacitate ready identification of populations likely to be using devices or drugs under surveillance, and social networking can enhance reach directly to individuals.

Our TuAnalyze poll was available to participants only for a limited time, and therefore subjects were asked to recall past events. Hence, our study design did not enable robust demonstration of the timeliness of reporting and recall bias may be a factor. However, if the system were made continually available and were widely publicized, rapid reporting could become normative after AEs.

Like MedWatch, TuAnalyze does not currently capture a denominator for its reporting. The TuAnalyze survey population is also engaged and highly experienced, the majority having had diabetes for over 5 years. While participatory surveillance of distributed online populations does not yield generalizable, denominator-based rates, it can characterize risk within online communities using a bidirectional communication channel that enables reach-back, feedback, and intervention. This novel channel can be used to recruit new reporters for traditional reporting mechanisms, such as MedWatch, an extremely useful, but also non-denominator-based, AE tracking system.

Further, there are emerging approaches to link denominator-based networks to social media. Resnic calls for collection of high quality, denominator-based data for device surveillance using registries.²² There is an emerging paradigm for connecting patients to patient-based registries. For example, the Helmsley Foundation has created the T1D Exchange,²⁴ a clinic-based registry for patients with type 1 diabetes, with a linked patient portal. For pediatric inflammatory bowel disease and rheumatologic disease, respectively, the ImproveCareNow²⁵ and Childhood Arthritis and Rheumatology Research Alliance²⁶ registries are developing social media linkages among patients and providers.

Conclusion

We demonstrate a complementary approach to current FDA reporting systems for detecting AEs related to diabetes management devices that harnesses the engagement of a distributed population of individuals participating in an online diabetes social network. In addition to relying on a system like TuAnalyze in isolation, a hybrid approach may also work wherein individuals who self-identify through social media are directed to online, smartphone-based, or even paper-based FDA MedWatch reporting systems. Further development of these approaches may advance the goal of the FDA's CDRH to identify new, creative ways of learning about and ameliorating AEs by reaching those patients, consumers, and caregivers who use products intended for outpatient or home use, to better understand healthcare product quality and patient-centered factors that may contribute to user errors and harms.