Use of an Interactive, Telephone-based Self-management Support Program to Identify Adverse Events Among Ambulatory Diabetes Patients

Abstract

BACKGROUND

There is growing interest in the use of interactive telephone technology to support chronic disease management. We used the implementation of an automated telephone self-management support program for diabetes patients as an opportunity to monitor patient safety.

METHODS

We identified adverse and potential adverse events among a diverse group of diabetes patients who participated in an automated telephone health-IT self-management program via weekly interactions augmented by targeted nurse follow-up. We defined an adverse event (AE) as an injury that results from either medical management or patient self-management, and a potential adverse event (PotAE) as an unsafe state likely to lead to an event if it persists without intervention. We distinguished between incident, or new, and prevalent, or ongoing, events. We conducted a medical record review and present summary results for event characteristics including detection trigger, preventability, potential for amelioration, and primary care provider awareness.

RESULTS

Among the 111 patients, we identified 111 AEs and 153 PotAEs. Eleven percent of completed calls detected an event. Events were most frequently detected through health IT–facilitated triggers (158, 59%), followed by nurse elicitation (80, 30%), and patient callback requests (28, 11%). We detected more prevalent (68%) than incident (32%) events. The majority of events (93%) were categorized as preventable or ameliorable. Primary care providers were aware of only 13% of incident and 60% of prevalent events.

CONCLUSIONS

Surveillance via a telephone-based, health IT–facilitated self-management support program can detect AEs and PotAEs. Events detected were frequently unknown to primary providers, and the majority were preventable or ameliorable, suggesting that this between-visit surveillance, with appropriate system-level intervention, can improve patient safety for chronic disease patients.

INTRODUCTION

Given the increasing complexity of ambulatory chronic disease management,¹ people with diabetes often must independently complete complex tasks to manage their health.² Although ambulatory visits constitute the majority of medical care encounters, little is known about patient safety in the ambulatory setting.^3–5

Prior work in ambulatory patient safety has focused largely on adverse drug events^6–8 and medical errors.^9–12 These studies have described errors following hospital discharge or identified in malpractice claims in ambulatory settings^12,13 or events identified at a point-of-care site, such as those resulting in emergency department visits.¹⁴ Some studies have used incident reporting in ambulatory care settings to identify adverse events (AE),^{10,11,15–17} but incident reporting captures a relatively small subset of AEs.^15–19 We are not aware of observational studies that examine safety specifically in patients’ most familiar environment—at home, between visits.

Across acute-care settings, health IT has been widely considered as a critical tool to monitor and improve patient safety. Indeed, technology-driven solutions, such as computer physician order entry and computer medication monitoring, are integral to patient safety guidelines.^20,21 The extent to which health IT improves patient safety, however, remains unclear. Whereas health IT has been shown to facilitate detection of AEs through reporting and administrative data,^22–27 both over- and underdetection of safety threats occur,^28,29 and clinical outcomes may not improve.^27,30–32 Most research and quality improvement efforts to monitor or improve patient safety have targeted health IT to health systems and/or providers rather than to patients directly.

Ambulatory diabetes care provides an exemplary model to better understand chronic disease safety. Diabetes is a communication-sensitive disease, requiring patient and provider collaboration to optimize self-management and avoid complications³³; communication deficiencies can contribute to suboptimal quality of care.^34–36 Moreover, because patients must perform daily self-management behaviors, much of diabetes care occurs outside of the clinical setting. To address patients’ self-management needs beyond the clinical encounter, some health systems, including the one in which we work, have developed mechanisms to communicate with diabetes patients at home through innovative health IT strategies.^37–40 The surveillance of these communication encounters can provide a unique opportunity to describe AEs and potential adverse events (PotAE) that arise between visits.

In the course of implementing an automated telephone self-management (ATSM) support program for patients with diabetes,⁴¹ nurse care managers identified that patients were reporting not only self-management difficulties, but also AEs and PotAEs. We reviewed and categorized previously collected clinical data to describe and characterize the AEs detected among diabetes patients between office visits using ATSM surveillance.

METHODS

Setting and Patients

This study was nested within the Improving Diabetes Efforts Across Language and Literacy (IDEALL) project,⁴¹ a 9-month, 3-arm randomized trial of 2 diabetes self-management support interventions compared to usual care. The project took place within the integrated delivery system of the San Francisco Department of Public Health. Patients were eligible for IDEALL if they were over age 17; had ICD-9 codes consistent with type 2 diabetes; spoke English, Spanish, or Cantonese; made ≥1 primary care visit in the prior year; and had a hemoglobin A1c value ≥8.0% at the time of recruitment. We examined patient safety among the 111 patients who participated in the ATSM intervention. The Committees on Human Research at the Community Health Network of San Francisco and the University of California, San Francisco, approved the study protocol.

Automated Telephone Self-Management Support

Automated telephone self-management (ATSM) is a health IT application that provides patient education and self-management support.^39,42–47 The IDEALL ATSM intervention included weekly interactive, automated telephone calls to patients, with review and follow-up by a nurse care manager.⁴⁸ Calls consisted of a rotating set of questions in patients’ preferred language of English, Spanish, or Cantonese. In a typical ATSM encounter, a patient would use the telephone touch-pad to respond to an automated prompt, such as, “in the last 7 days, how many days did you check your blood sugar?” The ATSM application generated daily written output of patient responses. The nurse care manager reviewed all responses and followed up via telephone with those patients whose responses triggered a callback based on preset criteria. She completed a progress note that provided documentation for each telephone encounter. The primary goal of the ATSM system was supporting patient self-management and behavior change, not safety monitoring or medication intensification.

Event Detection

Events could come to light in 3 ways. First, when a patient entered a response into the ATSM system that met a predetermined threshold, we categorized that to be a health IT–triggered event. For instance, a patient-entered blood glucose value of less than 60 mg/dl constituted a health IT trigger. Second, a patient could request a callback from the nurse during any ATSM call, providing an additional opportunity to report an event. Events uncovered following a patient callback request were categorized as patient-triggered. Finally, patients could report an unrelated event during a live telephone nurse encounter. For example, during a call about exercise, a nurse could elicit that a patient did not have his diabetes medicines; we categorized this as a nurse-triggered event.

Because we could not characterize events if the nurse had been unable to reach the patients by telephone, we excluded those candidate events. In addition, we did not include events if the nurse care manager spoke with the patient but did not explicitly document confirmation of the possible event.

Ambulatory Taxonomy and Event Ascertainment

Although several medical error taxonomies exist,^{6,7,13,49–51} we found that they were either designed for acute care settings⁴⁹ or for ambulatory incident report systems.^10,52 Our model, with longitudinal surveillance among a cohort of diabetes patients, required that we adapt existing taxonomies to both capture problems resulting from self-management of chronic diseases specifically and to describe events as they developed over time. We distinguish between incident events, those that are new to the patient, and prevalent, or ongoing, events. If the same event persisted across multiple ATSM calls without resolution, we considered it to be 1 event. However, if an event completely resolved and then recurred during the study period, we considered each as a separate event. We adapted established definitions of preventability, ameliorability, and harm from prior patient safety studies,^6,7,13 and tailored them to reflect patients’ active roles in managing diabetes. Selected elements of the taxonomy are listed in Table 1.

Table 1.

Elements of Taxonomy for Ambulatory Adverse Events

Element	Classification
Event type*	AE: an injury, with varying possible levels of harm, which results either from medical management or patient self-management
	PotAE: an unsafe state, not currently an AE, but likely to lead to an AE if it persists without intervention
Time-frame*	Incident: new to the patient
	Prevalent: ongoing in the patients’ life
Trigger*	“Out-of-range” ATSM response
	Patient requested a callback
	Nurse care manager elicitation
	None
	Unable to determine
Preventability*	Preventable: could have been avoided, probably a result of an error or system design flaw, or could have been avoided with a patient action that is reasonable to expect
	Ameliorable: (AEs only) severity could have been substantially reduced with different actions or procedures (on the part of patient, provider, or system)
	Nonpreventable, nonameliorable
	Unable to determine
Primary provider awareness*	Aware in real time or prior to the study nurse becoming aware
	Unable to determine
	Unaware
Description*	Hyperglycemia
	Hypoglycemia
	Glucose monitoring
	Medication problem
	Other physical complaint/symptom
	Other
	Unknown
Self-management domain*	Glucose monitoring
	Medications
	Foot care
	Screening (non-receipt)
	Diet
	Exercise
	Diagnosis (missed or incorrect)
	Other
	Unable to determine
Harm†	Not applicable—PotAE without harm
	Laboratory abnormalities only
	1 day of symptoms
	Several days of symptoms
	Nonpermanent disability (including hospitalization)
	Permanent disability
	Death
Health care utilization‡	Nurse–patient telephone contact
	Nurse–pharmacy telephone contact
	Nurse–PMD telephone contact
	Additional laboratory monitoring
	Nurse in-person visit
	Primary care office visit/ urgent care visit
	Specialist physician visit
	Emergency visit
	Hospitalization
	911 (emergency medical services)
	Unknown
	None

AE = adverse event, PotAE = potential adverse event

*Coded with 1 best response

†Coded as 1 response, for most severe harm

‡Coded as all responses that apply

Previous studies have defined an AE as an injury, with varying levels of harm, which results from medical management rather than the natural history of disease^6,7,13 and a PotAE as an error with the potential to cause harm.^53,54 In the context of chronic disease management, we did not feel we could or should separate medical management from patient self-management. Problems with self-management could reflect inadequate patient education on the part of providers or patient errors, which also contribute to AEs.⁵⁵ Our study definitions are listed in Table 1.

To operationalize these definitions, we convened a consensus group of providers, involving 4 primary care physicians and a diabetes specialist. Because the definitions for AEs and PotAEs were general rather than specific, we developed several clinical thresholds a priori. For glucose abnormalities, if patients reported a glucose level ≤60 or ≥300 mg/dL, this was classified as an AE, regardless of whether the patient experienced any symptoms. The consensus group included these glucose thresholds because they believed that values in these ranges would warrant notification to the physician prior to the next visit and because some patients with long-standing diabetes may not experience symptoms from glucose abnormalities.⁵⁶ Because a blood glucose level ≥300 mg/dL exceeds the renal threshold for glucose excretion and causes physiologic changes such as osmotic diuresis,⁵⁶ the consensus group chose this cutoff for hyperglycemia.

In addition, the consensus group set thresholds for several types of PotAEs. The clinician panel deemed less than 50% adherence to diabetes and cardiovascular medications to be a clinically significant PotAE. This threshold is lower than both optimal and average adherence for other chronic diseases.^57,58 Finally, the consensus group determined that being on a hypoglycemic agent and not having a functional glucometer, and thereby not being able to check ones’ blood glucose at home, represented a PotAE. We consulted with 2 external patient safety experts who helped us refine our taxonomy, definitions, and thresholds and approved our final method for event ascertainment.

Using the definitions and thresholds, 2 physicians independently reviewed all ATSM call records and ATSM progress notes to identify candidate events. In the first phase of the study, 85 candidate events (33% of final total) were presented to the consensus panel for review and refinement of criteria. After this, the 2 study physicians identified remaining events independently and reviewed these events jointly to categorize candidate events. Any disagreements were brought to the consensus panel for final determination. We defined an event as an AE or PotAE only if the 2 reviewers agreed.

Event characterization and classification

Following event ascertainment via ATSM data, the 2 physicians reviewed all other patient-related documentation, including the permanent medical record from 3 months before, 9 months during, and 3 months after the intervention period. They applied the patient safety taxonomy described above to characterize each event. To explore the incremental yield of this method of surveillance over usual ambulatory care, they examined the ATSM data and medical records to determine whether the physicians had been aware of events. To ensure consistent coding, the study physicians reviewed all coded events jointly. If the medical record did not allow the coders to determine aspects of the taxonomy, the field was coded as “unable to determine.”

Data Analysis

To better characterize the yield of ATSM as a surveillance system, we calculated the proportion of completed ATSM outgoing telephone calls that resulted in event detection overall and according to the triggers for detection. We present below summary results for event characteristics, including the frequency of incident and prevalent AEs and PotAEs, as well as preventability and primary care provider awareness of events. Also, because the glycemic thresholds for AEs were inherently somewhat subjective, we performed a sensitivity analysis of AE frequency in 2 ways: (1) we increased the AE glucose threshold for asymptomatic hyperglycemia to >400 mg/dL and (2) we recategorized all asymptomatic hyperglycemia as PotAEs, instead of AEs. A subsequent study describes the self-management domains from which events originated and explore the events’ contributing causes.

RESULTS

The 111 patients who participated in at least 1 ATSM call were ethnically diverse, and most participants had long-standing diabetes. The average hemoglobin A1c was 9.3% at enrollment, and 54% had limited health literacy as determined by the Test of Functional Health Literacy for Adults⁵⁹ (English- and Spanish-speaking patients only). On average, patients completed 16 ATSM calls and received a mean of 10 callbacks from the bilingual nurse care manager over the 39-week program (Table 2).

Table 2.

Baseline Patient Characteristics, N = 111

	N/mean	%/SD
Age	55.3	12.8
HbA1c	9.3	1.8
Female	65	58.6
Health literacy*
Limited	52	53.6
Adequate	45	46.4
Language†
English	51	45.9
Spanish	46	42.3
Cantonese	13	11.7
Insulin use	40	36
Oral medications	100	90
Poor or fair health status	82	74
Number of completed ATSM calls	16	9
Number of nurse callbacks	10	7

HbA1c = hemoglobin A1c, ATSM = automated telephone self-management

*Health literacy was calculated using the short form of the Test of Functional Health Literacy for Adults (s-TOFHLA) for English and Spanish speakers (N = 97): limited = 0–22; adequate = 23–36. The s-TOFHLA is not available in Cantonese

†This refers to the language in which the intervention was delivered to the patient

The ATSM intervention generated 2,318 completed automated calls among the 111 patients. We detected 264 confirmed events; 11% of completed ATSM calls resulted in event-detection. In addition to these 264 confirmed events, 8 possible events were excluded because the ATSM nurse could not contact the patient and 12 possible events were excluded because they were not confirmed during subsequent ATSM nurse callbacks. Ninety-three patients (84%) experienced at least 1 event, with a mean of 2.4 events per patient (SD 1.8, range 0–8).

The 264 events were detected as follows: 158 (59%) were health IT–triggered, 28 (11%) were patient-triggered, and 80 (30%) were nurse-triggered. Two events had 2 triggers each, a health IT and a patient trigger. Example events are described in Table 3. The distribution of events included 32 (12%) incident AEs, 79 (30%) prevalent AEs, 52 (20%) incident PotAEs, and 101 (38%) prevalent PotAEs.

Table 3.

Selected Examples of Events

Type	Example
Incident AE	Hypoglycemia with symptoms following dose increase of 1 oral diabetes medicine and addition of 2 new oral diabetes medicines at previous ambulatory physician visit
	Dysuria, urinary urgency, and frequency consistent with a urinary tract infection; patient did not report symptoms to provider or seek treatment
Prevalent AE	Hyperglycemia with polyuria and polydipsia related to incorrect medication use; patient does not understand medication dosing
	Ongoing gastrointestinal distress when taking metformin
Incident PotAE	Patient on insulin unable to check blood glucose because of glucometer malfunction
	Despite physician prescription, pharmacy does not dispense sufficient medication quantity for patient traveling for 6 weeks, leading to nonadherence
Prevalent PotAE	Patient mistakes acetaminophen for metformin and has not taken metformin for 6 months; did not receive pharmacy or provider medication review
	Patient nonadherent to statin because of concern about possible liver toxicity

AE = adverse event, PotAE = potential adverse event

The majority of all types of events were deemed to be preventable or ameliorable. Thirty-four percent of incident AEs, 87% of prevalent AEs, 96% of incident PotAEs, and 99% of incident PotAEs were preventable. An additional 34% of incident AEs and 8% of prevalent AEs were considered ameliorable (Fig. 1). Compared to incident events, a higher proportion of prevalent events were preventable or ameliorable (97 vs 87%, p = .01).

Figure 1.

Degree of preventability and ameliorability of events (by event type). Most events were preventable, and the proportion of preventable events was higher for prevalent than for incident events (p < .01)

Overall, we determined that primary care physicians were aware of fewer than half of events (N = 122, 46%) at the time that the nurse care manager became aware of events. Primary care providers were less likely to have been aware of incident versus prevalent AEs [28 vs 67%, odds ratio (OR) = .19, CI 0.08–0.47], and incident versus prevalent PotAEs (10 vs 53%, OR = .09, CI 0.03–0.25).

Using the threshold of blood glucose ≥300 mg/dL, we found that 1 of 32 (3%) incident AEs and 33 of 79 (48%) prevalent AEs were related to asymptomatic hyperglycemia. If we increased the threshold for an asymptomatic hyperglycemia AE to ≥400 mg/dL, the number of incident AEs decreased to from 32 to 31, and prevalent AEs decreased from 79 to 60. If we reclassified all asymptomatic hyperglycemia as PotAEs, there would be 31 incident AEs, 46 prevalent AEs, 53 incident PotAEs, and 134 prevalent PotAEs.

DISCUSSION

This telephone-based health IT–facilitated self-management support intervention uncovered a significant number of AE and PotAEs among diabetes patients cared for in an urban safety-net setting. Both the surveillance method, which used patient-responsive health IT to identify AEs, and the study findings, of frequent and preventable events, provide important insights and implications for patient safety in the ambulatory setting.

By harnessing information from a system primarily designed for self-management support, we detected 1 AE or PotAE for every 9 completed ATSM calls. Although we cannot directly compare it to other methods of event detection, our surveillance strategy detected an especially high frequency of events. It is possible that between-visit surveillance captures more events than previously reported because such events may escape notice during time-pressured ambulatory encounters or because the threshold for clinician reporting and documentation of events is higher.¹⁷ As interactive telephone disease-management technology becomes more widely used across clinical settings, health systems can explore its utility to monitor patients for AEs. In fact, a pilot study recently concluded that interactive telephone technology may be feasible for post-hospital discharge monitoring.⁶⁰ Further research should explore whether such telephone-based, between-visit surveillance can mitigate or prevent AEs.

Because ATSM programs can be readily adapted and disseminated, this method has the potential to increase the reach of ambulatory safety surveillance. The technology can efficiently engage a large patient population, including those with limited English proficiency and limited health literacy,⁶¹ and then use patient responses to appropriately target nurse follow-up. In our sample, the ATSM system triggered detection of 59% of events, which implies that appropriate prespecified thresholds can identify quality problems that require intervention. Moreover, the vast majority of participants experienced at least 1 event, suggesting that broad-based safety surveillance may be an important component of chronic disease self-management programs, particularly among vulnerable patients.

Our findings both confirm and extend prior data on ambulatory AEs. As in prior studies, we found that AEs and PotAEs are not uncommon in the ambulatory setting,^6,7,9 and that PotAEs are more common than AEs.^62,63 However, we found a higher proportion of preventable and ameliorable events than prior studies of ambulatory adverse drug events^6,7 and of AEs in the post-hospital discharge period.¹³ This suggests that a between-visit, health IT–assisted surveillance mechanism may detect different events than previously described, and it may detect events in time to mitigate harm to patients.

Prior studies have not distinguished between new and ongoing AEs, but we detected more prevalent than incident events. This suggests that quality problems in the outpatient setting may be persisting for significant periods of time; effective ambulatory surveillance may require that patients be observed longitudinally. Finally, we found that primary care physicians were largely unaware of incident AEs and incident PotAEs. This lack of physician awareness may contribute to delayed or missed diagnoses,⁶⁴ an important subset of ambulatory errors.¹² Although we could not measure whether physicians, in the absence of the ATSM program, would have learned of these events later, through other means, the lack of awareness of over one third of prevalent AEs and one half of prevalent PotAEs suggests that health IT–facilitated surveillance can uncover events unknown to clinicians and perhaps develop timely responses.

Our sensitivity analysis showed that a lower threshold is more sensitive for detection of AEs, although there may be more “false positives,” e.g., participants who do not require further close observation, at this level. When the AE threshold for asymptomatic hyperglycemia was increased to 400 mg/dL, or when all such events were reclassified as PotAEs, there was still a significant number of AEs and PotAEs. This suggests that the AEs experienced by this population extend beyond poorly controlled diabetes. In the setting of ATSM surveillance for diabetes, we advocate using a hyperglycemia threshold of >300 mg/dL, as this lower threshold triggers a subsequent live contact with the care manager. At the time of follow-up, the care manager can better assess the patient’s self-management ability and determine whether there is an urgent threat to safety.

Despite its strengths, this study has several limitations. First, because our sample was a socioeconomically disadvantaged, racially/ethnically, linguistically diverse group with poor health status, our findings may not be generalizable to all ambulatory diabetes patients, and this may partly explain the ubiquity of events. Second, we harnessed a health IT–facilitated self-management support application to explore patient safety. The system was not designed to systematically identify threats to patient safety, nurses were not specially trained or instructed to identify or elicit threats to patient safety, and patients varied in their degree of engagement with the system.⁶¹ As such, we may have underestimated the number of events, as is true for other methods of event detection/reporting.^{15–17,19,65} Third, because the ATSM protocol was designed to target participants with out-of-range responses, patients who reported AEs were more likely to receive nurse follow-up. The differences in surveillance times and follow-up probability prevented our being able to calculate event rates. Fourth, event ascertainment by review requires clinical judgment^66,67 and can vary from clinician to clinician. Similar to prior studies, we used a 2-physician review model.^{13,65,68–70} We attempted to standardize event definitions and thresholds, reviewed one-third of candidate events with the consensus panel, and performed a sensitivity analysis to examine the effect of glucose thresholds on our results. Future work in ambulatory patient safety research will need to arrive at common definitions and validate thresholds. Finally, our AE definition included physical harm only. We plan to include emotional and financial harm in future prospective work.

The growing application of health IT–facilitated self-management support tools provides an opportunity to better understand and possibly promote ambulatory patient safety. Automated telephone self-management support provides a scalable model with broad reach to monitor patient safety. Our findings suggest that it may be useful to employ such surveillance both for individual patients and to inform targets for systems-directed solutions to improve ambulatory safety.