Real-time clinical note monitoring to detect conditions for rapid follow-up: A case study of clinical trial enrollment in drug-induced torsades de pointes and Stevens-Johnson syndrome

Abstract

Identifying acute events as they occur is challenging in large hospital systems. Here, we describe an automated method to detect 2 rare adverse drug events (ADEs), drug-induced torsades de pointes and Stevens-Johnson syndrome and toxic epidermal necrolysis, in near real time for participant recruitment into prospective clinical studies. A text processing system searched clinical notes from the electronic health record (EHR) for relevant keywords and alerted study personnel via email of potential patients for chart review or in-person evaluation. Between 2016 and 2018, the automated recruitment system resulted in capture of 138 true cases of drug-induced rare events, improving recall from 43% to 93%. Our focused electronic alert system maintained 2-year enrollment, including across an EHR migration from a bespoke system to Epic. Real-time monitoring of EHR notes may accelerate research for certain conditions less amenable to conventional study recruitment paradigms.

INTRODUCTION

Identifying acute events such as adverse drug events (ADEs) as they occur can be challenging in large hospital systems. Electronic health records (EHRs) have been used to identify ADEs retrospectively using a combination of approaches such as billing codes, medication exposures, and natural language processing.^1,2 These efforts have been used to support clinical, population, and genomic research, but in general, their methods, often relying on custom-built research databases,³ are not amenable to real-time identification of patients with given conditions. Drug-induced torsades de pointes (diTdP) and Stevens-Johnson syndrome (STS) and toxic epidermal necrolysis (TEN) are 2 ADEs for which genetic studies have required multinational efforts.^4,5 Besides the rarity of each ADE, the opportunity to gather a disease sample is limited by disease acuity and mortality. Mean adjusted mortality for SJS/TEN in the United States has been estimated as high as 19.4%,⁶ and blister fluid disappears within 48 hours. Likewise, mortality among patients with diTdP is high, owing to sudden cardiac death.⁷ Retrospective capture of diTdP by billing codes is not possible without additional efforts to identify non-ADE phenotypes, including congenital long QT syndrome.⁸ We describe here a text processing tool that we have used for recruitment in 2 prospective clinical studies of rare, life-threatening conditions: diTdP and SJS/TEN.

This article presents a description of the tool and an analysis of event capture after implementation of the alert system. In our hospital system, the alerts maintained enrollment during the transition from a bespoke EHR system to the Epic EHR system (Epic, Verona, WI), suggesting that this approach has the potential to improve the rate of enrollment and diversity of study populations in multiple EHR systems.

MATERIALS AND METHODS

Data collection

Prior to the text processing system’s implementation in October 2016, recruitment for both studies was performed by intradepartmental manual chart review for at least 10 months. Figure 1 illustrates the workflow of the recruitment system. Considering the rarity of each disease, we chose to develop text match alerts rather than use EHR features (eg, Epic’s Best Practice Alerts) to minimize the effort required from providers, thereby maximizing sensitivity, and to allow rare disease specialists to determine applicability of the case presentation. The tool was integrated into the extant Learning Portfolio (LP) system,⁹ which aggregates clinical notes for educational purposes. This system receives notes from the EHR signed by trainees (medical students, residents, fellows), nurse practitioners, and some attending physicians. Text queries were implemented via regular expressions designed to handle abbreviations, acronyms, and common misspellings matched from the entirety of the note. For notes that matched the queries, an email alert was sent to the study team identifying the matching keywords, type of note that generated the alert, date of the note, and a direct link to view the note in LP (after login).

Figure 1.

Alert system workflow. Email accrual is representative of notes created by members in the educational tool Learning Portfolio. A positive text match automated both a REDCap (Research Electronic Data Capture) entry via a REDCap application programming interface call and a secure email to study personnel with a customized view of pertinent information from Learning Portfolio. Via an emailed link, the study researchers, each with their own institutional review board approval, could review the case in Learning Portfolio and/or the electronic health record (EHR). Study investigators could then decide whether or not to approach the patient for enrollment and/or the primary care team with clinical decision support, including suggestion of a formal consult. Active study recruitment roles indicated by square boxes. API: application programming interface.

Following the transition to Epic in November 2017, LP continues to receive trainee and other LP user notes from the Vanderbilt University Medical Center Health Data Repository, which collects these notes in real time from the Epic system. We used our university’s instance of REDCap (Research Electronic Data Capture) to program the email alert system. REDCap is a widely available, secure Web application for building and managing online surveys and databases.^10,11 Email alerts prompted both a database entry via the REDCap application programming interface, in which study coordinators could evaluate the effectiveness of the emails, and an email link to pertinent patient information, in which study investigators, each with their own institutional review board approval, could evaluate appropriateness for study inclusion first in either LP or the EHR.

After system launch, conventional recruitment methods continued alongside recruitment automated by text processing. Study coordinators, one for each study, were consistent before vs after launch of the recruitment tool. No limit was applied to the number of full-time employment hours devoted to recruitment. Pre- and posttransition of EHR systems, the recruitment tool searched notes created by LP users only. Potential delays due to clinical note entry were not accounted for in this report.

Evaluation of the text processing system

Recall for each study was calculated in reference to the manual chart review of ADE records during the 3-year period (January 2016 to December 2018). The major potential source of missed cases was hypothesized to be a result of notes not received by the LP system (eg patients not seen by any providers in the system). To test the significance of differences between enrollment rates by recruitment method (ie, conventional vs text match alert) a Poisson regression was run for the time with alerts on.

RESULTS

Text processing system improved study recall from 43% to 93%

Of 149 patients with either diTdP or SJS/TEN at our tertiary care center between October 2016 and December 2018, 138 were returned by text match (Table 1). Before system launch, 57% of inpatient ADEs were missed by conventional recruiting methods. The precision of the alert system was 33% and 4% for diTdP and SJS/TEN, respectively.

Table 1.

Performance of email alert match by study after system launch

Study	Patients with text match alert	Text match true positives	Text match precision (%)	Total enrolled	Enrolled by text match alert (%)	True positives institution-wide	Recall (%)
diTdP	293	100	32.9	37	68	107	94
SJS/TEN	880	38	4.3	25	56	42	91
Total	1173	138	11.8	62	63	149	93

diTdP: drug-induced torsades de pointes; SJS: Stevens-Johnson syndrome; TEN: toxic epidermal necrolysis.

Enrollment rates were higher with text processing system for diTdP

The text processing system was used to enroll 39 patients over 2 years (Figure 2). With alerts in place, the monthly rate of diTdP enrollment was 4.15 times greater with text match than with conventional recruitment (P = .005). The monthly inpatient enrollment rate for SJS was similar before and after the alerts. Text match contributed to 63% of total enrollment (Figure 3). Months with no inpatient enrollment (n = 2) coincided with months that text match alerts were turned off. Average time to consent inpatient participants was lower after EHR migration (n = 17) than before (n = 22; 46 vs 64 hours).

Figure 2.

Total monthly enrollment by recruitment method. Cumulative email alerts shown for nonunique medical record numbers (n = 1490). Conventional recruitment methods included canvassing of key medical units, engaging departments of subspecialists, and a custom-built electronic dashboard of all electrocardiograms with prolonged corrected QT intervals. No email alerts triggered for 5 months of the study due to a combination of electronic health record (EHR) transition and fewer unenrolled patients with a history of disease.

Figure 3.

Patients with adverse drug events enrolled by email text match. Total email alerts shown for unique medical record numbers (n = 1173). Contextual mentions classified by REDCap users and confirmed by manual chart review. Major exclusion criteria consisted of significant comorbidities (eg, history of bone marrow transplant) and age less than 18 years. Both studies required documentation (eg, rhythm strip) and event onset ≤4 weeks of initiation of drug. Had event indicated patients with a confirmed history of the adverse drug events drug-induced torsades de pointes or Stevens-Johnson syndrome and toxic epidermal necrolysis. Other rash indicates text mentions included as part of a differential diagnosis (n = 171) or negated (n = 253). CLQT: congenital long QT syndrome.

Time between drug event and study consent decreased 35% after system launch

Minimum time between keyword match and patient consent was 31 minutes (Table 2). After the launch of the text processing system, 12 patients were recruited within 24 hours of the ADE. Six patients were enrolled ≤7 days preceding their deaths. Two experimental drugs, previously unknown to cause either ADE, were implicated in events captured by text match. Similar patient demographics were captured in new critical care (eg, neuro intensive care unit) and stepdown units (eg, colorectal surgery) previously unfamiliar with either study. Five blister fluid samples from SJS/TEN patients were collected before the patients left the emergency department. ADE treatment was comparable between the 2 groups.

Table 2.

Inpatient characteristics and study parameters before and after launch of the text processing system

	Before system launch			After system launch
	diTdP (n = 5)	SJS/TEN (n = 9)	Total (n = 14)	diTdP (n = 21)	SJS/TEN (n = 18)	Total (n = 39)
Age, y	54 ± 19	60 ± 25	57	62 ± 17	56 ± 22	59
Female, %	60	89	75	71	72	72
Unique drug classes implicated in ADEa	2	6	8	10	6	16
Hours from event to consent (range)	124 (75-197)	44 (2-86)	84 (2-197)	76 (6-174)	33 (0.5-108)	55 (0.5-174)
Recruited in unit unfamiliar with study	1 (20)	4 (44)	5 (36)	7 (33)	7 (39)	14 (36)
Admitted for diagnosis unrelated to study	2 (40)	1 (11)	3 (21)	13 (41)	1 (6)	14 (36)
Patients consented in event location	4	9	13	12	17	29
Intubated before ADE	0	6	6	5	3	8
QTc, ms	537 ± 47	–	–	579 ± 58	–	–
Required ACLS	2 (40)	–	–	11 (52)	–	–
Required pacemaker	1 (20)	–	–	4 (19)	–	–
Involved BSA, %	–	40 ± 29	–	–	33 ± 30	–
Required skin graft	–	2 (22)	–	–	5 (28)	–

Values are mean ± SD or n (%), unless otherwise indicated.

ACLS: advanced cardiac life support; ADE: adverse drug event; BSA: body surface area; diTdP: drug-induced torsades de pointes; QTc: corrected QT interval; SJS: Stevens-Johnson syndrome; TEN: toxic epidermal necrolysis; ms: milliseconds.

^aRxNav Anatomical Therapeutic Chemical Class 1-4 from the National Library of Medicine.

DISCUSSION

To meet the enrollment objectives of 2 rare disease studies, we created an event alert system that improved recall for 2 drug-induced ADEs from 43% to 93% over the prior manual approach, identifying 138 true cases. In addition, higher rates of participant accrual by text match alerts were observed for diTdP. This is likely due to the fact that those relatively fewer departments caring for patients with SJS/TEN (ie, the burn unit) became aware of the study with the text match alerts in place and later informed the study investigator of potential cases directly.

Prior studies have implemented clinical trial alert systems to identify patients with type 2 diabetes, acute coronary syndrome, or sepsis.^12–14 These studies report 51%-82% sensitivity limited to a single department or hospital setting (either inpatient or outpatient). We desired a system with high recall, rather than high precision, to maximize potential for recruitment of patients with severe disease. In 1 case, blister fluid was retrieved in an intensive care unit just 3 hours before a patient’s death.

An unanticipated benefit of this approach was the potential to improve recruitment experience. In informal interviews conducted after the completion of the study, study coordinators reported fewer hours of recruitment time per patient. Before system launch, study coordinators reported maximum travel time of 6 hours round trip to meet 4 patients with known histories of disease on home visits and another at a neighboring hospital. After, same-day enrollment was achieved for 6 diTdP patients. A patient with SJS/TEN was enrolled 31 minutes after the history and physical was signed by a resident while the patient was still in the emergency department, a location that study coordinators were unable to monitor before the alerts were in place.

Previously, project recruiters noted fluctuation in enrollment with staff turnover (eg, residents and fellows), but with the system in place, recruitment was less dependent on staff education and reminder efforts. In one instance, a study investigator received an alert on a clinical note that he himself had written but had failed to remember to approach the patient for the study (the patient was then successfully enrolled after the alert reminder).

The alerts maintained enrollment during the transition of EHR softwares, from a bespoke EHR system to Epic. During this time, there was an outage of emails for 5 months due to changes in note formatting and needed system upgrades for EHR transition. However, we found that transfer of notes from Epic to LP was still essentially real time once the system upgrades were in place. In fact, recruiters were able to approach inpatients for consent faster after the transition. We attribute this finding to both study users’ familiarity with the alert system (eg, less time spent on review of the patient in the EHR) as well as greater familiarity with the rare disease studies over time, prompting departments with potential study participants to consult study investigators directly. While text queries were embedded within our LP system, a text processing approach is feasible in all EHR systems with access to a clinical note data repository.

As previously observed in other systems,^15,16 the benefits observed in diagnosis and capture are limited by the availability of recruiters to consent a patient. In our system, time to recruitment was further limited by time to note entry, including note signature (if applicable, cosignature was not required to trigger an alert). It is possible that our tertiary care setting observed higher prevalence of these rare ADEs and therefore enrollment rates. It is also possible that limiting note capture to LP users confined the scope of text match return.

The current approach was simple and did not include negation or removal of family history. The REDCap survey, completed at the time of recruitment, revealed areas that could be targeted to reduce false positives. Additionally, false positives were reviewed retrospectively after the study’s completion. For example, 37% of the diTdP alerts were for psychiatric patients starting haloperidol (“monitor QT to prevent torsade”). Most others were excluded by a few minutes of EHR or LP review of the linked note. Regular expression matches returned a transcutaneous electrical nerve stimulation (“TENS”) unit and patients with Sjogren’s syndrome (“SjS”). Study coordinators were sent alerts triggered by notes referencing “Dr Stephen Johnson” 63 times (the majority of which were eliminated on study staff review of the email alert itself, which identified the note author).

Future efforts include refinement of the contextualized (negation, temporality, and experiencer) mentions to improve precision and surveying user experience in a variety of settings. We evaluated the tool’s ability to improve clinical trial recruitment but envision the system in place for acute intervention trials, quality improvement projects, or studies of more common acute events such as myocardial infarction or ruptured appendicitis. Studies looking to implement text matching or natural language processing systems with high precision must weigh the risk of increasing rates of false negatives, especially for rare or life-threatening conditions. While we compared text processing with traditional recruitment methods, other studies should consider comparing text-based alerting methods with integrated EHR approaches, such as Epic’s Best Practice Alerts or other clinical decision support tools. Real-time monitoring of EHR notes may accelerate research for certain types of conditions less amenable to conventional recruitment paradigms. We believe that further studies will benefit from our experiences.

CONCLUSION

This report summarizes the findings of an email alert system for 2 studies for which estimated incidence in the United States is limited to case reports or small observational series. We developed a text processing alert tool that quickly connects providers with relevant patients that we used for enrollment into clinical studies. Efforts to optimize EHR-assisted event identification tools might make possible large-scale studies for which time-based variable collection was previously impossible.

FUNDING

This work was supported by P50GM115305 from National Institutes of Health/National Institute of General Medical Sciences (to DR and JCD).

AUTHOR CONTRIBUTIONS

SD, JW, PS, JB, EP, DR, and JCD contributed to the research project conception, organization, and execution. TS and KW contributed to the research project conception and organization. LAT, GD and JCD contributed to review and critique of the statistical analysis. SD, GD, JCS, DG, EP, DR, and JCD contributed to review and critique of the manuscript. SD contributed to design, execution, and review an critique of the statistical analysis as well as writing of the first draft and review and critique of the manuscript.

CONFLICT OF INTEREST STATEMENT

Since the study’s completion, JCD has assumed the role of Chief Executive Officer All of Us Research Program at the National Institutes of Health. In addition to this role, he remains an Adjunct Professor in the Department of Biomedical Informatics at Vanderbilt University Medical Center.