An Automated System for Detecting Nonadherence in Laboratory Testing and Monitoring for Myelosuppression in Patients Receiving Self-Administered Oral Chemotherapy

The authors sought to assess monitoring lapses and incidents of myelosuppression in patients undergoing self-administered chemotherapy for glioblastoma, as well as test software designed to detect and alert clinicians to lapses in monitoring.

Abstract

Introduction:

Patient compliance with routine monitoring for self-administered chemotherapy is problematic. We sought to assess monitoring lapses and incidents of myelosuppression in patients undergoing self-administered chemotherapy for glioblastoma, as well as test software designed to detect and alert clinicians to lapses in monitoring.

Patients and Methods:

A retrospective analysis was conducted to identify patients (N = 117) who received standard oral temozolomide for glioblastoma at our institution from 2003 to 2010. Gaps in monitoring were classified as minor (10 to 12 days) or major (13 to 28 days), and adverse events were graded using standard criteria. During the prospective portion of the study, we tested a software-based system that alerted clinicians of monitoring lapses and adverse events among patients receiving self-administered temozolomide for glioblastoma (n = 37).

Results:

Our retrospective review found that 34 of 117 patients experienced monitoring gaps during treatment. No association between gaps and risk of myelosuppression were found. Patients with gaps were more likely to be male (P = .04). Patients monitored prospectively with the software experienced no major gaps in monitoring (P = .007 compared with retrospective patients).

Conclusion:

Our retrospective review demonstrated that monitoring nonadherence was occurring at a substantial rate. Our computerized system eliminated major gaps in monitoring in the prospective portion of our study. Although there is no association between monitoring gaps and the occurrence of adverse events, when they do coincide, continuing oral chemotherapy during an unrecognized adverse event may worsen the patient's condition. Automated systems are justified and serve a function not currently being addressed.

Introduction

The problems of compliance in the administration of oral chemotherapy have long been recognized.^1,2 Initial discussions centered on patient self-dosing compliance.³ For those chemotherapies with potential adverse effects, the problems of adherence to monitoring were also discussed, with researchers mainly focusing on patient and nursing education strategies.^4–6 However, little attention has been paid to the detection of monitoring lapses in day-to-day oncology practice.

The traditional administration of intravenous chemotherapy in infusion centers guaranteed that clinical staff had access to patients in order to assess therapy-related toxicities and laboratory testing compliance. Over time, information systems and monitoring practices were thus built primarily to detect abnormal values but were not structured to readily note missing test results. The introduction of outpatient oral chemotherapy regimens has challenged this blind spot. Patients who receive extended courses of oral agents can continue self-medication despite missed follow-up visits or laboratory draws, which can potentially prolong or exacerbate adverse events (AEs) that would have been detected and addressed had patients complied with monitoring. Although systems exist in most research centers to detect nonadherence to monitoring of patients on clinical trials, similar systems for patients not on trials do not exist.

Temozolomide is an oral alkylating agent given concurrently with external-beam radiation as standard therapy for patients with newly diagnosed glioblastoma.^7,8 These patients are typically seen daily by radiation technologists, weekly by radiation oncology staff, and at the midpoint and end of concurrent therapy by their medical neuro-oncologist. Because of the risk of myelosuppression associated with this therapy, standard practice is to obtain a weekly complete blood count (CBC) with differential and platelets during the 6 weeks of concurrent chemoradiotherapy to monitor for myelosupression.⁹

After we observed severe myelosuppression in some patients who had unrecognized lapses in monitoring, we sought to retrospectively measure this risk and then use our findings to design and prospectively test a software-based monitoring system designed to flag and notify team members of nonadherence to standard monitoring practices.

Patients and Methods

Retrospective Review

A retrospective analysis was performed on 117 consecutive patients who had started a standard 42-day continuous course of oral temozolomide as therapy for a newly diagnosed primary malignant brain tumor at the Comprehensive Cancer Center of Wake Forest University between 2003 and 2010. Patients were identified by a search of a clinical data warehouse. After patients were identified, their therapy start and stop dates were recorded and all laboratory tests during the course of their therapy performed at the center were downloaded from the same data warehouse. Using those results, we assessed monitoring compliance and AEs. Gaps in monitoring were defined as no testing for 10 days or more and were classified as minor (10 to 12 days) or major (13 to 28 days). Serious AEs involving reduced neutrophils, hemoglobin, platelets, or leukocytes were graded according to Common Terminology Criteria for Adverse Events (CTCAE; version 3).¹⁰

Prospective Software Monitoring System

The Comprehensive Cancer Center previously had developed a laboratory download and auto-grading system to assist in gathering laboratory AE data for clinical trials. Staff also had previous experience building data-driven systems to detect errors in medical care by means of such downloads,¹¹ so the development of a prototype system to monitor patients for missing laboratory results was not particularly difficult. The Wake Forest University Health Sciences Institutional Review Board approved the software for use on all patients with newly diagnosed gliomablastoma who were scheduled to begin concurrent radiation and temozolomide. Per the board, no informed consent document was required.

Patients were registered for monitoring by entering their medical record number and the start date of therapy; patients were taken off monitoring by entering their therapy stop date. The software was built within an existing database software system (SQL Server 2008) and used data streams already systematized for use by the Clinical Trials Research Management System at our institution. These data streams included all laboratory results, scheduled visits for outpatients, and the inpatient census (Figure 1). Using these data streams, the program identified the specific patients being monitored and performed a series of simple date-based checks.

Figure 1.

Monitoring system overview.

First, the program used the last date of collection of the key laboratory tests (absolute neutrophil count [ANC], WBCs, platelets, and hemoglobin) to determine if more than 7 days had elapsed since the previous test. Second, the program determined if the next scheduled visit for each patient was more than 7 days in the future. If either of these conditions was met, an automated e-mail was sent to the clinical team, including the nurse and physician. Third, the program used a pre-existing CTCAE auto-grading program to notify the clinical team about any abnormal laboratory results that indicated myelosuppression-related events. The program initially used grade 3 as the threshold, but it was later changed to notify of any grade 2 or higher hematologic events. Finally, the inpatient census was scanned to identify monitored patients that had been admitted within the previous 24 hours; this also generated an e-mail alert to the clinical team. The initial version of the software monitoring system was implemented in October 2010; as of September 2012, 37 evaluable patients had been monitored using this system.

Statistical Analysis

In the analysis of the retrospective data, we calculated the percentage of patients (and associated 95% CIs) who experienced gaps or had grade 2 or higher toxicities. We also calculated the median and range of the length of the observed gaps. Sex and toxicity rates were compared between patients who did and did not experience gaps by using χ² or Fisher's exact tests; age was compared by using a t test. We also used a Fisher's exact test to compare severe (grade 3 or 4) events in patients who had previously had a grade 2 event versus those who had not. In the analyses of the prospective data, we calculated the percentage of patients (and associated 95% CIs) who experienced gaps or had grade 2 or higher toxicities. We used Fisher's exact text to compare the rates of major gaps in the retrospective period to the prospective period. All analyses were done in SAS (version 9.2, Cary, NC), and a two-sided .05 alpha level was used to indicate statistical significance.

Results

Retrospective Review

In the retrospective review, 34 (29.1%; 95% CI, 20.8 to 37.3) patients had one or more gaps during treatment; 19 (16.2%; 95% CI, 9.6 to 22.9) had major gaps, and 18 (15.4%; 95% CI, 8.9 to 21.9) had minor gaps. The median gap time was 10 days for minor gaps (range, 10 to 12 days), and 16 days for major gaps (range, 13 to 28 days).

Grade 3 or higher hematologic events were observed in nine (7.7%; 95% C, 2.9 to 12.5) patients, and 12 (10.3%; 95% CI, 4.8 to 15.8) patients had grade 2 alone or grade 2 events that preceded higher graded events. Overall, 16 (13.8%; 95% CI, 7.5 to 19.9) patients experienced a grade 2 or greater event. There was no association between gaps in testing and occurrence of myelosupression (Table 1); however, two patients with gaps in coverage exhibited grade 3 or 4 events immediately on their return to monitoring. Patients who experienced gaps were more likely to be male (76.5% v 56.6%, P = .04). Patients who had grade 2 events were 11 times more likely to later have a grade 3 or 4 event (41.7%; n = 5 of 12), compared with those who did not have grade 2 events (3.8%; n = 4 of 105; P < .001).

Table 1.

Age, Sex, and Toxicities by Gaps in Monitoring

Characteristic	Patients With Any Gap (n = 34)		Patients With No Gaps (n = 83)		P*
	No.	%	No.	%
Age, years					.51
Mean	52.5		54.3
SD	14.7		13.6
Male sex	26	76.5	47	56.6	.04
Any toxicity	5	14.7	11	13.3	.99
Grade 2 event	3	8.8	9	10.8	.99
Grade 3 event	3	8.8	6	7.2	.72

Abbreviation: SD, standard deviation.

^*P value from t test for age; from χ² or Fisher's exact test for other variables.

Prospective Monitoring

We used the prototype software to prospectively monitor 37 evaluable patients throughout the course of their therapy. The software sorted through 14,836 laboratory results (917 of which were pertinent), and 311 scheduled visits. Twenty-nine patients had at least one flag for possible nonadherence, and 26 patients had at least one flag for missing laboratory results. Eight patients had at least one flag for appointments too far in the future, nine (24.3%) had at least one AE flag, and five (14%) were flagged for inpatient admissions. In total, 159 automatic e-mails were sent to the clinical team.

Although the system produced adherence flags in 29 (78.4%) of the patients, 14 were false flags, and eight were due to holidays. Among the remaining seven patients, eight nonadherence events were detected in progress, with five detected before nurses or physicians recognized the problem. This was corrected by prompting nursing staff to call the patient or their family to reschedule testing. Three patients (8.1%, 95% CI, 1.7 to 21)still experienced minor monitoring gaps.

Compared with the retrospective analysis, no patients monitored by the prospective system had any major gaps in monitoring (P = .007). Five patients (13.5%; 95% CI, 2.5 to 24.5) had grade 3 or higher AEs and an additional 10 (27.0%) had grade 2 AEs before the end of monitoring. It is worth noting that all of the AE flags and admission flags detected by the system had already been noted by the clinical staff (as would be expected, given the established systems for detecting such events).

Discussion

Changes in treatment modalities can create new challenges to monitoring patients, and our initial findings suggest that oral chemotherapy poses just such a challenge. Current medical informatics systems are not designed to allow clinicians to detect missed visits and laboratory findings that might indicate emerging AEs. Although clinicians in other domains have explored automated systems for monitoring adherence,¹² we sought to explore their utility in the clinical oncology setting.

The retrospective review found that close to one-third of the patients had a gap in monitoring. Despite the fact that these gaps were not associated with the occurrence of an AE, patients may still be at increased risk of complications when an AE occurs during such a gap and the patient does not receive appropriate clinical care. In addition, continuing to take chemotherapy during a monitoring gap may exacerbate the AE, with the likely risk to the patient increasing the earlier in the gap window that the AE occurs. Although larger studies might be needed to determine whether AEs are more severe in this scenario, we feel confident that the issue is serious enough to justify basic systems such as the one we have designed.

Several problems were encountered during the development of our monitoring system. A majority of these problems were false-positive e-mail alerts that were eventually determined to be unrelated to an unplanned gap in monitoring and instead were associated with the following issues: (1) system delays in downloading the laboratory values, (2) failure to denote chemotherapy stop dates, (3) gaps resulting from holiday clinic schedules that fell on days of scheduled visits, or (4) occurrence of the last pretreatment laboratory testing several days before the start date of therapy. Software modifications were made in response to false positives. Alerts during the last 4 days of the patient's therapy were disabled because the patient would be stopping therapy regardless of his or her laboratory values. In addition, the length of time from the beginning of therapy to the latest testing was use during the first week of therapy rather than from the last pretreatment testing. Solutions to the problem of holiday schedules are currently being considered. In fall of 2012, our institution started using a new electronic medical record system (Epic, Verona, WI), which temporarily suspended the data streams used by this system. We hope to resume use and expand the program's testing soon.

Our intention was to develop a prototype system to demonstrate the feasibility and utility of monitoring nonadherence to laboratory testing and monitoring for myelosuppression in patients receiving self-administered oral chemotherapy. Although we have not implemented this system for a time period sufficient to thoroughly evaluate its effectiveness, the absence of major gaps in adherence among patients monitored by the system validates its potential utility. As of this writing, no such functionality exists in routine electronic medical record systems; therefore, physicians and nurses must search through large amounts of data in order to detect abnormal values and in doing so are unlikely to notice monitoring gaps. Systems such as the one we developed are simple to build and have the potential for inclusion in the electronic medical record, which could reduce the frequency of errors in medical care.¹³ Indeed, the ability to add automated detection of missing monitoring data in a broad variety of preplanned and standardized regimens should be a configurable function in hospital systems.