Evaluation of an Online Platform for Cancer Patient Self-reporting of Chemotherapy Toxicities

Abstract

The current mechanism for monitoring toxicity symptoms in cancer trials depends on a complex paper-based process. Electronic collection of patient-reported outcomes (PROs) may be more efficient and accurate. An online PRO platform was created including a simple data entry interface, real-time report generation, and an alert system to e-mail clinicians when patients self-report serious toxicities. Feasibility assessment involving 180 chemotherapy patients demonstrated high levels of use at up to 40 follow-up clinic visits per patient over 16 months (85% of patients at any given visit), with high levels of patient and clinician acceptance and satisfaction (>95%). Alerts were used as the basis for delayed chemotherapy treatments, dose modifications, and scheduling changes. These results demonstrate that online patient-reporting is a feasible strategy for chemotherapy toxicity symptom monitoring, and may improve safety and satisfaction with care. Ongoing multi-center research will evaluate the impact of this approach on clinical and administrative outcomes.

Introduction

Monitoring of patient symptoms during chemotherapy is a cornerstone of oncology practice and is mandated in cancer clinical trials. The standard method for collecting this information includes paper forms filled out by staff members, followed by transcription into local databases. This approach has been criticized as inefficient, error-prone, and not representative of the true patient experience. ¹ An alternative method, direct patient reporting of symptoms as electronic “patient-reported outcomes” (PROs), has been suggested to improve the comprehensiveness, quality, and timeliness of symptom data collection. ^2–8 PRO collection has been advocated by the Food & Drug Administration, ^9–13 and is already standard in other research contexts such as quality of life assessment and non-oncology trials. ^14–16 However, there has been scant evaluation of the feasibility or potential benefits of using electronic PROs for toxicity monitoring in cancer patients. ^17–19

To address these issues, a multidisciplinary research program was developed at Memorial Sloan-Kettering Cancer Center with the following goals: 1) to create and assess a technology platform for electronic patient self-reporting; 2) to asses the feasibility of using this data gathering method; and 3) to evaluate the impact of this approach on clinical outcomes and administrative efficiency. In the following report we describe the development of this patient self-reporting portal, and the results of sequential studies evaluating the feasibility and clinical impact of this system.

Methods

A working group was assembled in 2003 to select a technology platform and content for a patient symptom-reporting portal; to facilitate design and implementation of the system; and to oversee a research program for evaluating the potential clinical and administrative benefits of the system. This group included physicians, nurses, behavioral scientists, and representatives from hospital administration and information services. The group outlined a work flow and timeline based on a set of identified considerations for the creation and scientific evaluation of an electronic PRO platform (▶). The underlying hypothesis was that patient self-reporting may improve the quality and efficiency of symptom toxicity data collection, and may ultimately benefit the well-being and satisfaction of patients.

Table 1.

Table 1 Questions and Considerations When Developing an Electronic PRO Questionnaire Platform

Catagory	Questions	Considerations
Questionnaire platform	Paper vs. electronic?	Cost, development time, data volume
	If electronic, what hardware type (Web, PDA, IVR)?	Cost, hardware availability, training, maintenance
	Internal software development vs. third-party product?	Cost, time, technical support, updates
	Data storage in standard/exportable format?	Yes, because cannot anticipate future needs
Questionnaire content	Use existing instrument vs. original development?	Objectives of research/data use
	Verify adequate validation of instrument?	Re-validation for major modifications
	Appropriate to target population?	Prior use in similar patients or disease
PHI privacy	Institutional privacy review?	Privacy board or IRB, HIPAA officer
	Security (polarized) screens on monitors?	Computer locations
	Password protection?	Encryption, user modification, secure recovery
	Remote access to system vs. limited local access?	Objectives of research
	Disclaimer at login?	Legal/liability issues
Data security	Institutional security review?	IRB, IS department
	Secure system configuration and database?	System validation, automated audit trail
	Login timeout?	Functionality, institutional requirements
Patient safety	Automated warnings for concerning responses?	Content of questionnaire items, liability, safety
	Remote access to previously entered data?	Objectives of system, privacy, security

PRO = patient reported outcomes; PDA = personal digital assistant; IVR = integrated voice response; PHI = protected health information; IRB = institutional review board; HIPAA = Health Insurance Portability and Accountability Act; CAT = computerized adaptive testing; IRT = item response theory; IS = Information Services; UAT = user acceptance testing.

Toxicity-related symptom burden was identified as the most pertinent information that can be collected from chemotherapy patients via a PRO approach. Such information is frequently the basis for dose modification or supportive medications, or in the case of clinical trials for regulatory reporting. The mandated instrument for this purpose in oncology trials is the National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE). ^20,21 The CTCAE is comprised of over 1000 individual items, each graded on a 5-point ordinal scale, including toxicity-related phenomena such as laboratory values (e.g., anemia) as well as symptoms (e.g., nausea). Each numeric grade is associated with a descriptor, such that grade 1 = mild; grade 2 = moderate; grade 3 = severe; grade 4 = disabling; and grade 5 = death. Grades of 3 and greater are reportable during trials, and may be the grounds for management changes. Although the CTCAE was designed for clinician reporting, the rating scale was readily adaptable for patient self-reporting, ²² and a paper correlation study was conducted to assess the relationship between patient and clinician responses to similar items. ²³ The CTCAE items developed to test PRO were added to a master item bank from which electronic questionnaires could be constructed based on the characteristics of a patient population of interest.

For the design of the online patient portal, flexibility was desired to integrate advanced questionnaire techniques, including item response theory and computerized adaptive testing. ^24–27 An administrator interface was necessary to control patient and clinician users, as well as a patient interface to securely enter symptom data, view past entered data, and dynamically generate longitudinal symptom reports.

Ease of use for those without prior computer experience was considered essential. Based on user acceptance testing involving 20 patients, a touchscreen design with one item per page was adopted (▶). The longitudinal report was designed to display individual symptom grades for any given day of data entry, as well as temporal relationships to chemotherapy administration. An automated rapid reporting function was included for potentially serious toxicities whereby a designated clinician would be informed electronically in real-time by e-mail whenever a concerning toxicity was self-reported by a patient.

Figure 1.

Example Data Entry Screen.

A custom form building application was developed to generate XML-encoded questionnaires using symptoms derived from the item bank, dynamically generate Oracle tables for each questionnaire, and to accommodate integration of item response theory and computerized adaptive testing techniques. To assure security, the data entry interface can be accessed via the Internet outside of the institution’s external firewall, the Web server resides inside an external firewall, the application server resides behind an internal firewall, and all protected health information is stored in a secure database server behind a second internal firewall. Medical record numbers are the only personally identifying information associated with reported symptoms. An automated audit trail records user access to patients’ reported data. We were required by institutional policy to institute a 6-minute time-out for lack of mouse movement and 20-minute absolute system time-out for all users. Touchscreen computers were installed in clinic waiting areas for patients to enter data at appointments.

User acceptance testing with 30 patients found that all participants were able to login, complete, and submit questionnaire forms successfully. Mean duration to complete a questionnaire including 19 items and an open-ended data entry field was 4.7 minutes (median 4.2 minutes). CTCAE items often include lengthy text and take longer per item to read than simple multiple choice items.

Results

To determine the level of access to computers and the Internet in our patient population, an initial baseline paper survey was administered to 443 cancer outpatients and their companions in 2000; 64% of patients and 76% of companions owned computers, and Internet access was available at home to 58% and 68%, respectively. ²⁸ A follow-up survey of 90 outpatients in 2005 found 80% of patients to have regular Web access, 73% with prior Web use, and 65% with e-mail experience. All patients without regular access were older than 75, whereas all patients younger than 75 had regular access. All of those with home access expressed interest in electronic symptom self-tracking. When shown a demonstration, all with Internet experience expressed interest in using the portal.

To assess the feasibility of the portal for relatively ill cancer patients receiving chemotherapy, a single-center study was conducted with 180 cancer patients receiving chemotherapy. ^22,29–31 More than 85% of participants logged in at each sequential clinic visit during an eight-week observation period. Two-thirds voluntarily logged in from home computers without prompting. In a subset of 100 patients who were followed over 16 months, similar levels of in-clinic login compliance was seen after up to 40 visits, with no significant attrition over time. Rates of voluntary home logins were high in younger female patients with gynecologic malignancies (66%), but were less impressive in older men with lung cancer (15%). Patient characteristics significantly associated with greater compliance included prior computer experience but not age, sex, education, baseline performance status, or cancer stage.

The impact of electronic transmission of PRO symptom information on clinician behavior was evaluated in 60 cancer patients receiving chemotherapy. Consecutive outpatients being seen in a cancer clinic were approached and invited to participate until 60 with Internet access (at home or office) were enrolled. The demographics of this population were similar to Internet-avid patients at this institution overall in terms of age, education, income, diagnosis, and performance status. ²⁸ All participants were given access to the online self-reporting portal, but no reminders were given to login between clinic visits. During the study period, all patients logged in at least once, with a mean number of logins of 9 (median 8; range 1–30). The mean number of clinic visits during this time period was 3 (range 1–6). Longitudinal reports of prior symptom entries were self-viewed by 55 (92%) participants.

Any time a patient reported a grade 3 (“severe”) or grade 4 (“disabling”) toxicity symptom, an e-mail was sent to the responsible clinical team (nurse and/or physician) with this information. No specific instructions were given to clinicians about how to respond to these alerts. During an eight-week period, 57 grade 3 or 4 toxicities were reported by 25 different patients, of which 42 originated from patients at home (▶). In response to these alerts, 17 clinical actions were taken, including delaying chemotherapy, calling patients, setting up new appointments, and medication changes.

Figure 2.

Automated e-mail alerts triggered by patient-reported grade 3/4 (severe/disabling) symptoms, and clinician responses to alerts.

Across studies, patient satisfaction and clinician acceptance of the portal was high overall. Most patients found the portal “easy to use” (96%), “useful” (92%), “improved communication” (85%), and wished to continue using the portal (96%). Among participating physicians, 89% discussed reports with their patients, 78% felt reports “accurately reflect true patient clinical status,” and 89% felt the system was “useful for toxicity monitoring.”

Discussion

Development of electronic patient interfaces requires consideration of administrative and regulatory requirements including data security and privacy. Administrators may be reluctant to support such systems due to expenses associated with installation and maintenance without tangible returns on investment and concern that clinicians and patients may resist adoption of any new system. It is therefore crucial to demonstrate the feasibility of implementing such systems, as well as the value of portals in terms of clinical and administrative endpoints and satisfaction with care.

Our research program was conceived to identify the optimal approach for building a flexible patient portal, to assess its feasibility, and to measure the potential value of such systems. Completed studies suggest feasibility, high levels of patient satisfaction, clinician acceptance, and willingness of staff to base management decisions on patient-reported information.

Ongoing work is measuring the impact of this approach on hard clinical outcomes. A randomized controlled trial is in progress in which patients are assigned to portal access vs. routine care (no portal). We are measuring differences between groups in the number of canceled or delayed chemotherapy treatments, dose modifications, emergency room visits, unscheduled clinic visits, telephone contacts with staff, quality of life, and satisfaction with care. In addition, 14 cancer centers have agreed to participate in a multi-institutional feasibility assessment through the National Cancer Institute cooperative group mechanism. This study is designed to evaluate barriers to implementing and maintaining the system, training of personnel, patient satisfaction, and clinician acceptance. Results of these studies are anticipated in 2009.

There has been recent attention at the National Cancer Institute to the development of systems for detection and communication of serious adverse events during clinical trials. ^32,33 Our results suggest that electronic patient reporting platforms can allow for rapid detection of potentially serious symptoms in real-time, and can elicit clinician responses. In the setting of chemotherapy and cancer clinical trials, during which patients receive toxic therapies, the ability to improve outpatient monitoring may significantly improve clinical outcomes, research data quality, and the patient experience.

Nonetheless, it remains unclear if severely ill or incapacitated patients are capable of consistently self-reporting via this means. Patients in our studies had advanced cancer, but most were sufficiently functional to receive chemotherapy in an ambulatory clinic setting. Therefore other strategies to collect real-time data from homebound patients may be necessary, for example via a multi-tier approach in which those who fail to self-report online receive a call via an interactive voice response system, with a human backup call for those who do not comply.

Beyond issues of feasibility or quality improvement, engaging patients as active participants in their own care conveys the message that symptom control and treatment tolerance are priorities to clinicians and researchers. Harnessing novel technologies to enhance communication and information gathering is therefore a key element in the growth of patient-centered models of care.