ELECTRONIC PATIENT-REPORTED OUTCOMES FROM HOME IN PATIENTS RECOVERING FROM MAJOR GYNECOLOGIC CANCER SURGERY: A PROSPECTIVE STUDY MEASURING SYMPTOMS AND HEALTH-RELATED QUALITY OF LIFE

Abstract

Purpose

We previously reported on the feasibility of a Web-based system to capture patient-reported outcomes (PROs) in the immediate postoperative period. The purpose of this study was to update the experience of these patients and assess patient and provider satisfaction and feedback regarding the system.

Methods

This is a prospective cohort study of patients scheduled to undergo laparotomy for presumed gynecologic malignancy. Patients completed a Web-based Symptom Tracking and Reporting (STAR) questionnaire preoperatively and weekly during a 6-week postoperative period. Email alerts were sent to study nurses when concerning patient responses were entered. The patient and the nurse assessments of STAR’s usefulness were measured via an exit survey.

Results

The study enrolled 96 eligible patients. Of these, 71 patients (74%) completed at least four of seven total sessions. Of the patients who completed the exit satisfaction survey, 98% found STAR easy to use; 84% found it useful; and 82% would recommend it to other patients. Despite positive feedback from patients, clinical personnel found that the STAR system increased their current workload without enhancing patient care.

Conclusions

Application of an electronic program for PROs in those recovering from major gynecologic cancer surgery is feasible, and acceptable to most patients. While most clinicians did not find STAR clinically helpful, the majority of patients reported a positive experience with the system and would recommend its use. The program helped many patients feel more empowered in their postoperative recovery.

INTRODUCTION

The NIH, NCI, FDA and numerous other stakeholders have asserted that the impact of medical interventions and surgery are best evaluated by patients directly, without filtering by clinicians, in the form of patient-reported outcome (PRO) measures [1]. As a result, there has been increasing emphasis on the incorporation of PROs into clinical trials and routine clinical practice [2, 3]. This is also relevant because the Affordable Care Act (ACA) allows for a financial reward, in the form of small bonuses, for providers who provide quality care. The ACA also allows for financial penalties for providers who fail to provide quality care. Assessment of reward or penalty is based on outcome or performance as measured by a quality indicator [4]. Because stakeholders in cancer care agree that the current quality metrics are insufficient, some have proposed new models. Many of these, such as the National Quality Forum (NQF) model, include PRO measures [5].

Most PRO surveys in cancer patients have been administered at baseline and 3 months post-treatment [2, 6]. There is limited data regarding PROs in gynecologic cancer patients in the immediate 6 weeks following surgery. Collecting PROs during this time period can enrich preoperative teaching, help identify complications earlier, and improve symptom control [2]. Currently, however, there is not enough data available to determine if patients are able or willing to self-report symptoms during this critical period, or if providers find this information constructive.

Our previously published pilot study suggested that the use of a Web-based system to capture PROs is feasible and highly acceptable by patients in the acute postoperative period after major gynecologic surgery[7]. The objectives of this study were to update the experience of these patients, and to assess patient and provider satisfaction and feedback regarding the system.

METHODS

This study was approved by the Institutional Review Board (IRB) at Memorial Sloan Kettering Cancer Center. The patients, study design, and online platform were previously described in a pilot report on the feasibility and acceptability of this Web-based system [7]. English-speaking patients 18 years of age and older, who were scheduled to undergo laparotomy for presumed or known gynecologic malignancy, were recruited to participate. All patients were required to have access to a home computer and a personal email account.

At the time of enrollment, patients were trained in the use of the Symptom Tracking and Reporting (STAR) system. They were asked to complete a baseline information questionnaire and seven STAR surveys. This paper questionnaire was administered by the consenting professional immediately after consent was obtained. It measures variables that we expected to be predictors of STAR utilization, including age, education level, employment status, and prior internet experience. Demographic data was gathered from the electronic medical records. The surveys consisted of the patient adaptation of the NCI Common Terminology Criteria for Adverse Events (CTCAE) version 3.0 and the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 version 3.0 [8, 9]. Both of these are validated instruments that have undergone extensive psychometric testing, and meet established standards for validity and reliability as detailed in the FDA Draft guidance for PROs [1, 10]. Patients completed the surveys preoperatively, and weekly during a 6-week postoperative period. Reminders to complete the reports were sent to participants via email.

Email alerts were also sent to the study nurses when concerning patient responses were entered. Alerts were considered concerning according to pre-specified limits set by the Gynecologic Oncology Service. This is the same system presently used to triage patient phone calls. Any actions taken by the nurses in response to these alerts were recorded. However, specific responses were not required. Patients were encouraged to call their physician’s office if medical attention was needed, as there was no regularly scheduled monitoring of information entered into the STAR system.

Patient and nurse assessments of STAR’s usefulness were measured via an exit survey. A “responder” was defined as a patient who logged in and completed at least half of the questionnaire, and participated in at least four of the seven potential login times.

RESULTS

Demographics

The study accrued 120 consecutive patients between July 2009 and January 2015. All participants were scheduled to undergo laparotomy for suspected or confirmed gynecologic malignancy. Twenty-four patients were eventually removed from the study, leaving 96 eligible, evaluable patients (Figure 1). The median age was 55.5 (range 18–74). Table 1 reports the demographic and clinical characteristics of the included patients.

Figure 1. Consort Diagram.

Diagram demonstrates study accrual between July 2009 and January 2015, exclusion, and participation. All participants were scheduled to undergo laparotomy for suspected or confirmed gynecologic malignancy.

Table 1.

Patient Characteristics

Characteristics	Number of patients (%) N = 110
ASA Class
II	51 (46)
III	59 (54)
Surgical Procedures Performed
Hysterectomy ± staging	76 (69)
Resection of tumor	28 (25)
Salpingo-oophorectomy	5 (5)
Other	1 (1)
Disease Origin
Ovary/Fallopian tubes	79 (72)
Uterus	24 (22)
Other	7 (6)
Final pathologic evaluation
Malignancy	86 (78)
Benign disease	19 (17)
Borderline tumors	5 (5)
Internet use frequency
More than once a week	85 (77)
At least once a week	7 (6)
Once a week	6 (5)
Less than once a week	2 (2)
Unknown	10 (9)
Email use frequency
More than once a week	87 (79)
At least once a week	7 (6)
Once a week	5 (5)
Less than once a week	1 (1)
Unknown	10 (9)
Highest educational level
Professional/graduate degree	29 (26)
College degree	32 (29)
Some college	18 (16)
High school or less	8 (7)
Unknown	23 (21)
Job status
Employed	58 (53)
Homemaker	11 (10)
Retired	11 (10)
Unemployed	5 (5)
Other	3 (3)
Unknown	22 (20)

Intervention

Seventy-one patients (74%) completed at least four of seven surveys, and were therefore considered responders. Sixty-nine (63%) patients completed the preoperative session in STAR. The remaining patients did not complete the preoperative session, but did complete subsequent surveys. Nine (9%) patients completed only one survey. Similar to the pilot study, patient compliance gradually decreased as the postoperative period elapsed (Figure 2). There was no statistically significant difference in the demographic or clinical characteristics of responders versus non-responders.

Figure 2.

Percentage of completed PRO surveys at each point, starting preoperatively to week 6

Alerts

One hundred and twelve patient-reported symptoms generated an alert, resulting in 28 contacts and two Emergency Department referrals. Overall, the CTC generated 81 individual episode alerts and the EORTC generated 31 episode alerts of 84 different symptoms. The most common CTC symptoms were poor performance status, nausea, and fatigue. The most common EORTC symptoms were difficulty with strenuous activity, constipation, and pain. Tables 2 and 3 show the distribution of all symptoms. Most alerts were read by a nurse within one day (mean 1, median 0 days). Ten alerts (12%) had already been addressed by a recent patient phone call or clinic visit. Three (4%) patients had already been scheduled for a clinic visit, during which the issue could be addressed. One (1%) patient was admitted while completing her survey, and her symptoms were addressed by the inpatient care team.

Table 2.

CTC Symptoms

CTC Symptoms 81 alerts	N (%)
ECOG Performance Status	17 (15)
Nausea	16 (14)
Fatigue	13 (12)
Pain	9 (8)
Dyspnea	7 (6)
Wound complication	7 (6)
Fever	6 (5)
Palpitations	3 (3)
Constipation	1 (.04)
Diarrhea	1 (.04)
Urinary frequency	0 (0)

Table 3.

EORTC Symptoms

EORTC Symptom Alert 31 alerts, 84 symptoms	N(%)
Trouble with strenuous activities	14 (17)
Constipation	9 (11)
Pain	9 (11)
Trouble taking a long walk	8 (10)
Need rest	5 (6)
Worried	5 (6)
Limited in doing either work or other daily activities	4 (5)
Limited in pursuing hobbies or other leisure time activities	4 (5)
Diarrhea	3 (4)
Nausea	3 (4)
Pain interfered with daily activities	2 (2)
Short of breath	2 (2)
Felt weak	2 (2)
Felt tired	2 (2)
Felt tense	2 (2)
Felt irritable	2 (2)
Felt depressed	2 (2)
Felt physical condition or medical treatment caused you financial difficulties	2 (2)
Felt physical condition or medical treatment interfered with family life	1 (1)
Emesis	1 (1)
Trouble sleeping	1 (1)
Poor QOL	1 (1)

Patient Satisfaction

Fifty-one patients (46%) completed the exit satisfaction survey. Table 4 shows the patient satisfaction survey (excluding 7 patients who did not use the STAR system to record their symptoms). Ninety-eight percent found STAR easy to use, 84% found it useful, and 82% would recommend it to other patients. One patient reported,

“During acute phase of rehab, looked forward to reporting symptoms (in control of something that is otherwise not controllable) Questions prompted the patient to consider contacting office regarding symptoms. [I] was appreciative of nurse follow up for reported symptoms.”

Table 4.

Patient Satisfaction Results

Question	Response (percentage of patients)
N= 44 pts who completed survey AND used system	Strongly agree	Agree	Disagree	Strongly Disagree
I found the STAR easy to use.	73	25	2	0
I found the STAR to be useful.	25	59	14	2
I found it easy to login to the STAR.	39	34	5	0
I found it easy to enter my symptom information into the STAR.	43	43	11	2
I found the questions in the STAR easy to understand.	52	46	2	0
Using the STAR made it easier for me to remember my symptoms and side effects when I went to office visits.	16	57	25	2
Using the STAR improved discussions with my doctor/nurse.	11	55	27	2
My doctor/nurse used information from the STAR for my care.	20	41	23	2
The quality of my care was improved because of using the STAR.	14	36	42	2
Communication with my doctor/nurse was improved because of the STAR.	16	34	37	5
I would recommend the STAR to other patients.	27	55	11	2
I would like to continue using the STAR in the future.	20	52	20	5

Of the less satisfied participants, two comments explain the issues they had with the STAR system.

“Program too rigid. Questions didn’t account for actual conditions. Was contacted by nurses for symptom that is expected (fatigue). Seems irrelevant”

“Felt very isolated, the technology limits the communication as opposed to human to human interaction”

Seven patients did not use the system to record their symptoms. Five reported that they forgot to use it; 1 of those women also experienced technical difficulties because she could not get her user ID and password to work. Of the remaining 2 patients, 1 did not find it useful because she did not feel sick; the other patient did not give a reason for not using the system. One patient with poor compliance completed two of the seven CTC evaluations, reporting that,

“Survey was the last thing I felt like doing.”

The most common suggestion made by the participants who filled out the satisfaction survey was to add a comment box to the CTC survey. One patient stated,

“A very big, big suggestion ……. the questions asked every week were good, however, the answers provided answers did not necessarily tell the exact truth because there were no comment sections that needed to explain further from the candid answer. There were many times that the answers provided were either black or white and did not allow the patient to elaborate. In fact when I had to answer one of the questions one of the weeks, I did not feel comfortable with picking any of the written answers because it did not describe what I was feeling at the time and because I had to pick one it seemed different from the way I was answering the questions in other weeks which prompted a call from the Surgeon’s nurse making sure I was ok …… I had to explain that there was no comment section to elaborate further how I was feeling because the blanket answers I had to use did not apply totally to the accuracy of my condition. Please add comment section below to each question.”

Another patient stated, similarly,

“No place to explain symptom attribution, wondered “Should I even tell them?” if the cause wasn’t surgery related…”

Clinician satisfaction

Nine nurses participated in the study, 3 of whom left the institution prior to the close of the study. Of the remaining 6 participating nurses, 4 completed the anonymous Clinician Exit Survey. Most nurses did not find STAR helpful. Seventy-five percent did not find the self-assessments of pain and quality of life to be accurate. None of the nurses felt that the STAR system affected their ability to detect patient symptoms. All nurses surveyed reported that the STAR system increased their workload.

DISCUSSION

PRO metrics are practical, meaningful tools for documenting and tracking symptomatology and quality metrics. Such tools will become increasingly important as The Merit-Based Incentive Payment System (MIPS) & Alternative Payment Models (APMs) are incorporated into health care. This study demonstrated that the utilization of a Web-based program for capturing PROs in the postoperative setting, in patients recovering from major gynecologic surgery, is feasible and acceptable for most patients who have a computer and home internet access. Our patients reported a positive experience with the system and would recommend its use. The program helped many patients to feel more empowered in their postoperative recovery.

PROs in the immediate postoperative period can also be useful for improving symptom assessment. Recently, a report from MD Anderson Cancer Center (MDACC) evaluated the PROs of 29 gynecologic oncology patients undergoing laparotomy, using a paper and telephone version of the MDACC Symptom Inventory (MDASI-OC) [11]. They similarly concluded that collecting PROs during the peri- and postoperative period is feasible. Pain and fatigue were among the most burdensome postoperative symptoms. This report was consistent with the pilot study showing poor performance status, nausea, and fatigue as the most common and distressing postoperative symptoms [7].

Our STAR system enhanced the patient experience; however, patients did not always feel that their symptoms could be adequately explained by the options given. Prior to implementation of an electronic PRO collecting system, patients should be involved in the development of a program that adequately reflects their symptomatology. Data from other surgical subspecialties have suggested that clinicians may underestimate patient symptoms [12, 13]. Seventy-five percent of our study nurses did not find the self-assessments accurate. As this was not an aim of the study, further research should be conducted to investigate the discordance between provider and patient perceptions of symptomatology.

In addition, our clinical staff did not find the information reported in the STAR system to be clinically useful. Therefore, we believe that providers should be involved in the development and implementation of such a system in order to appropriately incorporate it into current workflow, thus reducing—rather than adding to—burden. In 2013, Cook et al reported on their experience with an e-health platform in a population of postoperative cardiac patients. They also found the intervention to be feasible and effective; however, it did require increased nursing and ancillary staff input [14]. Some providers understand the benefit and importance of this type of outcome measure but share similar concerns, not only about the work required to implement such a system, but about receiving excessive information or diminishing the patient experience [15]. The best use of a system like this may be to facilitate patient empowerment and clinical research, in addition to or in lieu of postoperative symptom management. Symptom reports could also be summarized and printed for provider review prior to appointments, which could make postoperative appointments more efficient.

Interestingly, we found a difference in the quantity of alerts generated by the CTC and EORTC surveys. Despite being administered less frequently, EORTC generated more alerts than the CTC. Additionally, the distribution of certain symptoms, such as constipation and nausea, varied between the surveys. This could be related to the times at which these surveys were administered, or to the actual composition of the survey questions. The EORTC questionnaire is one of the most thoroughly tested tools among PRO measures in gynecologic oncology, demonstrating good reliability and validity [16]. The NCI CTCAE is the gold standard for assessing patient symptom severity, and has previously been used successfully in Web-based PRO reports [8]. It is possible that the CTC is better able to filter less severe symptomatology. Some patients did report feeling that they were contacted for “concerning symptoms” that were actually an expected aspect of the postoperative recovery process. Going forward, studies or clinical practices that make use of these tools may want to include a comment box in which patients can elaborate on the severity of their symptoms, and suggest whether or not urgent intervention is actually warranted.

The strengths of this study include its design as a large, prospective, longitudinal study of patients undergoing complex open gynecologic surgeries, using two validated PRO questionnaires. This is one of few studies to report on PROs in the immediate postoperative period in this group of patients. The constructive feedback elicited from both patients and providers is one of the strengths of this study, and will facilitate modifications and opportunities to create a more efficient symptom management tool.

The STAR system has the potential for extrapolation to other major surgeries. Caution should be exercised for complete adoption, as this approach may require technology savvy on the part of patients. In our study over 50% of participants reported a college level degree or higher, which may have skewed adoption and participation. This is a significantly greater percentage of highly educated patients than has been reported in other gynecologic cancer populations [17]. Additionally, the economics of a home computer and internet access may limit applicability.

This study demonstrates that a Web-based platform can be used to collect and measure PROs in the immediate postoperative period, that patients are willing to self-report common postoperative symptoms, and that patients found this system useful. Additional work must be done to improve clinician satisfaction, and to determine the system’s usefulness and its optimal incorporation into clinical practice. Electronic methods of capturing PROs have also been reported in the inpatient setting, and in the use of mobile devices [14, 18]. Further research should focus on the utility of these interventions in assessing patient recovery after major gynecologic surgery.

HIGHLIGHTS.

1. A web-based model for assessing patient reported outcomes is feasible in the immediate postoperative period.

2. Many patients feel empowered by documenting and reporting PROs during the post-operative recovery period.

3. A Web-based system for capturing PROs may require additional resources for clinically useful application.