A Prospective Study of the Feasibility and Acceptability of a Web-Based, Electronic Patient-Reported Outcomes System in Assessing Patient Recovery after Major Gynecologic Cancer Surgery

Abstract

Purpose

The purposes of this study are to evaluate the feasibility of capturing patient-reported outcomes (PROs) electronically and to identify the most common distressing symptoms in women recovering from major gynecologic cancer surgery.

Methods

This was a prospective, single-arm pilot study. Eligible participants included those scheduled for a laparotomy for presumed or known gynecologic malignancy. Patients completed a Web-based “STAR” (Symptoms Tracking and Reporting for Patients) questionnaire once preoperatively and weekly during the 6-week postoperative period. The questionnaire consisted of the patient adaptation of the NCI CTCAE 3.0 and EORTC QLQ-C30 3.0. When a patient submitted a response that was concerning, an automated email alert was sent to the clinician. The patient’s assessment of STAR’s usefulness was measured via an exit survey.

Results

Forty-nine patients completed the study. The procedures included the following: hysterectomy +/− staging (67%), resection of tumor (22%), salpingo-oophorectomy (6%), and other (4%). Most patients (82%) completed at least 4 sessions in STAR. The CTC generated 43 alerts. These alerts resulted in 25 telephone contacts with patients, 2 ER referrals, one new appointment, and one pharmaceutical prescription. The 3 most common patient-reported symptoms generating an alert were as follows: poor performance status (19%), nausea (18%), and fatigue (17%). Most patients found STAR useful (80%) and would recommend it to others (85%).

Conclusion

Application of a Web-based, electronic STAR system is feasible in the postoperative period, highly accepted by patients, and warrants further study. Poor performance status, nausea, and fatigue were the most common distressing patient-reported symptoms.

Introduction

There is a paucity of research on patient-reported outcomes (PROs) in gynecologic oncology. Publications from other surgical disciplines have found that physicians tend to underestimate patient symptoms [1, 2]. Reducing the burden of treatment-related symptoms, especially after aggressive therapy, is an important cancer care goal; as a result, the National Institutes of Health (NIH), the Food and Drug Administration (FDA), and other regulatory agencies have recognized the importance of PRO in evaluating disease and treatment [3–5]. Recently, a randomized controlled clinical trial that assessed automated telephone-based monitoring in patients who underwent thoracotomy for lung cancer or lung metastasis demonstrated reduced symptom severity with this approach [6]. However, it is unclear whether those data can be generalized to other populations or procedures.

Gynecologic cancer surgeries are associated with multiple symptoms and moderate to severe complications, especially during the first postoperative month. Patient self-reporting from home between clinic visits can improve the quality of postoperative care through earlier detection of symptoms, improve patient-doctor communication, and provide an efficient means to capture data evaluating the effects of surgical interventions on patient safety and health-related quality of life (HRQOL) [7]. Electronic patient self-reporting has the potential to become a widespread approach for communicating with patients between visits, automatically monitoring for concerning symptoms, reducing morbidity, and improving patient satisfaction overall. Furthermore, data collected online can be analyzed and used for the development of patient education tools. In the clinical trial setting, this approach may improve the efficiency of collecting data from patients and provide a source of symptom and HRQOL information for use as clinical trial endpoints and/or toxicity documentation. Real-time electronic reporting also affords an opportunity to improve the response time of physicians to severe toxicities, which may have particular relevance in the postoperative setting. However, it is not known whether patients will be able or willing to self-report symptoms during this period, or if providers will find this information clinically useful.

The purpose of this pilot study was to: (1) clarify whether patients are willing to self-report common toxicity and HRQOL information using the STAR (Symptom Tracking and Reporting for Patients) system via the internet; (2) evaluate the impact of online symptom self-reporting on patient care; (3) identify the most common distressing symptoms reported by patients after gynecologic cancer surgery; and (4) measure HRQOL during the immediate postoperative period.

Patients and Methods

Patients

The study was approved by the Institutional Review Board of Memorial Sloan-Kettering Cancer Center (MSKCC). All participants gave informed consent. Eligible patients were at least 18 years old, able to read and speak English fluently, and were scheduled for laparotomy for presumed or known gynecologic malignancy at MSKCC. Participants needed to have access to a home computer and have a personal email account.

Study Design

This was a single-arm pilot study in which patients self-reported their symptoms and HRQOL information during the postoperative period using the online platform STAR. At the time of enrollment, each patient underwent a 10-minute training session in the use of the STAR system. Participants were asked to logon to STAR and complete the questionnaires once preoperatively and then weekly starting 7 days after surgery until the 6-week postoperative period ended. Patients were reminded to logon via email. If an enrolled patient failed to logon and self-report within 24 hours of the automated reminder, a second reminder was sent. If the patient again failed to respond, a backup telephone call to the patient was made. A STAR report summarizing the patient symptom trends was printed and made available to the clinician at the time of the postoperative visit. Additionally, if a patient submitted one or more concerning symptoms, according to pre-specified limits, at a given session, an automated email alert with the description of those symptoms was sent in real-time to the study and clinical teams. The email was flagged as “urgent” and titled as “severe patient-reported symptom”. No specific responses to these emails were required, but any actions taken in response to these alerts by clinicians were tracked. It was emphasized to patients that there was no regular monitoring of information entered into STAR, and that they should call their physician’s office if they felt that they required medical attention. In addition, any time a patient entered a potentially concerning response into STAR, a pop-up box appeared on the screen reminding her to consider calling her physician.

Online Platform

Based on expert consultation and literature review, questionnaires and items were selected to focus on symptoms that reflect general and specific postoperative complications and distressing symptoms. The STAR Questionnaire is composed of items from validated instruments, with a focus on postoperative symptom and HRQOL assessment. These include single-item assessments of symptoms using the patient-adaptation of the National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) developed for prior STAR studies [8], and the well-established HRQL instrument, the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 version 3.0 [9]. Each instrument included in this protocol has undergone extensive prior psychometric testing [3, 5]. The CTC questionnaire (Appendix 1) was completed once preoperatively and then weekly after the surgery for 6 weeks. The QLQ-C30 was completed once preoperatively and then on weeks 3 and 6 in the postoperative period.

Outcomes Evaluation

Feasibility

To gauge patients’ willingness to record their experiences, we evaluated the extent to which patients who were enrolled in the protocol used the STAR system to self-report symptoms and HRQOL information. Perfect compliance was defined as logging on and completing all the questions 7 times—once preoperatively, and then weekly for 6 weeks following surgery. In keeping with standards for the assessment of feasibility of online platforms, as a crude measure of feasibility, we considered the study results to suggest that STAR was a strategy warranting larger scale evaluation if approximately 80% of participants logged on to STAR at least 4 times during the study period [8].

HRQOL measure and distressing symptoms

A composite score for the QLQ-C30 version 3.0 questionnaire was calculated following the scoring manual [10]. SPSS software was used to perform the statistical calculations. To examine change over the time, the mean scores of patients who completed the QLQ-C30 version 3.0 questionnaire at the all 3 time points (preoperatively, week 3, and week 6) were compared using repeated-measure ANOVAs. A p value < 0.05 was considered statistically significant. A descriptive analysis of the CTC questionnaire responses was also performed. The most commonly reported distressing symptoms were documented.

Impact of STAR on patient care

“Severe patient-reported symptoms” were monitored and responses documented.

Patient assessment of STAR

Patient assessment of the usefulness of STAR was measured via an exit survey after 6 to 8 weeks of participation in the study. The items of the exit survey had been used in similar research and validated [11].

Results

Enrollment

Between July 2009 and June 2011, 49 eligible patients were enrolled in the study. Three additional patients were initially consented to participate in the study; however, their procedures were cancelled and the patients were excluded from the study.

Patients

Demographic and clinical characteristics of the participants are described in Table 1. The median age was 56 (range, 23–74). Most enrollees (78%) were between the ages of 31 and 64. Three patients (6%) were ≤ 30 years old, and 8 patients (16%) were ≥ 65 years old. The most common performed surgical procedure was hysterectomy +/− staging (33 patients, 67%). Approximately 80% of patients were diagnosed with invasive cancer on final pathologic evaluation. Most (94%) reported use of the internet at least once a week. The majority of patients held a graduate or college degree (88%), and only 3 patients (6%) were not educated beyond high school.

Table 1.

Patient characteristics

Characteristics	Number of Patients	%
ASA class
II	28	57
III	21	43
Surgical procedures performed
Hysterectomy +/− staging	33	67
Resection of tumor	11	22
Salpingo-oophorectomy	3	6
Other	2	4
Disease origin
Ovary	26	53
Uterus	16	33
Fallopian tubes	4	8
Other	3	6
Final pathologic evaluation
Malignancy	39	80
Benign disease	8	16
Borderline tumors	2	4
Computer at home
Yes	48	98
Unknown	1	2
Internet use frequency
More than once a week	43	88
Once a week	3	6
Less than once a week	2	4
Unknown	1	2
Highest educational level
Professional/graduate degree	16	33
College degree	20	41
Some college	7	14
High school or less	3	6
Unknown	3	6
Job status
Employed	35	71
Homemaker	6	12
Retired	5	10
Student	1	2
Unknown	2	4

ASA class = American Society of Anesthesiologists physical status class

STAR use

Figure 1 demonstrates the proportion of patients who completed a STAR session at each consecutive week. As the postoperative period elapsed, the compliance of patients gradually decreased. Approximately 92% of patients completed the preoperative session in STAR, and 74% of patients responded to the week 6 STAR session. The majority of patients (82%) completed at least 4 of 7 total sessions in STAR.

Figure 1.

Proportion of patients using STAR (Symptoms Tracking and Reporting for Patients) at a given session, starting postoperatively to week 6 postoperatively (x axis: time points, y axis: proportion of patients)

Alerts

During the study period, the CTC generated 43 automated email alerts. The responses of 25 (51%) of 49 patients generated email alerts. Twelve patients had one generated alert, 9 patients had 2 alerts, 2 patients had 3 alerts, and 2 patients had 4 alerts. These alerts resulted in 25 telephone contacts with patients. Two patients were referred to the ER—one for an incisional check and the other for evaluation of dyspnea. A new appointment was given to one patient due to complaints of nausea and fatigue. A prescription for pain medication was ordered for one patient. Overall, 72 patient-reported symptoms generating an alert were documented in our study with the following distribution: Eastern Cooperative Oncology Group (ECOG) performance status, 14 (19%); nausea, 13 (18%); fatigue, 12 (17%); pain, 11 (15%); dyspnea, 7 (10%); fever, 5 (7%); and wound complications, 4 (6%). Palpitations (3, 4%), vomiting (1, 1%), diarrhea (1, 1%), and constipation (1, 1%) were less common, and none of the patients reported a distressing level of urinary frequency/urgency.

HRQOL

Thirty-one (63%) of 49 patients completed the QLQ-C30 version 3.0 questionnaire at all 3 time points (preoperatively, week 3, and week 6). A repeated-measure ANOVA was performed for statistical evaluation, and the results are presented in Table 2. Our data demonstrate that dyspnea and gastrointestinal symptoms (nausea and vomiting, constipation, and diarrhea) did not change significantly during the time of observation (p>0.05). An improvement in cognitive, emotional, and social functioning was noted by postoperative week 6 (p<0.05). Pain, fatigue, insomnia, appetite loss, and global functioning worsened significantly by week 3, but returned almost to the baseline by postoperative week 6 (p<0.05). Physical and role functioning declined significantly 3 weeks after surgery, and although improved at the 6-week time point, they did not return to the baseline (p<0.05). Financial difficulties caused by the disease and its treatment continued to worsen significantly in the postoperative period (p<0.05).

Table 2.

Patient-reported health-related quality of life (HRQOL) (QLQ-C30 version 3.0 questionnaire)

Parameter	Preoperative Mean (95% CI)	Postoperative WEEK 3 Mean (95% CI)	Postoperative WEEK 6 Mean (95% CI)	F statistics	P value
Physical functioning	77 (68, 87)	28 (20, 35)	48 (38, 58)	54.9	<0.0001
Role functioning	77 (68, 86)	34 (25, 44)	61 (48, 73)	30.8	<0.0001
Cognitive functioning	62 (54, 69)	58 (52, 64)	70 (61, 79)	4.3	0.018
Emotional functioning	56 (48, 65)	62 (56, 69)	72 (63, 81)	7.8	0.001
Social functioning	51 (40, 61)	67 (58, 75)	75 (65, 86)	15.4	<0.0001
Financial difficulties	49 (39, 60)	33 (25, 42)	25 (14, 35)	15.4	<0.0001
Global functioning	81 (74, 88)	62 (54, 70)	73 (64, 81)	11.4	<0.0001
Dyspnea	6.4 (1.5, 11.4)	13 (6, 20)	10 (1.8, 17.5)	1.4	0.245
Pain	17 (11, 23)	32 (25, 39)	23 (14, 31)	9.7	<0.0001
Nausea and vomiting	6.5 (1.5, 11.4)	13 (6, 20)	9.7 (1.8, 17.5)	1.6	0.206
Constipation	5.4 (0.8, 10)	13 (7, 19)	10 (1.8, 17.5)	2.1	0.128
Diarrhea	6.5 (1.5, 11.4)	13 (6, 20)	9.7 (1.8, 17.5)	1.6	0.206
Fatigue	26.5 (18, 35)	51 (41, 60)	35 (25, 46)	10.3	<0.0001
Insomnia	26 (18, 33)	51 (41, 60)	37 (25, 48)	10.1	<0.0001
Appetite loss	26 (18, 33)	51 (41, 60)	35 (25, 46)	11.5	<0.0001

Patient satisfaction survey

Twenty-six (53%) of 49 patients completed an exit questionnaire. Results are shown in Table 3. Most patients found STAR easy to use (92%) and the questions easy to understand (100%). The majority expressed a desire to continue using STAR (85%) and would recommend it to other patients (85%).

Table 3.

Patient Impressions of STAR (n=26)

Variable	Response (percentage of patients)
	Strongly Agree	Agree	Disagree	Strongly Disagree
I found STAR easy to use	73	19	8	0
I found STAR to be useful	20	60	20	0
Questions were easy to understand	54	46	0	0
STAR made it easier for me to remember symptoms at my clinic visits	12	52	36	0
STAR improved discussion with my doctor/nurse	0	63	37	0
STAR improved my communication with my doctor/nurse	4	42	50	4
STAR made me feel more in control of my own care	8	38	50	4
STAR improved the quality of my care	4	42	54	0
I would like to continue using STAR	19	66	15	0
I would recommend STAR to other patients	27	58	15	0

Discussion

Our study demonstrates that online symptom self-reporting in the early postoperative period is a feasible outpatient care strategy in patients recovering from major gynecologic cancer surgery. This conclusion is based on the fact that the majority of patients (82%) completed at least 4 of 7 total sessions in STAR. These findings establish the groundwork for further evaluations, including assessment of feasibility in other populations (eg, underserved, non-English speaking), a randomized trial to determine if patient self-reporting in the postoperative period improves clinical outcomes (complication, readmission and reoperation rates, satisfaction with care), and evaluation of this data collection method in a multi-center clinical trial setting (eg, nested in the Gynecologic Oncology Group or the Cancer and Leukemia Group B treatment trials). The importance of the patient self-reporting of adverse events has been recently recognized by the NCI. As a result, the STAR system design is currently being used to develop a standard tool for adverse event monitoring, referred to as PRO-CTCAE [12]. Follow-up work is needed to refine the questionnaire items and electronic platform. In addition, for patients who are unwilling or unable to complete electronic questionnaires, back-up data collection methods such as automated telephone interactive voice response (IVR) merits further investigation.

Our data show that the online symptom self-reporting in the early postoperative period could be helpful in the early identification of disturbing postoperative symptoms. Timely symptom reporting could potentially reduce postoperative symptom burden, severity of complications, and readmission/hospitalization; however, definitive predictors have not yet been established. Our study is the first step in the evaluation of online self-reporting and in the identification of those predictors. Moreover, early detection of toxicities and complications has a significant value in clinical trials, and systems such as STAR may add a substantial benefit to a rapid reporting [13, 14].

The lack of normative data available for the symptoms during the postoperative period has implications for patients and clinicians. One of the goals of our study was to identify the most common distressing symptoms reported by patients after gynecologic cancer surgery. We found that performance status, nausea, fatigue, pain, dyspnea, fever, and wound complications were the most common distressing symptoms during the postoperative period. Interestingly, palpitations, vomiting, diarrhea and constipation were reported less commonly (<5%). Without information on the symptoms from the perspective of the patient, it is difficult to adequately prepare patients for what to expect as they recover from major surgery. Montgomery et al found, in a cohort of women undergoing ambulatory breast cancer surgery, that postsurgical pain and fatigue were related to pre-surgical expectations [15]. Our data will be helpful for the preoperative education of patients regarding symptoms to be expected during the postoperative period.

In many trials of postoperative gynecologic cancer patients, HRQOL surveys have been administered at baseline and 3 months postoperatively—time points that do not address the immediate postoperative period. Due to lack of data, physicians’ expectations of what is normal HRQOL in the immediate postoperative period are based largely on extrapolations and anecdotal experience. To the best of our knowledge, this is the first trial that prospectively assessed patient-reported HRQOL during the immediate postoperative period after major gynecologic cancer surgery. Our results demonstrate that cognitive, emotional, and social functioning improved significantly by postoperative week 6. However, other aspects of HRQOL, such as physical and role functioning, declined significantly during the postoperative period and did not return to baseline levels by week 6. These results establish a future direction for patient-centered HRQOL studies and provide evidence-based data for patients and health-care providers in what to anticipate during the immediate postoperative recovery period.

The process of self-reporting has a positive impact on patient satisfaction [7]. The majority of patients in our study expressed a desire to continue using STAR and would recommend it to others. Moreover, all patients (100%) found questions easy to understand and approximately 2 of 3 patients (63%) reported that STAR improved the discussion with providers. Those results need to be interpreted with caution due to the response rate of 53%.

There are several limitations of this study. The enrolled patients were only women with suspected or conformed gynecologic malignancies undergoing surgery in a single tertiary cancer center, and the results may not be extrapolated to other populations. Our patients reported a high level of internet use, and the majority of patients were well educated. Moreover, patients were reminded to logon to STAR, which may have led to a high compliance rate.

In conclusion, our study demonstrates the feasibility of collecting Web-based patient-reported symptoms during recovery after major gynecologic cancer surgery. Such automated systems, like STAR, provide an excellent opportunity to obtain invaluable clinical data from a patient perspective and trigger early intervention that has the potential to improve patient outcomes. Further research should focus on determining the potential value of PRO for health care providers, finding the optimal approach for the integration of PRO into clinical practice, and identifying factors associated with compliance.

Appendix A

Patient language adaptation of the National Cancer Institute Common Terminology Criteria for Adverse Events (version 3.0) and Eastern Cooperative Oncology Group performance status

1. Nausea

   • Grade 0: I have not had nausea.
   • Grade 1 (mild): I have lost my appetite due to nausea, but I am able to eat.
   • Grade 2 (moderate): The amount I eat or drink is decreased due to nausea, but I have not lost weight or become dehydrated or malnourished. I have not needed intravenous fluids for more than 24 hours.
   • Grade 3 (severe): I have not been eating or drinking adequately and I have required intravenous fluids, tube feedings, or intravenous nutrition (TPN) for more than 24 hours.
2. Vomiting

   • Grade 0: I have not had vomiting.
   • Grade 1 (mild): I have had vomiting, but I have not vomited more than once in a 24-hour period.
   • Grade 2 (moderate): I have had vomiting between 2–5 times in a 24-hour period, or I have needed intravenous fluids for less than 24 hours due to vomiting.
   • Grade 3 (severe): I have had vomiting 6 or more times over a 24-hour period, or I have needed intravenous fluids or nutrition for more than 24 hours due to vomiting
   • Grade 4 (disabling): My vomiting has been disabling.
3. Diarrhea

   • Grade 0: I have not had more bowel movements than usual each day.
   • Grade 1(mild): I have had 1–3 bowel movements more than usual each day.
   • Grade 2 (moderate): I have had 4–6 bowel movements more than usual each day, but I have not needed intravenous fluids for more than 24 hours, and diarrhea is not interfering with my normal daily activities.
   • Grade 3 (severe): I have had more than 6 bowel movements more than usual each day, or I have needed intravenous fluids for more than 24 hours and diarrhea is interfering with my normal daily activities.
4. Constipation

   • Grade 0: I have not had constipation.
   • Grade 1 (mild): I have had occasional or intermittent constipation, or I am occasionally using stool softeners, laxatives, enemas, or dietary changes to help move my bowels.
   • Grade 2 (moderate): I have had persistent (ongoing) constipation, and cannot have bowel movements without the regular use of laxatives or enemas.
   • Grade 3 (severe): Constipation has interfered with my normal daily activities, or I have required manual disimpaction.
5. Dyspnea (shortness of breath)

   • Grade 0: I have had no shortness of breath (with exercise or rest).
   • Grade 1 (mild): I have been short of breath with exercise, but I can walk up one flight of stairs without stopping.
   • Grade 2 (moderate): I have been short of breath with exercise but I am not able to walk one flight of stairs or one city block without stopping.
   • Grade 3(severe): I have been short of breath during my normal daily activities (e.g., dressing, showering, cleaning, cooking).
   • Grade 4 (disabling): I have been short of breath even when I am resting in a bed or chair.
6. Fatigue

   • Grade 0: I have not had fatigue compared to my usual baseline.
   • Grade 1 (mild): I have had mild fatigue compared to my usual baseline.
   • Grade 2 (moderate): I have had moderate fatigue compared to my usual baseline, or fatigue causing moderate difficulty performing my normal daily activities.
   • Grade 3 (severe): I have had severe fatigue that interferes with my normal daily activities.
   • Grade 4 (disabling): My fatigue has been disabling.
7. Urinary frequency or urgency

   • Grade 0: I have not had problems with urinary frequency or urgency.
   • Grade 1: The number of times I need to urinate during the day or night is twice my prior normal level; or I have some wetting of the bed.
   • Grade 2: The number of times I need to urinate is more than twice my prior normal level, but < 1 per h.
   • Grade 3: I need to urinate > 1 per h, or I frequently feel an urge to urinate, or I require the use of catheter in my bladder to control incontinence.
8. Pain

   • Grade 0: I have had no pain.
   • Grade 1 (mild): I have had mild pain, but it does not interfere with my normal functioning.
   • Grade 2 (moderate): I have had moderate pain, and my pain or my use of pain medication interferes with my normal functioning but I am still able to carry out my normal daily activities.
   • Grade 3 (severe): I have had severe pain, and my pain or my use of pain medications severely interferes with my normal daily activities.
   • Grade 4 (disabling): My pain has been disabling.
9. Palpitations

   • Grade 0: I have not had palpitations.
   • Grade 1: I have had palpitations with or without other with symptoms (e.g., lightheadedness or shortness of breath).
10. Fever

    • Grade 0: I have not had fever.
    • Grade 1: I have had fever (>100.4°F or > 38.0°C).
11. Wound complications

    • Grade 0: I have not had wound complications.
    • Grade 1 (mild): I have had mild wound complications including mild redness, and mild tenderness, but no discharge, swelling and no wound opening.
    • Grade 2 (moderate): I have had moderate wound complications including moderate redness and tenderness, but no discharge, swelling, or wound opening.
    • Grade 3 (severe): I have had severe wound complications including redness and pain, discharge or swelling or wound opening.
12. ECOG performance status

    • Grade 0: I am fully active and able to carry out activities the same as before my cancer diagnosis, without any restrictions.
    • Grade 1: I have difficulty with physically strenuous activity but I am able to walk and carry out work that is light or based in one location; such as light house-work or office-work.
    • Grade 2: I can walk and take care of myself, but I am not able to carry out work activities; I am up and about more than half the hours that I am awake.
    • Grade 3: I am capable only of limited self-care and spend more than half the hours that I am awake in bed or in a chair.
    • Grade 4: I am completely disabled, cannot carry on any self-care, and am totally confined to a bed or chair.