Abstract
Purpose:
Remote symptom monitoring shows promise promoting post-discharge contact between patients and clinicians. Unique strategies may be needed to tailor reach and engagement to specific patient populations. We aimed to assess implementation and effectiveness outcomes of a patient-reported symptom assessment tool (“Recovery Tracker”) after major urologic operations.
Materials and Methods:
Patents undergoing one of four procedures (2016–2022) at a metropolitan cancer center: radical prostatectomy, nephrectomy, radical cystectomy, and retroperitoneal lymph node dissection for testicular cancer. Electronic delivery of Recovery Tracker was embedded in routine perioperative patient care. Outcomes were assessed according to the RE-AIM framework. Descriptive statistics was reported for “reach,” “effectiveness,” and “adoption”; a linear mixed-effects model for “implementation”; and a general additive model and fixed-effects meta-analysis for “maintenance”.
Results:
The cohort comprised 8,934 patients. Reach, defined as patients correctly receiving the survey, was 98% overall with 81% (95% CI 80-82%) of patients completing at least one survey and the majority completing >7. Median time to completion was <2 minutes and improved as patients completed more surveys (p<0.001), with slight variation among procedure types. The survey was effective, initiating patient-clinician contact when alert thresholds were triggered, with a marginal increase in the need for clinician office phone calls. Patient engagement with Recovery Tracker was maintained over several years, with a slight improvement after the addition of email reminders (between 3-8%).
Conclusion:
Implementing a daily electronic survey after hospital discharge following major urologic surgeries is feasible and used often by patients.
Keywords: patient monitoring, patient-reported outcome measures, postoperative period, symptom assessment
Introduction
Greater adoption of minimally invasive surgery and enhanced recovery protocols has shortened inpatient hospital stays and shifted much of the convalescence period away from a monitored inpatient setting and towards patients’ homes. Traditionally, post-discharge communication between the patient and surgical team is on a reactive basis and may occur only once symptoms have progressed to the point that management requires a return to the hospital or office. Mitigation of developing complications requires prompt detection and intervention. At-home symptom monitoring has shown promise in achieving this goal, but its feasibility across the breadth of major urologic oncology surgeries is not fully understood.
To systematically monitor patient recovery after discharge, our team designed and implemented “Recovery Tracker” (RT), an electronic remote symptom-monitoring platform.1,2 RT aims to measure and improve post-discharge care through early detection processes where patients self-report their symptoms via an online portal. Since the urologic oncology setting includes a diverse patient population and variety of post-surgical needs, these patients have unique barriers or facilitators when considering an electronic symptom tracker. Therefore, the goals of this study were to: 1) report feasibility of implementing RT for these procedures, 2) identify potential gaps in utilization, 3) report measures of symptom severity and frequency in the early post-discharge period, and 4) categorize the utilization of clinical resources associated with symptom management. A better understanding of the feasibility and resources required to implement a similar system will help other institutions promote patient-level engagement and early detection of complications following urologic oncology procedures.
Materials and Methods
Patient population and setting
Patients undergoing urologic oncology procedures at Memorial Sloan Kettering Cancer Center (MSK), either at the specialized Josie Robertson Surgical Center (JRSC) or main campus, are assigned a patient-reported outcomes version of the common terminology criteria for adverse events (“Recovery Tracker”, RT). RT is a brief, electronically delivered survey that is sent on each of the 10 days after discharge. It is administered via our secure patient portal, and patients are automatically and prospectively identified based on procedural coding. Assessed symptoms include pain, fatigue, constipation, nausea, vomiting, fever, chills, shortness of breath, and wound complications (redness, bruising, or swelling at site of surgery). Symptoms related to complications specific to each urologic procedure are also included, including catheter and stoma-related issues (Data Supplement 1). The symptom-tracking survey content and platform were created through an interdisciplinary, evidence-based approach with disease site-specific symptoms, as previously described.1,2 To promote patient engagement, we used a multistep patient education strategy which begins at the pre-operative consent appointment, was reinforced during the discharge process by nursing, and is reiterated with a phone call within 48 hours of discharge. After initial implementation, a reminder email was added to the protocol to promote greater patient engagement.
To monitor patient response and facilitate early intervention of developing issues, we developed an alert system wherein thresholds were defined for select symptoms. Yellow alerts, reflecting intermediate severity symptoms, prompt a notification to the surgeon’s office team, and the patient is assessed by telephone or portal message during regular office business hours. Red alerts lead to an electronic pop-up instructing the patient to contact the doctor immediately in addition to an electronic office practice notification.
Evaluation framework: RE-AIM
We wished to evaluate implementation outcomes with a focus on feasibility, patient engagement, and additional marginal burden of resources. We applied the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) evaluation framework3,4, a widely used, practical, and systematic means of evaluating health interventions and strategies. This approach also allows researchers to align and define outcomes using standard domains and definitions to promote generalizability.
We analyzed RT surveys from their initiation at JRSC (September 2016) and at the main campus (June 2021) for four major urologic oncology procedures: radical prostatectomy, radical cystectomy, nephrectomy (partial and radical), and retroperitoneal lymph node dissection (RPLND) for testis cancer. Missing data were imputed using multiple imputation where estimates were combined using Rubin’s rules. For this study, the “intervention” is the RT survey, and the “implementation” strategy encompasses all components of our attempts to get patients to engage with RT.
Outcomes in the five RE-AIM domains
The “reach” domain explores whether the target population is correctly identified and sent the survey. This is an early marker of feasibility because patients cannot respond to the survey if they are not identified correctly. We report the proportion of patients who were correctly assigned at least one survey among those who underwent an eligible procedure. For the “adoption” domain, we calculated the proportion of patients who completed at least one survey (numerator) among those assigned at least one survey (denominator), including median and quartiles of surveys completed. The “implementation” domain was addressed by estimating the time to survey completion. We used a linear mixed-effects model with patients as a random effect and time to survey completion in minutes as the outcome, adjusted for number of prior completed surveys to account for the fact that patients become more familiar with surveys the more times they complete it.5
The “effectiveness” domain evaluates how well the intervention promotes communication between the patient and physician practice. We assessed this at the patient and nursing levels: the number of yellow or red alerts generated per day and the number of nursing calls triggered by RT. We summarized the number of nursing calls triggered by RT using five different definitions: (i) only postoperative assessment (POA) calls related to relevant RT symptoms, (ii) only POA calls triggered by unspecified general symptoms, (iii) any POA calls related to managing relevant symptoms and/or general symptom management, (iv) or any POA call regardless of reason (e.g. care coordination, medications, education, etc.). We compared these definitions to (v) any call.
For the “maintenance” domain, we assessed whether there was a temporal change in response rates which explores whether enthusiasm and engagement wane over time after implementation. We began sending additional email reminders in May 2022 and modified the reminders in July 2022. We were interested in whether the addition of our email reminder impacted engagement. We projected the pre-email change-over-time model onto the post-implementation period to estimate whether the proportion of responders was different than expected, had there not been any email reminders, with a 95% CI generated via 2000 bootstraps. To ascertain whether the effect of email reminders differed by surgery type, we tested for an overall effect size different from zero between surgery types using a fixed-effects meta-analysis.
All analyses were conducted using R (v4.3.1) with the tidyverse (v2.0.0) and gtsummary (v1.7.0) packages [1–3]. This study was approved by MSK’s Institutional Review Board (IRB #18-260), and informed consent was waived because the study was considered minimal risk and only involved survey procedures. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.
Results
8,934 patients underwent radical prostatectomy, cystectomy, nephrectomy (partial and radical), or RPLND, of which 8,753 patients were assigned at least one RT survey, corresponding with an overall assignment rate of 98%. Patient clinicodemographics are consistent with expected characteristics for the included procedures (Table 1).
Table 1.
Patient, disease, and surgical characteristics by completion status among procedure type. Data are presented as N (%) or median (IQR). P-values were generated using fixed-effect meta-analysis.
| Variable | Cystectomy | Nephrectomy | RPLND | Radical prostatectomy | P | ||||
|---|---|---|---|---|---|---|---|---|---|
| Responder status | Completed at least one RT (n=310) | Non-responder (n=98) | Completed at least one RT (n=1906) | Non-responder (n=804) | Completed at least one RT (n=159) | Non-responder (n=18) | Completed at least one RT (n=4623) | Non-responder (n=835) | |
| Number of assessments completed* | 7 (5, 8) | — | 6 (3, 8) | — | 8 (7, 9) | — | 7 (5, 8) | — | |
| Age | 70 (63, 77) | 70 (63, 78) | 60 (53, 67) | 61 (55, 67) | 31 (25, 36) | 32 (26, 36) | 63 (58, 68) | 63 (58, 69) | <0.001 |
| Male | 238 (77%) | 74 (75%) | 1181 (62%) | 498 (62%) | 159 (100%) | 18 (100%) | 4623 (100%) | 835 (100%) | 0.8 |
| BMI | 28.6 (25.3, 32.7) | 27.6 (24.6, 31.2) | 29.4 (26.4, 33.6) | 29.3 (26.4, 33.1) | 27.2 (24.2, 30.2) | 28.3 (25.5, 31.5) | 28 (25.4, 31) | 28.2 (25.8, 31.4) | 0.7 |
| LOS in hours | — | — | 28.9 (26.2, 40.5) | 33.6 (27.2, 52.1) | - | - | 27 (25, 29) | 28.2 (25.8, 31.4) | <0.001 |
| LOS in days | 6.3 (5.3, 8.9) | 6.8 (5.3, 9.5) | — | — | 7.2 (6.2, 8.2) | 7.6 (6.3, 8.6) | — | — | 0.7 |
| OR time in hours | 6.0 (4.8, 7.6) | 6.0 (4.9, 7.5) | 2.4 (2.0, 3.1) | 2.5 (2.0, 3.2) | 4.3 (3.7, 5.6) | 4.4 (3.9, 5.9) | 4.1 (3.6, 4.6) | 4.1 (3.7, 4.7) | 0.004 |
| JRSC (vs. main) | 0 (0%) | 0 (0%) | 934 (49%) | 394 (49%) | 0 (0%) | 0 (0%) | 4253 (92%) | 759 (91%) | 0.10 |
| MIS | 94 (30%) | 28 (28%) | 1372 (72%) | 571 (71%) | 5 (3%) | 1 (4%) | 4613 (100%) | 834 (100%) | 0.020 |
| ASA score of 3/4 | 284 (92%) | 92 (94%) | 1219 (64%) | 570 (71%) | 106 (67%) | 14 (75%) | 1802 (39%) | 400 (48%) | <0.001 |
ASA, American Society of Anesthesiologists; JRSC, Josie Robertson Surgical Center; LOS, length of stay; MIS, minimally invasive surgery; RPLND, retroperitoneal lymph node dissection; RT, Recovery Tracker.
Reach
Nearly all patients were assigned at least one RT survey: 99% of cystectomy patients, 95% of nephrectomy patients, 100% of testis RPLND patients, and 99% of radical prostatectomy patients.
Adoption
81% (95% CI 80-82%) of patients who received a survey completed at least one. The lowest level of adoption was among patients undergoing nephrectomy (70%), while the highest was among patients who underwent RPLND (90%). The adoption rate was 76% for cystectomy and 85% for prostatectomy. Within groups of patients undergoing each procedure, there were also variations in response rates. For example, for those having a nephrectomy, patients who did not respond had very slightly longer lengths of stay and worse ASA scores (Table 1). The median number of surveys completed was seven, six, eight, and seven for cystectomy, nephrectomy, RPLND, and prostatectomy, respectively (Table 1).
Effectiveness
The proportion of patients with a yellow and red alert for at least one RT question on each postoperative day is shown in Figure 1 and Supplementary Table 1. There was a notable increase in the proportion of patients with alerts on postoperative day 2 to 3 before gradually plateauing. When assessing the increase in nursing phone calls associated with RT, depending on the surgical procedure, between 19 and 46% of patients received any postoperative call in the 10 days following discharge. However, among patients who received at least one nursing call, the majority received a call triggered by the RT and symptoms across procedures (Table 2). From a resource perspective, no additional nursing or ancillary health staff were required to help manage this slight increase in patient call volume.
Fig. 1.
Patients whose responses triggered a yellow or red alert for at least one RT question on each post-discharge day separately. Patients who responded to the survey triggering both a red and yellow alert are included in the count for red alert. We include surveys starting at day 2, since historically, patients do not respond to the first survey post-discharge.
Table 2.
Proportion of patients receiving at least one nursing call triggered by the RT by different definitions of what constitutes a call triggered by the RT.
| Definition of nursing call triggered by RT | |||||
|---|---|---|---|---|---|
| Group | Only POA call related to managing relevant symptoms | Only calls related to POA and managing symptoms | Only POA noted with relevant symptoms | Any call related to POA | Any call |
| Cystectomy (n=310) | 142 (46%) | 53 (17%) | 51 (17%) | 132 (43%) | 205 (66%) |
| Nephrectomy (n=1906) | 609 (32%) | 76 (4%) | 73 (4%) | 569 (30%) | 700 (37%) |
| RPLND (n=159) | 73 (46%) | 18 (11%) | 18 (11%) | 71 (45%) | 94 (63%) |
| Radical prostatectomy (n=4623) | 895 (19%) | 265 (6%) | 263 (6%) | 885 (19%) | 1072 (23%) |
POA, postoperative assessment; RPLND, retroperitoneal lymph node dissection; RT, Recovery Tracker.
Implementation
Median time to survey completion was ≤2 minutes for all patients. The number of prior surveys completed and surgery type was independently associated with decreased time required (p<0.001). When compared to radical prostatectomy patients, cystectomy patients take longer, while nephrectomy and RPLND patients are faster (Table 3, overall p<0.001).
Table 3.
Linear mixed-effects model where time to survey completion in minutes is the outcome. We include surveys starting at day 2, since historically, patients do not respond to the first survey post-discharge. β is the change in time to completion for a one-unit change in the predictor.
| Characteristic | β | 95% CI | p-value |
|---|---|---|---|
| Number of prior surveys completed | −0.16 | −0.16, −0.15 | <0.001 |
| Surgery type | <0.001 | ||
| Radical prostatectomy | — | — | |
| Cystectomy | 0.23 | 0.07, 0.38 | |
| Nephrectomy | −0.09 | −0.18, −0.01 | |
| RPLND | −0.77 | −0.98, −0.56 |
RPLND, retroperitoneal lymph node dissection.
Maintenance
The proportion of patients who responded to at least one RT survey did not decrease over time (Figure 2). After adjusting for the secular trend, there was an overall effect size different from zero in the probability that a patient will complete a survey across surgery types (Table 4). The largest improvement over what was expected in the post-email period was among cystectomy patients (8.2%, CI 95% 2.6%, 13.8%).
Fig. 2.
Proportion of patients who responded to at least one RT survey over time. The vertical black lines represent the initiation of sending email reminders to patients, which started in May 2022 and was modified in July 2022. The solid blue lines represent the proportion of patients observed to have completed at least one survey. The dotted blue line represents the proportion predicted to complete one survey had there been no email reminders.
Table 4.
Probability differences of survey completion with and without email reminders. Overall email reminder effect was calculated as the overall treatment effect size from fixed-effect meta-analysis. P-value was calculated using fixed-effects meta-analysis.
| Surgery type | Probability with email reminders | Probability without email reminder | Difference | 95% CI | P-value |
|---|---|---|---|---|---|
| Overall | 4.5% | 3.1%, 6% | <0.001 | ||
| Cystectomy | 80.0% | 71.8% | 8.2% | 2.6%, 13.8% | |
| Nephrectomy | 84.7% | 78.2% | 6.5% | 3.3%, 9.7% | |
| RPLND | 92.3% | 84.2% | 8.1% | 2.6%, 13.6% | |
| Radical prostatectomy | 92.9% | 89.7% | 3.2% | 1.4%, 5% |
RPLND, retroperitoneal lymph node dissection.
Discussion
Remote symptom monitoring is feasible and can be implemented following major urologic oncologic operations. Our approach was associated with good patient reach, adoption, and minimal additional time required for patients without a significant burden on clinical staff. Symptoms that were severe beyond a predetermined threshold led to clinician-patient contact. It remains to be seen if this early identification will translate into reducing avoidable emergency room visits or readmissions over time. We also identify room for improvement with our strategy. Since not all patients were assigned the survey, certain patient characteristics were associated with lesser engagement, generating concern that with our approach, variations in care delivery can develop or already exist among patients. We believe that our results support the growing evidence that systematic assessment of patient-reported outcomes provides a system-level opportunity to decrease the time required for health professionals to learn about and respond to severe postoperative symptoms in the urologic oncology patient population.
While almost all patients correctly received the RT survey following discharge, actual participation was lower. One issue is that, to use the RT, patients needed to be registered in our online patient portal; lesser participation in the portal among older patients partly explains the nonresponse. A difference in response was also seen among patients with a higher ASA score, which reflects a greater comorbidity burden and potentially patients with other competing health needs. There were no differences in adoption in patients undergoing either RALP or nephrectomy at our main hospital versus our outpatient surgery center, which was reassuring for generalizability since there are different staff and workflows at each location.
That said, overall adoption and engagement with the survey were quite good; most patients responded to more than half of the available surveys. This continued participation, along with the <2-minute time required for completion, suggests that the survey was not burdensome and that rather than a waning interest or desire to participate in general, the major barrier to overcome is getting people involved. Prior qualitative interviews suggest that people who used RT found it to be a helpful extension of their care that was easy to use and provided reassurance while helping to set expectations postoperatively.6 However, in that report, some patients experienced symptoms that were not available as “options,” so further development of disease-specific symptom assessment instruments may be beneficial.
In addition to demonstrating feasibility, we report the first systemic, temporal description of patient-reported symptomatic concerns in the early postoperative period for four major urologic oncology procedures. Most patients did not report significant symptoms, but there were clear spikes in more severe symptoms in the first 2-3 days following discharge, particularly for prostatectomy and nephrectomy. This is similar to previously reported early issues with pain and shortness of breath following prostatectomy.7 It is also possible that some of these alerts were misclassifications and were actually related to common catheter-related issues in RALP patients.
Although we have previously demonstrated that our symptom-tracking platform successfully reduces avoidable emergency department visits and readmission rates in ambulatory surgery patients and after short-stay urologic procedures such as RALP1, future research is needed to assess whether this benefit remains for more complex inpatient surgical procedures, particularly cystectomy given its high 90-day complication rate.7 The generalizability of this approach to remote symptom monitoring is critical, and we hope that by using a well-described implementation evaluation framework such as RE-AIM, other groups can meaningfully compare and contrast outcomes and differences in multilevel strategies.
This study’s limitations include the single-institution design and that patients are probably disinclined to complete the survey if they have no symptoms and an uncomplicated recovery. Therefore, the number of alerts for severe symptoms may overestimate overall patient concerns during the immediate postoperative period. Balancing outcomes, such as the burden of time required to address alerts, particularly “false positive” or unneeded alerts, must also be considered before widespread adoption or intensification of symptom trackers. It is also possible that our particular strategy, which was digitally built within our proprietary patient portal, will not be generalizable to other centers. However, the similarity between our online platform and commonly used electronic health records makes this less likely. Not discussed here are the time and resources required to develop this type of intervention, which may limit other centers’ ability to adopt a similar approach. Lastly, not all practices or hospitals have this type of electronic platform available, thus limiting generalizability to those who have EHRs that do not integrate survey delivery functionality.
Conclusions
Implementing an electronic survey where patients report postoperative recovery and symptoms for 10 days after hospital discharge following major urologic surgeries is feasible and used often by patients. Improving implementation outcomes may be possible through future work that addresses gaps in delivery, usability, and access to RT in a more personalized manner.
Supplementary Material
Context Summary:
Key Objective
How effective and feasible is implementing a remote symptom-monitoring tool (“Recovery Tracker”) for post-discharge care in patients undergoing major urologic oncology surgeries?
Knowledge Generated
The Recovery Tracker achieved a 98% survey delivery rate, with 81% of patients completing at least one survey. Patient engagement remained stable over the study period, supported by email reminders that increased response rates by up to 8%, and resulted in a marginal increase in the need for clinician office phone calls.
Relevance
The authors show feasibility of a digital recovery tracker in clinical routine to enable effective and improved follow up.
Funding:
This research was funded in part by Cancer Center Grant P30 CA008748. RSM, SC, and JC were supported by a grant from Pfizer Inc. through a collaboration with PROTEUS. SG was supported by NIH/NCI T32-CA082088. The funders had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data, or preparation review, or approval of the manuscript.
Abbreviations:
- POA
postoperative assessment
- RE-AIM
reach, effectiveness, adoption, implementation, and maintenance
- RPLND
retroperitoneal lymph node dissection
- RT
Recovery Tracker
Footnotes
Conflict of Interest Disclosures: The authors have no conflicts of interest to disclose.
References:
- 1.Simon BA, Assel MJ, Tin AL, et al. Association Between Electronic Patient Symptom Reporting With Alerts and Potentially Avoidable Urgent Care Visits After Ambulatory Cancer Surgery. JAMA Surg. 2021;156(8):740–746. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Assel MJ, Laudone VP, Twersky RS, Vickers AJ, Simon BA. Assessing Rapidity of Recovery After Cancer Surgeries in a Single Overnight Short-Stay Setting. Anesth. Analg 2019;129(4):1007–1013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Front Public Health. 2019;7:64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 1999;89(9):1322–1327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Vickers AJ, Assel M, Hannon M, et al. A comparison of brief versus explicit descriptors for verbal rating scales: interrupted time series design. Health Qual Life Outcomes. 2023;21(1):105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.McCready TM, Stabile C, Vickers A, et al. A Remote Symptom Monitoring Tool As Part of Ambulatory Cancer Surgery Recovery: A Qualitative Analysis of Patient Experience. JCO Oncol Pract. 2023;19(8):595–601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Shabsigh A, Korets R, Vora KC, et al. Defining early morbidity of radical cystectomy for patients with bladder cancer using a standardized reporting methodology. Eur. Urol 2009;55(1):164–174. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.