Development and pilot implementation of a remote monitoring system for acute toxicity using electronic patient-reported outcomes (ePROs) for patients undergoing radiation therapy for breast cancer

Abstract

Purpose/Objective(s):

We aimed to develop and study the implementation of a remote system for toxicity assessment and management of acute breast radiation side effects using electronic patient-reported outcomes (ePROs).

Materials/Methods:

A response-adapted PRO-CTCAE-based assessment for breast radiation toxicity was administered weekly during and for 8 weeks after radiation from June 2019 to July 2020. The care team received alerts when “severe” symptoms were reported by patients who were then contacted. Treatment, clinic, and socio-demographic characteristics were abstracted from patient records. A subsample of patients and care team members were qualitatively interviewed at follow-up.

Results:

Overall, 5,787 assessments were sent to 678 patients, of which 489 patients (72%) completed 2,607 assessments (45%). “Moderate” or greater toxicity was reported by 419 (86%; 95% C.I. 82% to 89%) responders. Clinician alerts for “severe” toxicity were generated for 264 assessments among 139 unique patients, of which 83% occurred post-treatment. The proportion of surveys that prompted an alert was significantly higher after treatment (n=219, 13%) than during treatment (n=45, 5%) (p<0.001). Survey completion rates in the post-treatment period were higher among patients undergoing partial breast irradiation than post-mastectomy radiation (IRR 0.70, 95% CI 0.60–0.81) (p<0.001) despite experiencing less severe toxicity. Interviews (n=15) found that patients had a positive experience with ePROs, although many thought the primary purpose was for research rather than symptom management.

Conclusion:

With the majority of toxicity occurring after breast radiation has ended, remote symptom monitoring with ePROs appears to fill a gap in clinical practice, particularly for patients undergoing shorter courses of radiation. It is important to properly onboard and educate patients that the purpose of ePROs are to aid clinical care. Further research is needed to determine whether the costs associated with ePROs can be offset by reducing routine clinic visits, and if this approach is acceptable and appropriate.

Introduction

Clinicians often underreport adverse symptoms and side effects (1), which can result in inadequate symptom management and decreased patient quality of life (2). Patient-reported outcome (PRO) measures have emerged to directly evaluate patient experience of symptoms and toxicity, and have become a standard component of prospective cancer clinical trials. In routine practice, PROs can facilitate more timely symptom assessment and management, thereby reducing symptom burden (3–5) and improving overall survival compared to routine follow-up care among patients undergoing systemic therapy for metastatic disease (6,7).

Electronic platforms have led to wider integration of PROs into routine care given the ability to more easily collect, access, and analyze data (8). Electronic PROs (ePROs) enable remote symptom monitoring, which can enhance patient-physician communication (9), and facilitate early identification of and intervention for acute symptoms (9,10). While the most data is available for PROs in the systemic setting, a randomized trial has also shown that ePROs can reduce symptom burden after surgery (5). As shorter courses of radiation therapy continue to be adopted, as in the treatment of breast cancer (11,12), patients increasingly experience acute side effects after treatment ends, when they are not being seen in clinic (13). Due to traditional models of acute radiation side effect management, data detailing symptom burden in the immediate post-treatment period in routine care is also somewhat limited. ePROs potentially provide an innovative, evidence-based and patient-centered solution.

The purpose of this study was to develop and implement an ePRO-based instrument assessing toxicity through an online patient portal for patients undergoing breast radiation. The ultimate goal of implementation was to improve patient-clinician communication, enhance symptom management, and reduce unnecessary clinic visits. This study aims to identify factors influencing ePRO completion in the setting of breast radiation, understand patient and clinician preferences for a symptom response system, and explore acute radiation symptom trajectories for patients undergoing all types of external beam radiation for primary breast radiation.

Methods

ePRO development & implementation

An ePRO-based instrument was developed using the Patient-Reported Outcomes Common Terminology Criteria for Adverse Events (PRO-CTCAE) (14,15), including 9 items previously identified and modified for acute toxicities of breast radiation (16). This resulted in a 7–17-question assessment that was response-adapted (i.e. branching logic for additional questions when a symptom was reported) (Supplemental Text 1). The instrument addressed acute toxicities of breast radiation including skin changes, pain in the radiated area, swelling, tenderness, fatigue, and pain with swallowing. The questions prompted patients to rate the frequency (“never” to “almost constantly”), severity (“none”, “mild”, “moderate”, “severe”, “very severe”), and distress (“not at all” to “very much”) associated with their symptoms each on a 5-point Likert-type scale. The instrument also included questions about anxiety using the Generalized Anxiety Disorder 2-item (GAD-2) screening tool (17).

All patients scheduled to undergo radiation for a diagnosis of breast cancer were assigned ePROs weekly during treatment and for 8 weeks following the completion of treatment. Patients also received standard of care toxicity management which included weekly on-treatment clinic visits and a follow-up visit one to three months post-treatment. Optional post-treatment nursing visits were also available if patients experienced significant toxicity. The survey could be completed remotely on a personal device (computer, tablet, or smart phone), or on tablets available in the waiting room for on-treatment assessments.

Clinician access to ePROs and system for clinical alerts

ePRO responses are available through the patient’s portal (called MyMSK), which can be accessed by clinicians through the electronic medical record. Importantly, alerts are sent via secure messaging to the care team when patients report the following subset of symptoms as “severe” or “very severe”: pain in the radiated area, skin breakdown, breast enlargement, breast tenderness, and pain with swallowing. In such instances, patients received a message through their patient portal to let them know that they had reported a symptom that the care team would help manage and to except someone to reach out to them. For “mild” to “moderate” symptoms, patients receive a notification that these symptoms are normal and to continue routine recommendations including skin care.

Patient cohort and patient-level data

Patients undergoing radiation to the breast or chest wall under the care of three physicians covering five clinical locations from June 3, 2019 to July 20, 2020 were included in this analysis. Patients were included if they were enrolled in the electronic patient portal, which at the time of this study was only available in English. Patients were excluded from this analysis if their medical records indicated that they were undergoing radiation to a body site that was not the breast. Demographic data and medical information, including radiation type (partial breast irradiation, whole breast irradiation, regional nodal irradiation, post-mastectomy radiation), were collected from the electronic medical records of all patients assigned surveys. Patients’ home addresses were used to calculate distance to clinic in miles using Google Maps (Google LLC, Mountain View California) and recording the shortest driving distance, regardless of estimated travel time. We determined socioeconomic status using the University of Wisconsin’s Neighborhood Atlas (18) which provides rankings of neighborhoods (defined by Census blocks) by socioeconomic disadvantage according to the Area Deprivation Index (ADI), and recorded the national percentile based on the patient’s home address. The Institutional Review Board approved this study.

Qualitative data collection – Clinicians

An advisory committee comprised of physicians, nurses, informatics experts, and web developers met biweekly during the first two months of the pilot. Following the pilot period, a member of the research team conducted 30-minute semi-structured interviews with three nurses. Interviews focused on nurses’ perceptions and use of the survey tool and were audio-recorded and transcribed for thematic analysis.

Qualitative data collection – Patients

We also conducted semi-structured, qualitative interviews with patients who completed at least one ePRO survey. Patients were approached at their first follow-up visit (4–12 weeks post-radiation) to participate. Interviews addressed four domains of the patients’ experience and preferences with the survey tool, including survey question comprehension, survey timing, barriers to survey completion, and care team response and feedback. Interviews were conducted until thematic saturation was reached, with no new information emerging from at least two subsequent interviews (19). Audio-recordings of each session were used to generate summary reports. Summary reports from patient interviews, advisory meeting notes, and transcriptions of nurse interviews were coded for themes and assessed using a matrix analysis approach. Supplemental Text 2 describes this process in detail.

Statistical analysis

We used descriptive statistics to summarize patient characteristics, providing baseline characteristics between patients who responded to at least one survey and patients who did not. We created a multivariable negative binomial regression model to determine whether there was an association between patient characteristics and the total number of surveys completed. Characteristics included in the model were age at the start of treatment, educational status, employment status, race/ethnicity, socio-economic status, receipt of chemotherapy prior to radiation, type of radiation treatment, treatment location (main hospital or regional location), and the distance between the patient’s home and the treatment facility. As patients who underwent a longer course of treatment were assigned more surveys, we incorporated the number of assigned surveys into the model as the offset parameter.

To assess the trajectories of patient-reported clinically meaningful symptoms over time, we created longitudinal models using generalized estimating equations with an autoregressive correlation structure. Survey non-response was assumed to be ‘missing at random’, and therefore whether patients responded was considered independent of how patients would respond. However, the generalized estimating equations method does implicitly include imputation. Symptoms of interest included moderate to very severe skin toxicity, pain, breast enlargement, breast tenderness, fatigue, and dysphagia. Anxiety was evaluated using the GAD-2 items, patient-reported anxiety was considered clinically meaningful when the GAD-2 item assessment summed to a score of three or more (binary), consistent with anxiety screening guidelines (17). For this analysis, patients who underwent radiation to breast and regional nodes were grouped with patients who underwent post-mastectomy radiation treatment due to small sample size. All statistical analyses were conducted using R 4.0.1 (The R Foundation, Vienna, Austria) with a p value <0.05 denoting statistical significance.

Results

Survey response and patient characteristics

A total of 5,787 weekly ePRO assessments of toxicity from breast or chest wall radiation were sent to 678 unique patients, accounting for 29% of the 2,375 total patients treated for this indication at the institution during the study period. Overall, 489 (72%; 95% CI 69%−75%) patients completed at least one assessment, and a total of 2,607 (45%; 95% CI 42%−48%) completed assessments were analyzed. Patients who responded to at least one ePRO completed a median of 5 assessments (IQR 2–9). A greater proportion of ePRO assessments assigned during treatment were completed (51%; 95% CI 48%−55%; n=833) compared to post-treatment (42%; 95% CI 39%−46%; n=1,724) (p<0.001). In the post-treatment period, response dropped below 40% after week 6 (see Supplemental Table 2). Patient characteristics, stratified by response to at least one assessment versus no response, are summarized in Table 1. The most common dose and fractionation regimens were as follows: 40 Gy in 10 fractions (2 weeks) for partial breast radiation (n=94), 42.4 Gy in 16 fractions with or without a 10 Gy in 4 fraction boost (3–4 weeks) for whole breast radiation (n=315), 50 Gy in 25 fractions (5 weeks) for post-mastectomy radiation (n=108), and 50 Gy in 25 fractions with a simultaneous or sequential tumor bed boost (5–6 weeks) for whole breast with regional nodal radiation (n=42), and (see Supplemental Table 1 for details).

Table 1.

Characteristics of patient cohort

	Responders (n=489)	Non-responders (n=189)
Characteristic
Age (years), median, IQR	55 (46, 64)	61 (50, 68)
Race, n (%)
White non-Hispanic	319 (70%)	113 (66%)
Non-white minority	134 (30%)	59 (34%)
N/A	36	17
Education
Graduate or professional school	109 (39%)	27 (29%)
College/Vocational school	107 (38%)	28 (30%)
High school diploma	61 (22%)	28 (30%)
Less than a high school diploma	3 (1.1%)	9 (9.8%)
N/A	209	97
Work status
Employed full-time	224 (63%)	53 (45%)
Not employed full-time	129 (37%)	66 (55%)
N/A	136	70
Treatment group
Whole-breast irradiation (WBI)	284 (58%)	96 (51%)
Post-mastectomy	91 (19%)	42 (22%)
WBI with regional nodal irradiation (RNI)	39 (8.0%)	17 (9.0%)
Partial-breast irradiation	75 (15%)	34 (18%)
Chemotherapy
Yes	241 (49%)	85 (45%)
No	248 (51%)	104 (55%)
Clinic Setting
Main	124 (25%)	64 (34%)
Regional	365 (75%)	125 (66%)
Distance to clinic (miles), median, IQR	13 (8, 22)	12 (7, 19)
N/A	7	3
SES (ADI), median, IQR	11 (5, 20)	11 (7, 21)
N/A	11	4

Characteristics associated with frequency of response

On multivariable negative binomial regression assessing all patients, we found evidence of a significant association between older age and fewer assessments completed (IRR per 10-years of age 0.93, 95% CI 0.90–0.95; p<0.001) (Table 2). Additionally, factors associated with completion of more assessments included higher level of education (i.e. completion of a graduate/professional degree), full-time employment, self-identification of non-Hispanic white race/ethnicity, and receipt of chemotherapy prior to radiation (p<0.01) (Table 2).

Table 2.

Multivariable negative binomial regression assessing the association between patient characteristics and number of surveys completed.

	IRR	95% CI	p-value	Predicted Number of Surveys
Characteristic
Age (per 10 years)	0.93	0.90, 0.95	<0.001	45-year-old: 5.9 65-year-old: 5.1
Race			<0.001
Non-white minority	—	—		4.7
White, non-Hispanic	1.17	1.09, 1.25		5.5
Education			<0.001
High school or less	—	—		4.2
College/Vocational school	1.49	1.34, 1.65		6.2
Graduate/Professional School	1.31	1.19, 1.45		5.5
Unknown	1.05	0.94, 1.17		4.4
Work status			0.001
Employed full-time	—	—		5.5
Not employed full-time	0.94	0.88, 1.01		5.2
Unknown	0.84	0.76, 0.92		4.6
Treatment group			<0.001
Partial-breast irradiation	—	—		5.3
Whole-breast irradiation (WBI)	1.03	0.95, 1.13		5.5
WBI with regional nodal irradiation (RNI)	0.79	0.70, 0.91		4.2
Post-mastectomy	0.95	0.85, 1.07		5.1
Chemotherapy			0.002
No	—	—		5.5
Yes	1.11	1.04, 1.18		6.1
Clinic Setting			0.6
Regional	—	—		5.5
Main	1.02	0.95, 1.09		5.6
Distance to clinic (per 50 miles)	1.01	1.01, 1.02	<0.001	5 miles: 5.5 100 miles: 5.6
SES (ADI, per 10 units)	1.02	1.00, 1.03	0.007	5 units: 5.4 20 units: 5.6

The IRR is the incidence rate ratio and corresponds to the ratio of the number of surveys completed. For illustrative purposes, we show the predicted num ber of surveys for different groups holding other predictors at the mean.

Symptom trajectories

Results of the generalized estimating equations model with the predicted probability of moderate to very severe skin toxicity, pain, breast enlargement, breast tenderness, dysphagia, and fatigue, and clinically meaningful generalized anxiety over time are depicted in Figure 1. Rate of moderate or greater skin toxicity, pain, breast enlargement, breast tenderness, and fatigue peaked approximately 1 week after whole breast and PMRT/RNI and 2 weeks after partial breast radiation, then decreased.

Figure 1.

Predicted probability of any moderate or higher toxicity* over time during and after treatment based on treatment type: post-mastectomy radiation treatment or regional nodal irradiation (PMRT/RNI; red), whole breast irradiation (WBI; blue), and partial breast irradiation (PBI; green). *Anxiety is defined as a Generalized Anxiety Disorder 2-item score of 3 or more.

Given the observation that peak toxicity occurred shortly after the end of treatment, we were further interested in assessing the association between radiation treatment type (as a surrogate for toxicity) and the frequency of assessment completion in the post-treatment period. We conducted a post-hoc exploratory analysis to evaluate this association, adjusting for all other variables including age, level of education, employment status, race, socio-economic status, chemotherapy prior to radiation, treatment setting, and distance between home and clinic. In the post-treatment period, receipt of partial breast irradiation was associated with more frequent response than receipt of whole breast irradiation (IRR 0.84, 95% CI 0.75–0.93), post-mastectomy radiation (IRR 0.70, 95% CI 0.60–0.81), or regional nodal irradiation (IRR 0.55, 95% CI 0.46–0.65) (p<0.001).

Alerts to the care team

Overall, 264 (10%; 95% CI 8.4–12%) of completed assessments prompted an alert to the care team. Most (83%) alerts occurred in the post-treatment period. Among all completed surveys, the proportion of surveys prompting an alert was significantly higher after treatment (n=219, 13%) than during treatment (n=45, 5%) (p<0.001). Most (60%) of alerts were triggered for skin breakdown related to radiation dermatitis.

Qualitative patient results

A total of fifteen patients participated in qualitative interviews. Overall, participants found the ePRO to be positive, comprehensive, and reflective of their experience. Interview themes are summarized in Table 3.

Table 3.

Major themes from patient interviews

Theme	Description	Patient Quote
Overall Acceptability	Participants found the Breast PRO to be comprehensive and relevant	“These are all the right questions because they cover the symptoms from radiation that people get.”
		“I knew what to expect, in case I had a symptom that I didn ’t know and needed something”
		“They were helpful with explaining how I was feeling”
		“She [the patient] liked them because she knows that someone was looking at them and someone cared about her”
Question comprehension and Feedback	Items pertaining to skin changes were redundant	“Skin color, sensation, itchy, dry, skin breakdown. They are all kind of similar”
	“Pain with swallowing” not relevant to some patients	“Struck me as a question for someone with a different type of treatment.”
	Difference between “tenderness” and “pain”	“To her it means if she touches her skin and there it is tender that is tenderness, while pain is already there even if she doesn ’t touch it”
	Difference between tenderness and “sensation”	“Sensation is more topical and tenderness is internal.”
	Preference to separate “tenderness” from “enlargement”	“A lot of people don’t know that enlargement is an issue. They see it as swelling and that it will go down but in reality it could get worse.”
		“A space for a text box to just include any comments you have for your doctor”
	Additional items: comment box and skin color changes	“I would have liked to see something that explores how the patient feels about the severity of the skin discoloration.”
Barriers to Survey Completion	Clinical relevance unclea.	“I didn’t see the worth... Towards the end of treatment I told somebody, ‘don’t ask me, I don’t want to participate in any more studies”
	Weekly responses a burden	“Not much changes on a week to week basis” “Once you are done with treatment you are very done.”
	Issues with Email alerts	“It is harder for older women who don’t do all of the electronic stuff”
Preferences for Care Team Response	Surprise at worsening symptoms	“The doctor had warned me but I was surprised by how much worst they got.”
	Reassurance from Care Team	“[They can reach out with] Either a solution or assurance I guess. I understand that somethings don ’t have a solution at that time but if you could assure me that the symptom is normal, that would make me feel better”
	Severity requiring outreach from nurse	“She [the nurse] put me at ease. She said what I experienced was accurate” “I have to be in pain that is consistent and is not going away or having a fever. It would have to be severe.”
Information Preferences	Interest in graph showing reported vs. expected symptoms	“Knowing what’s expected gives more understanding.”
	Concern about impact of graph on anxiety levels	“I don’t want to know where I fall because if Ifall bad I am never going to forget that.”

Patients did offer feedback on specific items in the questionnaire. Patients found the numerous questions on skin changes (i.e. itchy skin, dry skin, skin breakdown) to be redundant. A few patients suggested adding a comment or free text box to allow them to elaborate on their symptoms, particularly those pertaining to skin color changes. Patients appreciated the item addressing anxiety but noted that the survey instructions should indicate that not all symptoms may be experienced, as this list of symptoms themselves might be anxiety-inducing.

Lack of clarity about the survey’s purpose was a major barrier to completion. Some patients thought that the survey was for research purposes and indicated that they would have been more likely to complete them had they known that it would support symptom management. Other patients found the weekly survey “burdensome” and did not feel incentivized to complete the survey post-treatment, especially if they did not experience a change in symptoms.

In interviews, 7/15 patients noted that they were “surprised” that their symptoms worsened after the last day of radiation. Most patients (9/15) indicated that they would want a nurse to reach out within 24 hours if they rated a symptom as “severe.” Otherwise, participants felt that a weekly status check would be enough while on-treatment.

Most patients (10/15) were interested in seeing a graph showing their reported symptoms over time compared to expected symptoms. Many felt that this graphic would be “reassuring”, since it can sometimes be difficult to determine whether or not a symptom is “normal.” However, other patients pointed out that if they were experiencing worse than expected symptoms, then the graph could be anxiety-inducing.

When asked how they prefer to follow-up with their care team during the month between end of treatment and first follow-up, patients varied in their preferences which included scheduling an appointment in clinic with a nurse one week after their last treatment (5/15), a call to the clinic if they encountered issues (5/15), and remote surveys with follow-up from a nurse when prompted by survey results (5/15).

Qualitative clinician results

During the pilot of the ePRO tool, we observed five advisory committee meetings and interviewed three nurses. Interviews and meetings elicited key themes related to survey content and use in practice, summarized in Table 4 and further described in Supplemental Text 2. In particular, clinicians felt the ePRO tool was beneficial and that it enhanced patient visits during treatment, provided insight into patients’ status, especially post-treatment, and served as a viable option for remote symptom management. However, clinicians also expressed concern over the patient burden of completing frequent surveys and regularly being alerted to symptoms previously addressed or already resolved. Figure 2 presents a proposed workflow incorporating feedback from both patients and clinicians.

Table 4.

Major themes from clinician interviews

Theme	Description	Patient Quote
Satisfaction with survey content	Additional insight into patients’ status	“The forms were more accurate than what the patient was telling me [...] I felt sometimes that if I didn ’t go in with the piece of paper [questionnaire] I would leave there feeling, ‘Okay they ’re fine ’ and then I would go back and look at the form and I’m like, ‘Wait a minute, they ’re not fine. ’”
Utility of survey in practice/during treatment	Reviewed surveys with patients	“I would print it out and I would bring it in with me. Then I would review it with them and then you know we would talk about what Ke a vswers were and how they felt about it. It was part of my work flow. ”
	Need to increase survey frequency	“The once a week thing has got to stop. The symptoms could have already changed, and we could have already addressed it because now you ’re talking about the prior week. ”
	Concern about survey burden for patients	“I don’t think patients would be that consistent if they had to do it more than once. ”
PRO tool may have more value post-treatment	Patients are no longer seen weekly post-treatment	“It’s more useful than the ones who are on treatment because we are seeing them every week anyways. And some nurses thought the same thing, we all thought its more useful for the ones who have finished treatment. ”
	Need for survey recall period to be clear and well understood	“They would fill out the form and then I would get an alert on my email to call the patient because the questionnaire had all these reds. Then I’d call them and they ’d be like ‘No, that symptom went away ’ or ‘It’s much better.’ It never was like ‘Oh okay, it’s still happening. ’”
Care team alert frequency and burden	Alerts for symptoms already addressed	“It would make me feel like you know, okay I called this patient, they ’re telling me they ’re fine, why is this form being singled out, like [why do I] need to call this patient when their symptoms are resolving or had resolved? ”
	Provide option to be contacted	“For the ones who completed treatment, sometimes for them it would be too much to come in or they would rather just follow-up by phone.”
Information Preferences	Interest in visualizing symptom change over time	“Seeing the trajectory of how things will change [,..]will help us eventually look at if we should put all the patients on Triamcinolone or Mometasone. It could help us to standardi when we start to use them.”

Figure 2.

Schematic outline of a proposed workflow for electronic patient-reported outcome assessment for acute toxicities of breast radiation

Discussion

To our knowledge, this is the first study to establish feasibility of a site-specific, weekly ePRO survey for patients receiving radiation in routine practice, particularly in the remote period following radiation. This study highlights increased toxicity burden in the immediate post-treatment period, when patients are not being routinely evaluated and managed for radiation toxicity. This is particularly relevant in patients undergoing shorter courses of radiation. Overall, patients felt the tool comprehensively captured the symptoms that they may want to communicate to their care team and qualitative analysis identified acceptable initial implementation of the ePRO (20), with identified areas for refinement. Finally, clinicians approved expanding the system to all breast cancer patients treated across the entire Department of Radiation Oncology in September 2020.

The finding that clinically meaningful toxicity peaks post-treatment is consistent with the limited available trial data (13,21,22). Although patients are often told to expect this pattern of symptom progression after treatment, qualitative interviews revealed that many patients were still surprised when their symptoms progressed following the completion of treatment. Prior data suggest barriers exist to reaching the care team under the current system of routine scheduled visits (23), and qualitative interviews support ePROs as a potential solution. Importantly, patients agreed that “severe” was the appropriate symptom alert threshold to engage clinicians about breast radiation toxicities. Subsequent efforts to enhance both patient and clinician utility of the ePRO tool have included adding hyperlinks to patient education materials within survey responses and instructions prompting patients to upload photos of dermatitis for clinician review. By providing patients with information and reassurance at the right time, the use of ePROs has the potential to reduce patient stress and anxiety in the post-treatment setting (24). Additionally, in order to reduce redundant efforts and make sure that an alert is only sent to the care team when clinically relevant, they system was updated to ask patients who report alert-prompting symptoms during treatment if they would like a nurse to call them about their reported symptoms.

Although the role of PROs has been less defined in radiation oncology (25,26), we hypothesized that data from the post-operative setting suggesting reduced symptom burden with more rigorous management could be applied to the post-radiation setting, especially among patients with breast cancer (5,27). Establishing feasibility of remote management post-radiation, given high response rates and positive qualitative data, suggests an appetite among patients and acceptance among clinicians, and lays foundation for future research to improve patient-centered care. The finding that patients undergoing partial breast radiation, which is both the shortest course and with the least toxicity, were more likely to complete ePROs supports efforts to better understand benefits beyond unplanned admission, such as improved patient-clinician communication and reduced patient anxiety, as is under prospective evaluation in the post-surgical setting (28). With shorter radiation regimens becoming a standard option in the treatment of rectal, prostate, and bladder cancer (29–31), there is increasing applicability of these findings. For example, patients who received partial breast radiation, the shortest treatment in the current study, were more likely to complete the ePRO. We hypothesize that this is the result of experiencing the majority of their toxicity following the end of treatment, which increased their motivation to engage in remote symptom management. There is nonetheless a cost related to ePRO implementation including resources for building and maintaining platform infrastructure. Further research is needed to evaluate the acceptability and appropriateness of subsequently reducing low-value scheduled follow-up visits, which could also reduce patient financial toxicity (32).

Scheduling and travel costs associated with in-person care can present a barrier to follow-up among patients with breast cancer, particularly among vulnerable populations (32). Our prior research suggests that the transition to telemedicine during COVID-19 may reduce these barriers (33). Remote ePROs can also facilitate communication for patients that have difficulty visiting the clinic. However we observed decreased use of the ePRO system among patients who were older, less educated, non-White, not employed full-time, and lived closer to the treatment facility. While these findings were statistically significant, the predicted number of surveys completed by different groups did not vary greatly and the results do not necessarily translate to clinical relevance. Still, it is important to assess new digital tools for the potential to paradoxically increase disparities in access to care, particularly among groups without access to a computer or smartphone. Level of literacy, cultural background, and functional abilities and limitations have all been shown to influence how patients interpret and respond to PRO measures (34). An analysis of NRG oncology clinical trials found patients that were younger and more educated consented at higher rates to completion of PROs, consistent with our findings (35). The use of text messages or even an automated telephone-based method of PRO collection could help make ePROs more accessible for older patients or for those without internet access (36). Alternative methods of communication, including a phone call from the nurse 1–2 weeks after radiation, may also be helpful to correct such disparities in access.

Although we expected response rates to drop after patients completed treatment, the post-treatment period is when remote symptom management is most valuable. A similar pilot study conducted at the institution had a response rate of 56%, which declined over time as patients progressed through treatment (37). Our post-treatment response rates were somewhat low, from 48% immediately to 32% by week 8. Findings from qualitative interviews suggest a major barrier to PRO completion is simply misunderstanding the purpose of ePROs to be research rather than routine symptom management. Thus, a more conscious, structured, and accessible approach to educating patients on the primary clinical purpose of the assessments (i.e. at the radiation simulation or first on-treatment visit) could enhance response and improve ePRO compliance. Such efforts ought to include assistance with onboarding to digital tools to further reduce barriers to use and an emphasis on the tool’s clinical importance, especially at the end of radiation. To further improve overall response rates and reduce survey fatigue, we also truncated weekly questionnaires from 8 to 6 weeks given the timing of peak toxicity and declining response rates at the end of treatment.

Finally, it is important to note that many validated PRO questionnaires are available in multiple languages, and modifications result in invalidation of these tools for non-English speakers. It is therefore critical to weigh the risks and benefits of modifying questions. While we opted to use skin questions previously modified for breast radiation, further research should compare these questions to the validated “skin burns from radiation” question from PRO-CTCAE, which is available in multiple languages. Furthermore, patient qualitative interviews indicated redundancy of skin-related symptom questions in the modified version. Also, since the pilot ended, we replaced GAD-2 with the PRO-CTCAE for anxiety to more easily facilitate clinical alerts using the severity scale. PRO-CTCAE questionnaires in their native form can be downloaded in multiple languages for free from the NCI’s website (38).

There are several limitations to our investigation including the single institution nature and highly selective patient population. Currently, the institutional patient portal and PROs are only available in English, resulting in selection bias and limiting generalizability. While semi-structured patient interviews allow for flexibility in the breadth of ideas collected, it may be difficult to direct a patient or clinicians comments towards topics of interest without biasing the conversation. To minimize this effect, all personnel who conducted interviews completed training with the qualitative research team prior to interviews. Interviews were only conducted with patients who had completed at least one survey, so assessments of patient acceptability are biased. However, the high ePRO response rate (>70%), median of five assessments completed per responding patient, and qualitative patient and clinician feedback, demonstrate feasible clinical implementation of the ePRO. Finally, we are unable to comment on “appropriateness” of the ePRO and might recommend including a short 4-question validated questionnaire from the implementation science literature (39).

Conclusion

For patients undergoing breast radiation, a weekly ePRO assessment distributed during and immediately after radiation was successfully implemented based on patient survey response rates and qualitative analysis. As radiation regimens become shorter, the burden of treatment-related side effects shifts to the post-treatment period and remote ePROs provide a potential solution to symptom management and follow-up. This finding may apply not only to breast cancer, given the general trend toward hypofractionation. Further research is needed to evaluate improvements in patient-centered outcomes with use of remote ePRO-based symptom monitoring, and to optimize efficiency of the clinical workflow. Patient non-response is associated with socio-demographic factors, which could potentially exacerbate existing disparities in care. Selection of questions available in multiple languages should be prioritized as well as efforts to onboard patients and educate them on the clinical utility of their ePROs. Further work is in progress to assess whether the ePRO system could be used to reduce unnecessary clinic visits following radiation, and the influence on clinician efficiency and financial toxicity to patients. It will be particularly pertinent to evaluate this in the setting of more routine use of telemedicine-based follow-up care, which reduces travel costs for patients (33) but not clinician time (40).

Data Availability Statement:

Research data are stored in a secure, institutional database and will be shared upon request to the corresponding author.