Patient-reported outcomes in radiotherapy (PRO-RT) utilizing an integrated patient portal; results from a prospective single-arm trial

Abstract

Objectives

This prospective single-arm trial assessed the use of an integrated patient platform (IPP) and effects on patient-reported outcomes survey (PROs) completion.

Materials and Methods

Head and neck, breast, and thoracic cancer patients undergoing radiotherapy (RT) were approached for registration for an IPP for PRO administration and integration of multiple health portals.

Results

Seventy-two patients were approached, 27 (38%) enrolled, and 18 used the IPP for PROs, 2 of which utilized integrative features. IPP user PRO completion rates were 100%, 89%, 89%, and 94% at baseline, end of RT, and 3- and 6-months post-RT, higher than the standard interface. Of the 45 who declined, 24 (53%) reported information security concerns.

Discussion

While the IPP increased PRO completion, many declined enrollment, citing security concerns. Few utilized the portal’s integrative features.

Conclusion

IPPs show promise in aiding PRO completion; continued efforts to demonstrate data safety and utility of IPPs are merited.

Background and significance

Patient portals have transformed how patients and providers share health information, improving patient satisfaction,^1–6 clinical outcomes,^2,4–13 and healthcare costs.¹¹ They are also a primary way of administering patient-reported outcomes surveys (PROs). Completion of PROs has been shown to improve symptom management, treatment adherence, and overall survival in cancer patients.^6–8,14,15 Despite the benefits and online availability of PROs, survey completion rates vary. PRO completion rates in closely-monitored clinical trials range from 80%-95%.^16,17 Without close supervision, completion rates fall with every additional timepoint and can reach as low as 25%.^18–21

Over the last 2 decades of portal development, studies identified challenges in creating interoperable, secure, and user-friendly interfaces. Despite their benefits and usability, only about 25% of patients with cancer consistently use their portals.^22–24 Multiple studies have shown that the most cited reasons for lack of portal engagement include concern about the safety and confidentiality of health information, lack of technical skills, and preference for in-person communication.^10,25–27

Given the improvement of cancer outcomes with PRO completion and the benefits of information accessibility, we aimed to identify an online platform that would promote increased PRO completion and medical data availability. We partnered with a digital health platform company that had previously, in the post-procedural setting, shown improved patient engagement, increased PRO completion, and successfully aggregated patient data across multiple platforms into an integrated patient platform (IPP).²⁸ This IPP could integrate portals from numerous health-related platforms while also serving as the PRO platform. We hypothesized that the availability of a consolidated IPP would interest patients and therefore improve patient completion of PROs.

Materials and methods

Materials and methods

Between March 2022 and March 2023, 72 patients presenting for breast, head and neck, or lung radiotherapy (RT) were prospectively approached for enrollment. Physicians and research personnel explained that the purpose of the trial was to administer the standard of care (SoC) PROs via the IPP as opposed to the established institutional patient portal (EIP). As a secondary benefit, the IPP could be used to consolidate numerous other health portals (other hospitals, commercial pharmacies, and personal wearable technologies) under one platform.

Those who did not enroll were given the opportunity to explain why. All patients, regardless of study enrollment or platform used, received the SoC PRO surveys concordant with their disease site. PROs included a 10-question institutional proprietary version of the Patient-Reported Outcomes of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) (Appendix SI), the Promis10 survey (Appendix SII), and the Mayo Breast Survey (Appendix SIII). The EORTC Head & Neck 35 and Lung Cancer Symptom Scale (LCSS) were also administered to their respective patients.^29,30 A technology use survey (Appendix SIV) was administered to study participants at 6 months to assess their perceptions of the overall survey and communication experience. PROs were administered at baseline, end of RT, and 3- and 6-months post-RT to both IPP and EIP users.

IPP interface

Research personnel aided with setup of the IPP PRO interface on any web-enabled device by providing a URL with instructions at time of consent or subsequently via the EIP. After entering basic demographic information, users could access a dashboard summarizing pending surveys. To encourage IPP use, research personnel followed up via telephone. Patients received $5.00 per completed survey. A social security number (SSN) was required for remuneration.

The IPP, at a minimum, allowed access to PROs. Patients could also elect to consolidate access to their other selected portals by logging into these portals via the IPP and syncing their health data. At initial syncing, patients were asked to grant the IPP access to their external portals via a pop-up screen. The pop-up message stated that the IPP is “not affiliated with your healthcare provider and not obligated by HIPAA privacy guidelines to protect your health information” but noted that the IPP has pledged to not sell, share, or use health data in any way. Once a patient agreed to this, they could elect the data to be synced continuously over a selected time ranging from 1 hour to 3 months. After linking external portals, the IPP offered unified access to clinical notes, test results, and fitness data through one secure login, mirroring nearly all features of the existing EIP except direct care team messaging and bill payment functions.

Statistical analysis

Median (interquartile range [IQR]) values and numbers (proportion in percent) are reported for continuous and categorical variables, respectively.

Statistical analyses included comparing demographic variables by agreement to participation in the study (Table 1, consent yes/no) and IPP use (Table 2, use yes/no). A chi-square test or Fisher exact test was used to compare discrete unordered (categorical) variables, and a Wilcoxon rank-sum test (comparison between 2 groups) was used to compare continuous variables (age and miles from clinic). Survey completion rates at listed time points across patient groups were compared using the Fisher exact test.

Table 1.

Patient characteristics by consent status (n = 72).

		Consent				Total (N = 72)		Univariate P
Variable		Yes (n = 27)		No (n = 45)
		N	%	N	%	N	%
Sex	Female	18	36.7	31	63.3	49	68.1	0.85
	Male	9	39.1	14	60.9	23	31.9
Agea		56	45.1-63.4	58.1	52.2-65.5	57	31.3-70.6	0.49
Race	White	25	36.2	44	63.8	69	95.8	0.55
	Asian	1	100	0	0	1	1.4
	Other	1	50.0	1	50.0	2	2.8
Ethnicity	Hispanic/Latino	2	50.0	2	50.0	4	5.6	0.63
	Non-Hispanic/Latino	25	36.8	43	63.2	68	94.4
Miles from clinica		72.0	35.0-92.0	80	19.0-186.0	77.0	27.5-162.5	0.43
Disease site	Breast	15	38.5	24	61.5	68	54.2	0.82
	Head & neck	12	38.7	19	61.3	31	43.1
	Lung	0	0	2	100	2	2.8
Activated IPP	Yes	18	66.7	N/A	N/A	18	66.7	—
	No	9	33.3	N/A	N/A	9	33.3

^aMedian (IQR).

Table 2.

Patient characteristics by IPP usage (n = 27).

		IPP user				Total (n = 27)		Univariate P
Variable		Yes (n = 18)		No (n = 9)
		N	%	N	%	N	%
Sex	Female	14	77.8	4	22.2	18	66.7	0.08
	Male	4	44.4	5	55.6	9	33.33
Agea		57.6	45.2-61.5	54.7	42.3-63.4	56	45.1-63.4	0.59
Race	White	18	72.0	7	28.0	25	92.6	0.10
	Asian	0	0	1	100	1	3.7
	Other	0	0	1	100	1	3.7
Ethnicity	Hispanic/Latino	0	0	2	100	2	7.4	0.10
	Non-Hispanic/Latino	18	72.0	7	28.0	25	92.6
Miles from clinica		42.5	4.0-83.0	90	75.0-139.0	72	35.0-92.0	0.02
Disease site	Breast	11	73.3	4	26.7	15	55.6	0.45
	Head & neck	7	58.3	5	41.7	12	44.4
	Lung	0	0	0	0	0	0

^aMedian (IQR).

The alpha level was set at P < .05 for statistical significance. SAS version 9.4 was used for analysis.

Results

Of the 72 patients approached, 27 (38%) consented. There were no differences in demographics, disease site distribution, or distance traveled to clinic between those who did and did not consent (Table 1). Of the 27 who consented, 25 were prior active EIP users, 18 (67%) activated their IPP, 18 were female, median age was 56 years (IQR 45-63), and 15 (56%) had breast cancer (Table 2). Of the 18 IPP users, 14 (78%) were female, 11 (61%) had a breast malignancy, and all were White. They had a median age of 58 years (IQR 45-62) and lived a median distance of 43 miles (IQR 4-83) from clinic (Table 2). Those who did not activate their IPP lived significantly further away from clinic on univariate analysis (median 43 miles vs 90 miles, P = .02). All 18 users completed at least one PRO but only 2 used the portal’s integrative features. Both users were White, Non-Hispanic female breast cancer patients in their early 60s. Of the 45 who declined enrollment, 24 (53%) reported concerns with information sharing and data safety, 13% declined surveys or online enrollment, and 18% did not enroll within the designated eligibility window. There were no significant differences between the 18 active users and 54 inactive users.

IPP user PRO completion rates were 100%, 89%, 89%, and 94% at baseline, end of RT, and 3- and 6-months post-RT. Of the 54 patients who used the EIP, survey completion rates were 98%, 77%, 76%, and 62% at the respective timepoints. Completion rates were significantly different only at the 6-month time point (94% vs 62%; P = .003). The technology use survey revealed that 14 of 18 patients felt “very” comfortable with using electronic forms of communication. However, only 12 listed an electronic method as a preferred mode of communication. Face-to-face and portal communication were most preferred. The results of the technology use survey can be found in Appendix SV.

Discussion

This study found that an integrated patient portal with remuneration for survey completion led to high PRO completion rates. However, enrollment and use of portal features were low. While all patients who activated their IPP completed baseline PROs and maintained higher completion rates than those using the EIP, only 38% of approached patients enrolled, and fewer used the integration features. This contrasts with a similar perioperative study that achieved higher participation, possibly due to differences in patient population, compensation, IPP guidance, existence of an established portal, and COVID-19-realted mistrust of healthcare.²⁸ Despite improved data safety and broader portal use, many patients who declined participation cited data security concerns, underscoring the need for stronger data safeguards and continued efforts to build patient trust in the healthcare system.

Most enrolled patients were treated for breast cancer with a low risk of severe RT toxicity and numerous touchpoints at which side effects and management were explained. Studies have shown that patients are more likely to complete PROs when they perceive a positive impact on their medical care or with a perceived need of close monitoring.^18,31 Despite efforts to recruit patients with a range of anticipated toxicities, our cohort’s reduced frequency and severity of side effects may have limited engagement. In contrast, Dhruva et al. enrolled patients at higher risk of complications, increasing the appeal of real-time monitoring and PROs. Future studies targeting a more diverse population with higher treatment-related side effects may improve utilization.

Our study also differed from the Dhruva et al study in compensation. Dhruva et al provided $10 per hour for initial IPP setup and up to 9.5 hours of survey completion time, devices, and a stipend for synchronizing devices regularly. In contrast, our study only compensated patients $5.00 per completed survey, and did not provide devices or synchronization incentives. Also, education on IPP use was limited to the PRO platform without scheduled education in the use of the integrative features, unless prompted by the patient. Additionally, research personnel anecdotally observed patient discomfort with sharing Social Security Numbers (SSNs) for remuneration.

An established institutional portal may have further limited IPP adoption. In the Dhruva et al study, one institution was transitioning to a new patient portal, requiring patients to set up a new account.²⁸ In contrast, our institution had a trusted portal in place, used by 80% of patients during the study time period. In fact, 25 of 27 (93%) of the patients who consented to this study were already active on the EIP, reducing motivation to switch. Thus, this IPP may hold greater value at institutions with limited existing portals.

Originally planned for 2020, our study was delayed until 2022 due to COVID-19. Enrollment in the midst of the public health emergency³² may have been adversely affected by patients’ lack of trust in the healthcare. During the pandemic, the surge in online medical care and information technology (IT) reliance made healthcare platforms vulnerable to cyber-attacks, with data breaches affecting 45 million records in 2021 and 52 million in 2022.^33,34 It is therefore conceivable that patients were apprehensive about using a third-party platform in the setting of a reputable EIP. The pandemic also contributed to mistrust of the healthcare system and workers.^12,35 A 2022 study reported significant declines in patient-perceived trustworthiness of and respect for healthcare workers after the pandemic.¹² Low confidence in data security and mistrust of the healthcare system may have contributed to lower rates of enrollment.

Finally, we acknowledge that more attention to privacy concerns during study design and patient discussions may have been necessary, as data safety issues existed before the pandemic. Research consistently shows that data security is the primary reason for mistrust of online portals.^{4,25–27,36–38} A 2013-2014 study found 58% of patients concerned about data privacy.³⁷ Although study physicians and personnel expressed their confidence in the safety of the IPP, 53% of non-participants cited data security as their main concern. This highlights the need for clear, thorough information on data security in both portal design and studies involving IPPs.

In 2022, one-third fewer patients used the EIP compared to subsequent years, presenting as a possible limitation to engagement in our study. With the overall increase in portal users since, a secure, integrated portal may achieve greater success as online platforms become more trustworthy and commonplace. Finally, although we were able to demonstrate high rates of survey completion, it is unclear whether these effects were a result of the portal or study incentives. Future work should disentangle the impact of platform features from compensation on rates of PRO completion.

Conclusion

This single-arm trial evaluated an integrated patient platform designed to improve the completion rates of patient-reported outcome surveys (PROs) and facilitate data aggregation from external sources. Despite low enrollment, those who engaged with the portal’s PRO interface demonstrated successful utilization, but integrative features were sparsely activated. These findings underscore the potential of user-friendly patient portals in enhancing incentivized PRO collection. However, they also highlight the necessity for further development of optimized electronic tools to ensure meaningful patient engagement and secure, efficient care delivery in the oncology setting.

Author contributions

Yasamin Sharifzadeh (Data curation, Formal analysis, Writing—original draft, Writing—review & editing), William S. Harmsen (Formal analysis, Methodology, Validation), Jessica F. Burlile (Writing—original draft, Writing—review & editing), Kevin F. Pyfferoen (Methodology, Project administration, Resources), Adam C. Amundson (Conceptualization, Methodology, Project administration, Resources), Mark R. Waddle (Conceptualization, Investigation, Supervision), Daniel J. Ma (Conceptualization, Investigation, Methodology, Supervision), Satomi Shiraishi (Data curation, Investigation, Methodology, Project administration, Software), Traci J. Hammer (Conceptualization, Methodology, Project administration, Supervision, Writing—review & editing), Nadia N. Laack (Funding acquisition, Supervision, Writing—review & editing), and Kimberly S. Corbin (Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Supervision, Writing—original draft, Writing—review & editing)

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflicts of interest

None declared.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author, Yasamin Sharifzadeh, MD, by emailing Sharifzadeh.Yasamin@mayo.edu.