Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Dec 12.
Published in final edited form as: JCO Clin Cancer Inform. 2025 Dec 10;9:e2500178. doi: 10.1200/CCI-25-00178

Patient Portal Engagement in Oncology: Results from the NU IMPACT Study in a Large Healthcare System

Kyle M Nolla 1,2,*, Maja Kuharic 1, Nicola Lancki 1, Callie L Walsh-Bailey 1, Ann Marie Flores 1, Sofia F Garcia 1, Roxanne E Jensen 3, Yingbao Wang 1, Quan Mai 1, Ambrosine M Mercer 1, Justin Dean Smith 4, Alexandra M Psihogios 1, Kimberly A Webster 1, Sheetal M Kircher 1, Patricia D Franklin 1, David Cella 1, Betina R Yanez 1
PMCID: PMC12698109  NIHMSID: NIHMS2118871  PMID: 41370780

Abstract

Purpose:

Electronic patient portals can promote patient-centered care, but determinants of engagement remain underexplored in oncology. This paper examines sociodemographic and clinical factors associated with engagement with four portal features, including invitations to complete patient reported outcome measures (PROs) before appointments.

Methods:

Secondary analysis of the NU IMPACT study, a stepped-wedge cluster randomized trial to promote symptom management using PROs in adult oncology. For each enrolled participant, we examined portal usage across one year.

Results:

3,457 patients enrolled between April 2020 and April 2023 from 30 Northwestern Medicine ambulatory oncology clinics. Patients were 65% female, 85% white, 85% non-Hispanic/Latino, with mean age 60.8. Cancer diagnoses was 30% breast, 12% lymphoma, and all others less than 10% of the sample. Patients accessed laboratory results most frequently (Median 23 days in the year), followed by messaging (Median 11 days), and physician notes (Median 2 days). 62.6% of patients completed at least one invited PRO. Controlling for sociodemographic factors, patient characteristics that were associated with greater engagement across three or more features included more oncology appointments, high health literacy, high anxiety, one or more severe physical symptoms, and high shared decision making with their healthcare team. Black race, Hispanic/Latino ethnicity, and Medicaid insurance were associated with lower portal engagement. Patients who used any other portal features were more likely to complete PROs. In contrast to other portal features, patients with at least one severe physical symptom were less likely to complete PROs (IRR 0.87, 95% CI 0.81-0.93, P<.001).

Conclusions:

Portal use among cancer patients varies by sociodemographic and clinical characteristics. Findings suggest a need for targeted interventions to promote equitable use among underrepresented groups and promote portal-based PRO completion for patients with higher symptom burden.

Keywords: oncology, patient engagement, electronic health record, patient portal, patient-reported outcome measures, social determinants

Introduction

Patient portals within electronic health records (EHRs) provide features such as secure messaging with providers, on-demand access to clinical information, and remote completion of patient-reported outcome measures (PROs). Portals have become vital for engaging patients, especially patients managing chronic conditions including cancer1-3. For example, routine collection of electronic PROs is associated with improved symptom control and health-related quality of life among cancer patients4-6. However, despite the potential benefits, patient engagement with portals remains suboptimal7. As portals become integrated with healthcare, it is crucial to identify barriers to patient engagement.

Prior research has linked portal engagement with individual and health system factors. At the patient level, greater portal engagement is associated with younger age in adults8-15, female sex8,9,11, white race and non-Hispanic ethnicity8-10,12,14-17, private health insurance8,11,17, and higher health literacy8,10,18. Additionally, patients managing chronic conditions such as cancer are more likely to use portals due to need for information about their healthcare14,17,18. At the health system level, higher ratings of patient-centered communication with clinicians is associated with greater portal engagement in oncology19. Furthermore, clinics differ in how they implement portals based on perceived value for their patient population, workflow integration, and organizational needs20-22.

While much work focuses on overall portal engagement, fewer studies examine specific portal features. Many features –including messaging providers, viewing lab results, and viewing physician notes– are useful for patients who seek health information or assistance. On the other hand, some features such as PROs instead request action from the patient. Thus, patient engagement with these features may differ. In a recent systematic review, electronic PRO completion rates were associated with lower symptom burden23,24 and no specific demographic factors25. Comparing use of features, including PRO completion, may give insight into the utility of those features.

This study examined how sociodemographic and clinical factors re associated with engagement with four patient portal features (messaging, lab result viewing, physician note viewing, and PRO completion) in a large ambulatory sample of cancer patients. Based on prior research on overall portal engagement, we hypothesized that higher health literacy, higher ratings of shared decision making, and sociodemographic characteristics (younger age, white race, non-Hispanic ethnicity, use of private health insurance) would be associated with greater use of all four portal features. We also expected that patients with severe physical and/or mental health symptoms – that is, patients with the greatest need for information and assistance – will more frequently use portal features such as messaging, notes views, and lab views. Finally, we compared portal-based PRO engagement to engagement with other features.

Methods

We conducted secondary analyses of an embedded randomized controlled trial within the NU IMPACT (Northwestern University IMproving the Management of symPtoms during and following Cancer Treatment) trial. NU IMPACT is one trial within the IMPACT Consortium which evaluates the effectiveness of regular symptom monitoring via PROs delivered through the portal26.

NU IMPACT is a type 2 hybrid effectiveness-implementation trial27,28. NU IMPACT used a stepped-wedge design to test implementation strategies for an electronic PRO and cancer symptom management intervention across 6 randomized (1 non-randomized) clusters of 30 clinics, including clinics from small community hospital to large academic medical centers. Additionally, patients were randomized into usual care or enhanced care, although the intervention was shown to have no effect29. The analyses in this paper included participants who consented to the patient-level study from both pre-implementation and post-implementation phases, regardless of their randomization status.

Eligible participants were English- or Spanish-speaking adults (≥18 years old) diagnosed with a solid tumor or hematologic malignancy within the past 10 years who received treatment or survivorship care at the 30 participating clinics between April 2020 and April 2023. Participants were included in this study if they completed baseline surveys, had an active patient portal account (MyChart, later called MyNM, Epic Systems EHR (Verona, WI)), and had a cancer diagnosis according to the EHR. The study period included one year of portal engagement data starting from date of consent. As in prior studies of EHR use9, restricting analyses to a one-year period minimizes missing data due to attrition while capturing lower-frequency patients such as survivors in post-treatment care. Hypotheses for this study were generated after data collection but before analyses. The trial was approved by the Institutional Review Board and all participants provided informed consent.

Data Collection

Sociodemographic characteristics were collected from a baseline survey and confirmed with the EHR. Also extracted from the EHR were clinical data such as cancer diagnosis as entered by the provider into Epic; number of oncology visits over the one-year study period; clinic location; and portal utilization data. Portal utilization data were quantified by the number of days using each feature. Participants were invited to complete PROs before each oncology appointment, at maximum once per month.

Measures

Physical symptom severity was assessed at study baseline using six items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)30. Patients rated frequency and/or severity of nausea, vomiting, shortness of breath, constipation, diarrhea, and insomnia over the last seven days on five-point scales. Symptoms with the highest rating (e.g., “Almost Constantly” for frequency) were counted. We dichotomized patients as either having no severe symptoms or at least one severe symptom.

Mental health was measured at study baseline using Patient-Reported Outcomes Measurement Information System (PROMIS) Depression and Anxiety measures31. T-scores greater than 60 indicate moderate-to-severe symptoms in cancer32; thus, we dichotomized scores as below 60 or at/above 60.

Health literacy was assessed at study baseline using a single item, “How comfortable are you filling out medical forms by yourself?” This item is validated as a brief screener for health literacy in diverse populations33,34. Scores were dichotomized as low (Not at all/A little bit/Somewhat) or high (Quite a bit/Extremely).

Shared decision making was measured using the CollaboRATE scale, which includes three items asking patients 1) how much effort was made to make sure they understood, 2) were listened to, and 3) were included in decisions during their most recent appointment. CollaboRATE scores were dichotomized based on a top-box scoring approach, where patients were counted only if they gave a top score to all three items35,36. Scores were collected at study baseline.

Statistical Approach

To understand trends in portal utilization and healthcare utilization, we calculated Spearman correlation coefficients between use of the four portal features and number of oncology clinic visits.

Next, we constructed Poisson regression models to understand the associations between patient use of each portal feature and thirteen variables including: sex, age, race, ethnicity, education, insurance, presence of at least one severe physical symptom, presence of moderate/severe anxiety, presence of moderate/severe depression, health literacy, CollaboRATE scores, clinic region, and number of oncology visits. We report the results of univariate models for each patient characteristic and full models. The employment category was excluded from the full models due to multicollinearity (VIFs > 5) with age and insurance. For models evaluating PRO completion, we accounted for varying number of invitations by including a log-transformed offset term of invitation count.

Finally, to explore how engagement with traditional portal features was related to engagement with PRO completion, we ran models testing whether frequency of messaging, lab views, and notes views were each associated with PRO completion, adjusting for the other thirteen variables.

All analyses were conducted in R version 4.3.2 software37.

Results

4104 patients were consented and 3457 were included in analyses. Patients were excluded if they were missing baseline survey responses (n=613), their MyChart account was inactive (n=23), or they did not have a cancer diagnosis in the EHR (n=22). These were not mutually exclusive. The final sample consisted of 3,457 patients recruited consecutively at 30 oncology clinics (Table 1). Patients were 65% female, 85% white, and 85% non-Hispanic/Latino. Mean age was 60.8 years (SD 12.8). The sample was highly educated, with 7.1% of patients having a high school education or less, 24% some college, 35% a college degree, and 34% a graduate degree. Most participants were employed (45%) or retired (38%).Most had private insurance (54%) or Medicare/Medicare Advantage (40%). The most common cancer types were breast (30%) and lymphoma (12%), and all other categories represented less than 10% of the sample.

Table 1:

Demographics of study population

Characteristic N = 3,4571
Sex
 Male 1,196 (35%)
 Female 2,261 (65%)
Age Category
 18-<40 years 230 (6.7%)
 40-65 years 1,851 (54%)
 >65 years 1,376 (40%)
Education Category
 High school or less 243 (7.1%)
 Some college/Technical/Associate degree 817 (24%)
 College graduate 1,207 (35%)
 Graduate/Advanced degree 1,175 (34%)
 Unknown 15
Employment Category
 Employed 1,551 (45%)
 On disability, On leave of absence 303 (8.8%)
 Not working for pay/Other 277 (8.0%)
 Retired 1,300 (38%)
 Prefer not to answer 26 (0.8%)
Race
 White 2,946 (85%)
 Asian 85 (2.5%)
 Black/African American 159 (4.6%)
 Other/Unknown/Not Reported 267 (7.7%)
Ethnicity
 Not Hispanic/Latino 2,939 (85%)
 Hispanic/Latino 136 (3.9%)
 Unknown/Not Reported 382 (11%)
Insurance category
 Private 1,865 (54%)
 Medicare or Medicare Advantage 1,393 (40%)
 Medicaid 111 (3.2%)
 Military 23 (0.7%)
 None 24 (0.7%)
 I do not know 34 (1.0%)
 Unknown 7
PRO-CTCAE Any Sx Frequency or Severity Grade 3/4 724 (21%)
PRO-CTCAE # of Sxs Frequency or Severity Grade 3/4
 0 2,733 (79%)
 1 535 (15%)
 2 144 (4.2%)
 3 38 (1.1%)
 4 5 (0.1%)
 5 2 (<0.1%)
Anxiety T-Score >=60 537 (16%)
Depression T-Score >=60 300 (8.7%)
How comfortable are you filling out medical forms by yourself (dichotomous)
 Not at all/A little bit/Somewhat 294 (8.5%)
 Quite a bit/Extremely 3,163 (91%)
Collaborate (top box)
 Less than top score 2,280 (66%)
 Top Score to all 1,177 (34%)
Region
 Central 2,060 (60%)
 North 439 (13%)
 West 937 (27%)
 South 21 (0.6%)
Cancer Type
 Breast 1,045 (30%)
 CNS/Brain 30 (0.9%)
 Colon or rectal 216 (6.2%)
 Other GI 311 (9.0%)
 Head/Neck 67 (1.9%)
 Lung/thoracic 139 (4.0%)
 Sarcoma/Bone 65 (1.9%)
 Melanoma 102 (3.0%)
 Non-melanoma skin 6 (0.2%)
 Ovary 94 (2.7%)
 Cervix 25 (0.7%)
 Endometrial (uterus) 101 (2.9%)
 Other Gyn 14 (0.4%)
 Prostate 114 (3.3%)
 Other GU 113 (3.3%)
 Leukemia 301 (8.7%)
 Lymphoma 400 (12%)
 Other heme 219 (6.3%)
 Other 3 (<0.1%)
 Multiple malignancy 64 (1.9%)
 Endocrine 28 (0.8%)
Number of Oncology Visits
 Mean (SD) 6 (7)
 Median, (Range) 3, (0, 68)
Open in a new tab
1

n (%)

The median number of oncology visits during the study period was 3 (IQR 0-68). Severe physical symptoms were reported by 21% (724) of patients; for mental health, 16% (537) reported moderate/severe anxiety and 8.7% (300) reported moderate/severe depression. The population was highly health literate where 91% (3,163) were quite a bit or extremely comfortable filling out medical forms on their own. Health literacy was meaningfully related to educational attainment, with a small effect size (χ²(3,3442)=100.3, p<.01, R²=.009). For patient-provider shared decision making, approximately a third of patients (34%; 1,177) gave the maximum positive ratings across all three questions.

Engagement with Portal Features

Patients viewed their laboratory results a Median of 23 days out of the year-long study period (IQR 10-46), and 97.5% (3,372) viewed their lab results at least once. Patients used the clinician messaging feature a Median of 11 days (IQR 5-21) with 95.5% (3,300) using at least once, and viewed physician notes a Median of 2 days (IQR 0-6) with only 71.2% (2,461) using this feature at least once. Patients received between one and ten invitations to complete PROs in the study period (Median 2, IQR 1-4) depending on their number of oncology appointments. PRO completion was described as a proportion complete out of total invitations. The average rate of PRO completion was 46.6% with a bimodal distribution (Supplementary Figure 1).

Factors Associated with Portal Engagement

Use of any portal feature was positively correlated with use of any other portal feature (ρ = 0.32 – 0.68). Additionally, number of oncology appointments was positively correlated with use of all four portal features (ρ = 0.37 – 0.62).

Patient-initiated Messages to Clinicians

Several factors were significantly associated with messaging frequency (Table 2). Patients with anxiety above a PROMIS T-score of 60 at baseline had 1.19 times the rate (IRR) of messaging clinicians compared to those with lower anxiety scores (95% CI 1.16-1.23, P<0.001). Patients with at least one severe physical symptom at baseline had a higher rate of messaging clinicians (IRR 1.07, 95% CI 1.05-1.10, P<0.001). Patients with higher health literacy had a higher rate (IRR 1.08, 95% CI 1.04-1.12, P<0.001). As education increased, patients had higher rates of messaging (comparing to high school or less category, all Ps<0.001; some college: IRR 1.10, 95% CI 1.05-1.15; college graduate: IRR 1.18, 95% CI 1.14, 1.23; graduate degree: IRR 1.25, 95% CI 1.20-1.30). Top-box scores for shared decision making were associated with a higher rate of messaging (IRR 1.03, 95% CI 1.01-1.04, P=<0.001). Number of oncology visits predicted higher rate of messaging by a factor of 1.05 (95% CI 1.05-1.05, P<.001). Conversely, the IRR was 0.80 for Black patients compared to white patients (95% CI 0.77-0.84, P<0.001), 0.91 for Hispanic/Latino patients compared to non-Hispanic/Latino patients (95% CI 0.86-0.95, P<0.001), and 0.91 for patients with Medicaid (95% CI 0.86-0.96, P<.001).

Table 2:

Predictors of Count of Messages to Providers

Unadjusted
(Univariate)		 Multivariable Model
Characteristic IRR1 p-value IRR1 p-value
Sex
 Male — —
 Female 0.91 (0.89, 0.93) <0.001 1.01 (0.99, 1.03) 0.369
Age Category
 18-<40 years — —
 40-65 years 0.89 (0.86, 0.92) <0.001 0.98 (0.94, 1.01) 0.190
 >65 years 0.94 (0.91, 0.97) <0.001 0.98 (0.94, 1.03) 0.428
Education Category
 High school or less — —
 Some college/Technical/Associate degree 1.12 (1.07, 1.16) <0.001 1.10 (1.06, 1.15) <0.001
 College graduate 1.23 (1.19, 1.28) <0.001 1.18 (1.14, 1.23) <0.001
 Graduate/Advanced degree 1.26 (1.21, 1.30) <0.001 1.25 (1.20, 1.30) <0.001
Employment Category
 Employed —
 On disability, On leave of absence 1.39 (1.35, 1.43) <0.001
 Not working for pay/Other 0.96 (0.93, 1.00) 0.025
 Retired 1.03 (1.01, 1.05) 0.001
 Prefer not to answer 1.36 (1.24, 1.48) <0.001
Race
 White — —
 Asian 1.04 (0.99, 1.10) 0.145 0.98 (0.943 1.04) 0.568
 Black/African American 0.91 (0.87, 0.95) <0.001 0.80 (0.77, 0.84) <0.001
 Other/Unknown/Not Reported 0.95 (0.92, 0.98) 0.002 0.92 (0.88, 0.95) <0.001
Ethnicity
 Not Hispanic/Latino — —
 Hispanic/Latino 0.86 (0.82, 0.91) <0.001 0.91 (0.86, 0.95) <0.001
 Unknown/Not Reported 1.09 (1.06, 1.12) <0.001 1.07 (1.03, 1.10) <0.001
Insurance category
 Private — —
 Medicare or Medicare Advantage 1.04 (1.02, 1.05) <0.001 1.08 (1.05, 1.11) <0.001
 Medicaid 0.91 (0.87, 0.96) <0.001 0.91 (0.86, 0.96) <0.001
 Military 1.02 (0.91, 1.13) 0.762 1.03 (0.92, 1.14) 0.582
 None 0.94 (0.85, 1.05) 0.287 0.97 (0.87, 1.08) 0.563
 I do not know 0.90 (0.82, 0.99) 0.029 0.99 (0.90, 1.09) 0.851
PRO-CTCAE Any Sx Frequency or Severity Grade 3/4 1.18 (1.16, 1.20) <0.001 1.07 (1.05, 1.10) <0.001
Anxiety T-Score >=60 1.24 (1.21, 1.27) <0.001 1.19 (1.16, 1.23) <0.001
Depression T-Score >=60 1.09 (1.06, 1.12) <0.001 0.99 (0.96, 1.02) 0.551
How comfortable are you filling out medical forms by yourself (dichotomous)
 Not at all/A little bit/Somewhat — —
 Quite a bit/Extremely 1.06 (1.03, 1.10) <0.001 1.08 (1.05, 1.12) <0.001
Collaborate (top box)
 Less than top score — —
 Top Score to all 1.00 (0.98, 1.02) 0.884 1.03 (1.01, 1.05) <0.001
Region
 Central — —
 North 1.01 (0.98, 1.03) 0.652 0.98 (0.96, 1.01) 0.238
 West 0.71 (0.69, 0.72) <0.001 0.74 (0.72, 0.75) <0.001
 South 0.79 (0.70, 0.88) <0.001 0.81 (0.72, 0.92) <0.001
Number of Oncology Visits 1.05 (1.05, 1.05) <0.001 1.05 (1.05, 1.05) <0.001
Open in a new tab
1

IRR = Incidence Rate Ratio

Laboratory Results Views

Several factors were associated with the frequency of viewing lab results (Table 3). Patients with moderate/severe PROMIS anxiety scores at baseline viewed lab results at a higher rate (IRR 1.13, 95% CI 1.11-1.15, P<.001), as did those with at least one severe physical symptom (IRR 1.09, 95% CI 1.08-1.11, P<.001). Patients with high health literacy had higher rates of lab views (IRR 1.07, 95% CI 1.05-1.09, P<.001) as did patients with top-box shared decision making (IRR 1.05, 95% CI 1.04-1.06, P<.001). Compared to patients with a high school diploma or less, patients with some college and patients with graduate degrees had higher rates (some college: IRR 1.04, 95% CI 1.02-1.07, P=.001; graduate degree: IRR 1.03, 95% CI 1.00-1.06, P=.023). Number of oncology visits predicted a higher rate of lab views by a factor of 1.05 (95% CI 1.05-1.05, P<.001). Female patients viewed labs at a lower rate than male patients (IRR 0.90, 95% CI 0.89-0.91, P<.001). Asian patients had a higher rate of lab viewing compared to white patients (IRR 1.23, 95% CI 1.19-1.27, P<.001). Black/African American patients viewed labs at a lower rate than white patients (IRR 0.76, 95% CI 0.74-0.78, P<.001), as did Hispanic/Latino patients compared to non-Hispanic/Latino patients (IRR 0.96, 95% CI 0.93-0.99, P=.012). Patients with Medicaid insurance had lower rates of viewing labs than patients with private insurance (IRR 0.84, 95% CI 0.81-0.87, P<.001).

Table 3:

Predictors of Laboratory Views

Unadjusted
(Univariate)		 Multivariable Model
Characteristic IRR1 p-value IRR1 p-value
Sex
 Male — —
 Female 0.80 (0.79, 0.81) <0.001 0.90 (0.89, 0.91) <0.001
Age Category
 18-<40 years — —
 40-65 years 0.79 (0.78, 0.81) <0.001 0.86 (0.84, 0.88) <0.001
 >65 years 0.85 (0.83, 0.86) <0.001 0.88 (0.85, 0.90) <0.001
Education Category
 High school or less — —
 Some college/Technical/Associate degree 1.05 (1.02, 1.07) <0.001 1.04 (1.02, 1.07) 0.001
 College graduate 1.01 (0.99, 1.04) 0.306 0.98 (0.95, 1.00) 0.076
 Graduate/Advanced degree 1.00 (0.98, 1.03) 0.695 1.03 (1.00, 1.06) 0.023
Employment Category
 Employed —
 On disability, On leave of absence 1.26 (1.24, 1.29) <0.001
 Not working for pay/Other 0.84 (0.82, 0.86) <0.001
 Retired 1.01 (1.00, 1.03) 0.031
 Prefer not to answer 1.19 (1.12, 1.26) <0.001
Race
 White — —
 Asian 1.20 (1.16, 1.24) <0.001 1.23 (1.19, 1.27) <0.001
 Black/African American 0.83 (0.81, 0.86) <0.001 0.76 (0.74, 0.78) <0.001
 Other/Unknown/Not Reported 0.92 (0.90, 0.94) <0.001 0.88 (0.86, 0.91) <0.001
Ethnicity
 Not Hispanic/Latino — —
 Hispanic/Latino 0.93 (0.90, 0.96) <0.001 0.96 (0.93, 0.99) 0.012
 Unknown/Not Reported 1.15 (1.13, 1.17) <0.001 1.07 (1.05, 1.10) <0.001
Insurance category
 Private — —
 Medicare or Medicare Advantage 1.02 (1.01, 1.03) 0.002 1.03 (1.02, 1.05) <0.001
 Medicaid 0.90 (0.87, 0.93) <0.001 0.84 (0.81, 0.87) <0.001
 Military 0.93 (0.87, 1.00) 0.063 0.95 (0.88, 1.03) 0.208
 None 1.04 (0.97, 1.11) 0.248 1.01 (0.94, 1.08) 0.846
 I do not know 1.22 (1.16, 1.29) <0.001 1.18 (1.11, 1.24) <0.001
PRO-CTCAE Any Sx Frequency or Severity Grade 3/4 1.19 (1.18, 1.21) <0.001 1.09 (1.08, 1.11) <0.001
Anxiety T-Score >=60 1.18 (1.17, 1.20) <0.001 1.13 (1.11, 1.15) <0.001
Depression T-Score >=60 1.07 (1.05, 1.09) <0.001 1.00 (0.97, 1.02) 0.846
How comfortable are you filling out medical forms by yourself (dichotomous)
 Not at all/A little bit/Somewhat — —
 Quite a bit/Extremely 0.99 (0.97, 1.01) 0.288 1.07 (1.05, 1.09) <0.001
Collaborate (top box)
 Less than top score — —
 Top Score to all 1.03 (1.02, 1.04) <0.001 1.05 (1.04, 1.06) <0.001
Region
 Central — —
 North 1.07 (1.05, 1.08) <0.001 1.05 (1.03, 1.07) <0.001
 West 0.90 (0.89, 0.91) <0.001 0.94 (0.92, 0.95) <0.001
 South 1.11 (1.03, 1.19) 0.004 1.11 (1.03, 1.19) 0.004
Number of Oncology Visits 1.05 (1.05, 1.05) <0.001 1.05 (1.05, 1.05) <0.001
Open in a new tab
1

IRR = Incidence Rate Ratio

Physician Notes Views

For physician note viewing (Table 4), patients with moderate/severe PROMIS anxiety scores at baseline had a higher rate of viewing physician notes (IRR 1.16, 95% CI 1.11-1.21, P<.001); those with moderate/severe depression scores at baseline had a higher rate of viewing notes (IRR 1.17, 95% CI 1.10-1.24, P<.001); and those with a severe physical symptoms at baseline had a higher rate of viewing notes (IRR 1.10, 95% CI 1.06-1.14, P<.001). Patients with high health literacy also had a higher rate of viewing notes (IRR 1.10, 95% CI 1.04-1.17, P=.001), as did patients with high shared decision making scores (IRR 1.15, 95% CI 1.11-1.19, P<.001). Education did not clearly predict rates of notes viewing. Number of oncology visits predicted a higher rate of notes views by a factor of 1.05 (95% CI 1.05-1.05, P<.001). Female patients viewed notes at a greater rate than male patients (IRR 1.19, 95% CI 1.15-1.24, P<.001). Asian patients viewed notes at a higher rate than white patients (IRR 1.19, 95% CI = 1.08-1.31, P<.001), and Black patients had a lower rate compared to white patients (IRR 0.79, 95% CI 0.73-0.86, P<.001). Patients with Medicaid insurance had a lower rate than those with private insurance (IRR 0.67, 95% CI 0.60-0.74, P<.001).

Table 4:

Predictors of Views of Physician Notes

Unadjusted
(Univariate)		 Multivariable Model
Characteristic IRR1 p-value IRR1 p-value
Sex
 Male — —
 Female 1.10 (1.06, 1.13) <0.001 1.19 (1.15, 1.24) <0.001
Age Category
 18-<40 years — —
 40-65 years 0.98 (0.93, 1.05) 0.624 1.05 (0.99, 1.12) 0.134
 >65 years 0.89 (0.84, 0.95) <0.001 1.14 (1.06, 1.24) <0.001
Education Category
 High school or less — —
 Some college/Technical/Associate degree 0.99 (0.93, 1.06) 0.840 1.01 (0.95, 1.08) 0.705
 College graduate 0.90 (0.84, 0.95) <0.001 0.89 (0.83, 0.95) <0.001
 Graduate/Advanced degree 0.89 (0.84, 0.95) <0.001 0.96 (0.90, 1.03) 0.258
Employment Category
 Employed —
 On disability, On leave of absence 1.12 (1.07, 1.18) <0.001
 Not working for pay/Other 0.81 (0.76, 0.86) <0.001
 Retired 0.91 (0.99, 0.94) <0.001
 Prefer not to answer 1.28 (1.09, 1.49) 0.002
Race
 White — —
 Asian 1.11 (1.01, 1.21) 0.034 1.19 (1.08, 1.31) <0.001
 Black/African American 0.80 (0.74, 0.87) <0.001 0.79 (0.73, 0.86) <0.001
 Unknown/Not Reported 1.54 (1.48, 1.61) <0.001 0.74 (0.69, 0.80) <0.001
Ethnicity
 Not Hispanic/Latino — —
 Hispanic/Latino 0.89 (0.92, 0.97) 0.008 0.99 (0.90, 1.08) 0.751
 Unknown/Not Reported 1.54 (1.48, 1.61) <0.001 1.39 (1.33, 1.46) <0.001
Insurance category
 Private — —
 Medicare or Medicare Advantage 0.84 (0.82, 0.87) <0.001 0.83 (0.79, 0.87) <0.001
 Medicaid 0.77 (0.70, 0.84) <0.001 0.67 (0.60, 0.74) <0.001
 Military 1.63 (1.41, 1.88) <0.001 1.58 (1.36, 1.82) <0.001
 None 0.90 (0.74, 1.08) 0.275 0.77 (0.64, 0.93) 0.007
 I do not know 1.21 (1.05, 1.38) 0.006 1.22 (1.06, 1.40) 0.005
PRO-CTCAE Any Sx Frequency or Severity Grade 3/4 1.26 (1.22, 1.31) <0.001 1.10 (1.06, 1.14) <0.001
Anxiety T-Score >=60 1.36 (1.31, 1.41) <0.001 1.16 (1.11, 1.21) <0.001
Depression T-Score >=60 1.31 (1.25, 1.37) <0.001 1.17 (1.10, 1.24) <0.001
How comfortable are you filling out medical forms by yourself (dichotomous)
 Not at all/A little bit/Somewhat — —
 Quite a bit/Extremely 1.08 (1.02, 1.14) 0.008 1.10 (1.04, 1.17) 0.001
Collaborate (top box)
 Less than top score — —
 Top Score to all 1.14 (1.11, 1.18) <0.001 1.15 (1.11, 1.19) <0.001
Region
 Central — —
 North 1.57 (1.51, 1.64) <0.001 1.44 (1.38, 1.50) <0.001
 West 1.26 (1.22, 1.31) <0.001 1.25 (1.20, 1.29) <0.001
 South 1.63 (1.37, 1.91) <0.001 1.26 (1.06, 1.49) 0.003
Number of Oncology Visits 1.05 (1.05, 1.05) <0.001 1.05 (1.05, 1.06) <0.001
Open in a new tab
1

IRR = Incidence Rate Ratio

PRO Completion

Several factors were associated with PRO completion rates (Table 5). Patients with top-box shared decision making had a higher rate of PRO completion (IRR 1.12, 95% CI 1.0-1.18, P<.001). Higher levels of education predicted PRO completion, such that each higher category of education predicted higher rates of engagement (compared to high school or less, some college: IRR 1.11, 95% CI 0.99-1.25, P=.076; college graduate: IRR 1.17, 95% CI 1.04-1.31, P=.010; graduate degree: IRR 1.29, 95% CI 1.15-1.45,P<.001). Female patients had a higher rate than male patients (IRR 1.06, 95% CI 1.00-1.12, P=.043). Black patients had a lower rate of PRO completion compared to white patients (IRR 0.81, 95% CI 0.70-0.93, P=.003), Hispanic/Latino patients had a lower rate compared to non-Hispanic/Latino patients (IRR 0.70, 95% CI 0.59-0.83, P<.001), and patients with Medicaid insurance had a lower rate than those with private insurance (IRR 0.78, 95% CI 0.65-0.93, P=.006).

Table 5:

Predictors of PRO Completion

Unadjusted
(Univariate)		 Multivariable
Model
Characteristic IRR1 p-value IRR1 p-value
Sex
 Male — —
 Female 1.06 (1.00, 1.12) 0.047 1.06 (1.00, 1.12) 0.043
Age Category
 18-<40 years — —
 40-65 years 1.19 (1.06, 1.34) 0.003 1.16 (1.04, 1.31) 0.012
 >65 years 1.17 (1.04, 1.32) 0.009 1.12 (0.98, 1.29) 0.108
Education Category
 High school or less — —
 Some college/Technical/Associate degree 1.13 (1.01, 1.28) 0.042 1.11 (0.99, 1.26) 0.084
 College graduate 1.21 (1.08, 1.35) 0.001 1.16 (1.03, 1.31) 0.013
 Graduate/Advanced degree 1.33 (1.19, 1.49) <0.001 1.89 (1.14, 1.45) <0.001
Employment Category
 Employed —
 On disability, On leave of absence 0.88 (0.80, 0.97) 0.007
 Not working for pay/Other 0.90 (0.81, 1.00) 0.050
 Retired 1.04 (0.98, 1.10) 0.167
 Prefer not to answer 0.77 (0.55, 1.05) 0.123
Race
 White — —
 Asian 0.91 (0.76, 1.08) 0.275 0.91 (0.76, 1.09) 0.315
 Black/African American 0.77 (0.67, 0.88) <0.001 0.81 (0.70, 0.93) 0.003
 Other/Unknown/Not Reported 0.87 (0.78, 0.96) 0.009 0.96 (0.86, 1.07) 0.492
Ethnicity
 Not Hispanic/Latino — —
 Hispanic/Latino 0.64 (0.54, 0.75) <0.001 0.70 (0.59, 0.83) <0.001
 Unknown/Not Reported 1.09 (1.01, 1.17) 0.029 1.08 (0.99, 1.17) 0.079
Insurance category
 Private — —
 Medicare or Medicare Advantage 0.97 (0.92, 1.03) 0.303 1.00 (0.92, 1.09) 0.992
 Medicaid 0.66 (0.55, 0.77) <0.001 0.78 (0.65, 0.93) 0.006
 Military 0.98 (0.72, 1.28) 0.870 1.05 (0.77, 1.38) 0.760
 None 0.76 (0.53, 1.06) 0.121 0.87 (0.60, 1.20) 0.417
 I do not know 0.90 (0.67, 1.17) 0.457 1.05 (0.79, 1.38) 0.719
PRO-CTCAE Any Sx Frequency or Severity Grade 3/4 0.85 (0.80, 0.91) <0.001 0.87 (0.10, 0.93) <0.001
Anxiety T-Score >=60 0.97 (0.91, 1.05) 0.459 1.07 (0.98, 1.16) 0.145
Depression T-Score >=60 0.88 (0.79, 0.97) 0.010 0.93 (0.83, 1.04) 0.199
How comfortable are you filling out medical forms by yourself (dichotomous)
 Not at all/A little bit/Somewhat — —
 Quite a bit/Extremely 1.22 (1.11, 1.35) <0.001 1.10 (0.99, 1.22) 0.076
Collaborate (top box)
 Less than top score — —
 Top Score to all 1.12 (1.06, 1.18) <0.001 1.12 (1.06, 1.18) <0.001
Region
 Central — —
 North 1.26 (1.17, 1.36) <0.001 1.22 (1.13, 1.32) <0.001
 West 1.07 (1.01, 1.14) 0.018 1.05 (0.99, 1.11) 0.135
 South 1.19 (0.90, 1.55) 0.195 1.17 (0.88, 1.54) 0.266
Number of Oncology Visits 1.00 (1.00, 1.01) 0.063 1.00 (1.00, 1.01) 0.009
Open in a new tab
1

IRR = Incidence Rate Ratio

Unlike other features, health literacy and high anxiety did not predict PRO completion rates. Notably, patients with at least one severe physical symptom had lower rates of PRO completion (IRR 0.87, 95% CI 0.81-0.93, P<.001).

Use of each of the three other portal features were associated with higher rates of PRO completion (Supplementary Tables 1-3).

Discussion

This study investigated how cancer patients enrolled in the NU IMPACT trial engaged with four key features of the patient portal. Our findings reveal important patterns of engagement across different demographic groups and clinical characteristics, highlighting both opportunities and challenges in the implementation of digital health technologies in cancer care.

Despite the high education level and health literacy ratings of our sample, our results replicate previously published differences in portal engagement among racial and ethnic minority groups and patients with lower socioeconomic status: Black patients, Hispanic/Latino patients, and those insured by Medicaid were less likely to engage with the portal across all or most features. These results suggest that differences in engagement across minority and low socioeconomic status groups are not strictly an issue of health literacy or education; other explanations for low engagement should be considered, such as disparities in access to digital devices and reliable internet among these populations (i.e., digital divide) 38. Results related to other demographic factors were less consistent. Sex, age, and education at times predicted more, less, or a non-linear pattern of engagement. . Finally, results on health literacy replicated existing work, such that patients with higher health literacy were more likely to engage with all four portal features.

As expected, patient clinical factors also were associated with portal engagement. Across all features, patients who had more appointments used the portal more frequently. For mental health symptoms, higher anxiety at baseline was associated with increased clinician messaging, lab viewing, and note viewing. This aligns with previous research on health information seeking behavior among patients with elevated anxiety39. For physical symptoms, patients reporting at least one severe symptom at study baseline showed higher engagement across the same three portal features (messaging, lab views, notes views), suggesting that those with greater symptom burden may rely more heavily on portal resources to contact their providers and seek information on their health. However, trends for physical symptoms differed for PRO completion: patients with a higher symptom burden were less likely to complete PROs, a trend that is not entirely unexpected as it was observed in a recent study involving patients undergoing routine cancer care in Germany24.

At the health system level, high scores of shared decision making were associated with greater use of all portal features, underscoring the role of effective patient-provider communication in fostering engagement with digital health technologies. Patients with high shared decision making may be the most likely to benefit from PROs, which inform conversations about symptom management needs. These results echo previous work on the importance of clinicians in establishing patient buy-in to PROs40,41.

Lastly, we consider PRO completion as a newer, qualitatively different feature of patient portals. Our results suggest that demographic and system-level predictors of portal engagement also apply to portal-based PRO completion. Furthermore, patients who engage with other portal features are more likely to complete PROs through the portal, controlling for known predictors of portal use. This suggests that portal-based PROs are viable for patients who already engage with the portal but may have lower uptake otherwise. Another important finding specific to PROs was that higher symptom burden – that is, at least one severe physical symptom – was associated with lower rates of PRO completion, even though it was associated with greater engagement with other features. This difference may be due to the utility of other portal features compared to PROs: messaging, viewing lab results, and viewing notes all meet patient needs for information or assistance, while PROs instead request effort from a patient. Future qualitative work should examine these differences and determine whether improved patient education on the utility of PROs could change this trend.

Limitations

The study has several limitations. Our analytic approach was to identify associations, therefore causal relationships cannot be inferred. The one-year measurement period cannot capture portal usage across the entire course of treatment. Patients varied in time since diagnosis and type of treatment received which was not accounted for in analyses. We did not directly assess patients’ reasons for engaging or not engaging with the patient portal, which could provide valuable context. Future research should employ mixed-methods or qualitative designs to provide deeper insights into engagement patterns throughout the cancer care trajectory.

Conclusions

In conclusion, our study finds that disparities persist in patient portal engagement among cancer patients, with lower engagement among racial and ethnic minorities and patients with lower socioeconomic status. We also found that moderate/severe anxiety, presence of at least one severe physical symptom, higher health literacy, and higher ratings of shared decision-making predict greater use of portal features that provide information or assistance: messaging providers, viewing lab results, and viewing notes. For PRO completion, trends for sociodemographic factors, health literacy, and shared decision-making were the same, while presence of at least one severe physical symptom predicted less PRO completion. Future research should focus on understanding the underlying causes of these differences and developing strategies to address them, ensuring that the benefits of patient portals and regular PRO reporting are accessible to all patients with cancer.

Supplementary Material

PV Appendix Figure 1
PV Appendix Tables 1-3
PV Data Sharing Statement

Table 6:

Summary of results across four portal features.

Messages Labs Notes PROs
Sex n.s. Female patients were less likely to view labs than male patients (IRR 0.90 [0.89, 0.91], P<.001). Female patients were more likely to view notes than male patients (IRR 1.19 [1.15, 1.24], P<.001). Female patients were more likely to complete PROs (IRR 1.06 [1.00, 1.13], P=.043).
Age n.s. Patients above the age of 40 were less likely to view labs than patients under 40. (IRRs 0.86 [0.84, 0.88], 0.88 [0.85, 0.90]; Ps<.001). Patients above the age of 65 were more likely to view physician notes than patients below the age of 40 (IRR 1.14 [1.06, 1.24], P<.001). Patients between 40-65 years old were more likely to complete PROs than patients under 40 years old (IRR 1.16 [1.04, 1.31], P=.012).
Education As education increases, likelihood of sending a message increases. (IRRs 1.10-1.25, all Ps<.001). Patients with graduate degrees and with some college were more likely to view labs than patients with high school or less or patients with a college degree. College graduates were less likely to view notes than other education categories (IRR 0.89 [0.83, 0.95], P<.001). As education increases, likelihood of completing PROs increases (IRRs 1.11-1.28, Ps from .084 to < .001).
Race Black patients are less likely than White patients to send messages. (IRR 0.80 [0.77, 0.84], P<.001). Black patients were less likely than White patients to view labs (IRR 0.76 [0.74, 0.78], P<.001), Black patients were less likely than White patients to view notes (IRR 0.79 [0.73, 0.86], P<.001) Black patients were less likely than White patients to complete PROs (IRR 0.81 [0.70, 0.93], P=.003).
while Asian patients were more likely to view labs than White patients (IRR 1.23 [1.19, 1.27], P<.001). while Asian patients were more likely to view labs than White patients (IRR 1.19 [1.08, 1.31], P<.001).
Ethnicity Hispanic/Latino patients are less likely than non-Hispanic/Latino patients to send messages. (IRR 0.91 [0.85, 0.95], P<.001). Hispanic/Latino patients were less likely to view labs that non-Hispanic/Latino patients (IRR 0.96 [0.93, 0.99], P=.012). n.s. Hispanic/Latino patients were less likely to complete PROs than non-Hispanic/Latino. (IRR 0.70 [0.59, 0.83], P<.001)
Insurance Compared to patients with private insurance, patients with Medicaid were less likely to send messages (IRR 0.91 [0.86, 0.96], P<.001), Compared to patients with private insurance, patients with Medicaid were less likely to view labs (IRR 0.84 [0.81, 0.87], P< .001), Compared to patients with private insurance, patients with Medicaid and Medicare were less likely to view notes (IRRs 0.76 [0.60, 0.74], 0.83 [0.79, 0.87], Ps<.001), Compared to patients with private insurance, patients with Medicaid were less likely to complete PROs. (IRR 0.78 [0.65, 0.93],P=.006)
and patients with Medicare were more likely to send messages (IRR 1.09 [1.05, 1.11], P< .001). and patients with Medicare were more likely to view labs (IRR 1.03 [1.02, 1.04], P< .001). and patients with Military health insurance were more likely (IRR = 1.58 [1.36, 18.2], P<.001).
PRO-CTCAE 1+ Severe Physical Symptom Patients with severe physical symptoms were more likely to send messages (IRR 1.07 [1.05, 1.10], P<.001). Patients with severe physical symptoms were more likely to view labs (IRR 1.09 [1.08, 1.11], P<.001). Patients with severe physical symptoms were more likely to view notes (IRR 1.10 [1.06, 1.14], P<.001). Patients with severe physical symptoms were less likely to complete PROs (IRR 0.87 [0.81, 0.93], P<.001).
PROMIS Anxiety T-score >60 Patients with high anxiety scores were more likely to send messages (IRR 1.19 [1.16, 1.23), P<.001). Patients with high anxiety scores were more likely to view labs (IRR 1.13 [1.11, 1.15, P<.001). Patients with high anxiety scores were more likely to view notes (IRR 1.16 [1.11, 1.21], P<.001). n.s.
PROMIS Depression T-score >60 n.s. n.s. Patients with high depression scores were more likely to view notes (IRR 1.17 [1.10, 1.24], P<.001). n.s.
Health literacy (How comfortable are you filling out medical forms?) Patients with higher health literacy were more likely to send messages (IRR 1.08 [1.05, 1.12], P<.001) Patients with higher health literacy were more likely to view labs (IRR 1.07 [1.05, 1.09], P<.001). Patients with higher health literacy were more likely to view notes (IRR 1.10 [1.04, 1.17], P=.001). n.s.
Patient-provider communication (CollaboRATE top-box scored) Patients with higher ratings of patient-provider communication were more likely to send messages (IRR 1.03 [1.01, 1.05], P<.001). Patients with higher ratings of patient-provider communication were more likely to view labs (IRR 1.05 [1.04, 1.06], P<.001). Patients with higher ratings of patient-provider communication were more likely to view notes (IRR 1.10 [1.04, 1.17], P<.001). Patients with higher ratings of patient-provider communication were more likely to complete PROs (IRR 1.12 [1.06, 1.18], P< .001).
Region Some differences by clinic region. Some differences by clinic region. Some differences by clinic region. Some differences by clinic region.
Number of oncology visits Patients with more oncology clinic visits were more likely to send messages (IRR 1.05 [1.05, 1.05], P<.001). Patients with more oncology clinic visits were more likely to view labs (IRR 1.05 [1.05, 1.05], P<.001). Patients with more oncology clinic visits were more likely to view notes (IRR 1.05 [1.05, 1.06], P<.001). Patients with more oncology visits were more likely to complete PROs (IRR 1.00 [1.00, 1.01], P=.009).
Open in a new tab

Notes. Orange = increase from reference group; blue = decrease from reference group; gray = differences across multi-category groups. n.s. = not significant.

Context Summary.

Key Objective:

What characteristics of oncology patients predict engagement with patient portal features, including a novel feature of remote patient-reported outcome measure (PRO) completion?

Knowledge Generated:

Oncology patients with high anxiety, severe physical symptoms, greater number of appointments, high health literacy, and high ratings of patient-provider communication use several patient portal features more often, while patients of Black race, Hispanic/Latino ethnicity, and Medicaid health insurance use these patient portal features less often. These trends held for the newer feature of remote PRO completion, with the exception that patients with severe physical symptoms were less likely to complete PROs through the portal.

Relevance (Dr. Warner, EIC):

This study illuminates several important associations with engagement with patient portal features, suggesting that additional resources are needed to support populations at risk of low engagement and perhaps fewer resources for those already expected to have high levels of engagement. Knowledge of differential engagement will help efforts to mitigate downstream bias in real-world PRO data.

Funding:

This work was funded by the National Cancer Institute grant UM1CA233035-01. K.N. was supported by the NIH/NCI training grant CA193193.

Disclaimer:

The article was prepared as part of one of the author’s (REJ) official duties as an employee of the US Federal Government. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the National Cancer Institute.

Footnotes

Trial Registration: ClinicalTrials.gov NCT03988543

Previous Presentation: This study was previously presented as a poster at the ASCO Conference in May 2024.

References

  • 1.Brands MR, Gouw SC, Beestrum M, Cronin RM, Fijnvandraat K, Badawy SM. Patient-Centered Digital Health Records and Their Effects on Health Outcomes: Systematic Review. J Med Internet Res. 2022;24(12):e43086. doi: 10.2196/43086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Tapuria A, Porat T, Kalra D, Dsouza G, Xiaohui S, Curcin V. Impact of patient access to their electronic health record: systematic review. Inform Health Soc Care. 2021;46(2):192–204. doi: 10.1080/17538157.2021.1879810 [DOI] [PubMed] [Google Scholar]
  • 3.Bao C, Singh H, Meyer B, Kirksey K, Bardhan I. Patient-provider Engagement and its Impact on Health Outcomes: A longitudinal study of patient portal use. MIS Q. 2020;44(2). doi: 10.25300/MISQ/2020/14180 [DOI] [Google Scholar]
  • 4.Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol Off J Am Soc Clin Oncol. 2016;34(6):557–565. doi: 10.1200/JCO.2015.63.0830 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Basch E, Schrag D, Henson S, et al. Effect of Electronic Symptom Monitoring on Patient-Reported Outcomes Among Patients With Metastatic Cancer: A Randomized Clinical Trial. JAMA. 2022;327(24):2413–2422. doi: 10.1001/jama.2022.9265 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Perry MB, Taylor S, Khatoon B, et al. Examining the Effectiveness of Electronic Patient-Reported Outcomes in People With Cancer: Systematic Review and Meta-Analysis. J Med Internet Res. 2024;26(1):e49089. doi: 10.2196/49089 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Grossman LV, Masterson Creber RM, Benda NC, Wright D, Vawdrey DK, Ancker JS. Interventions to increase patient portal use in vulnerable populations: a systematic review. J Am Med Inform Assoc. 2019;26(8-9):855–870. doi: 10.1093/jamia/ocz023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Nishii A, Campos-Castillo C, Anthony D. Disparities in patient portal access by US adults before and during the COVID-19 pandemic. JAMIA Open. 2022;5(4):ooac104. doi: 10.1093/jamiaopen/ooac104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Griffin JM, Kroner BL, Wong SL, et al. Disparities in electronic health record portal access and use among patients with cancer. J Natl Cancer Inst. 2024;116(3):476–484. doi: 10.1093/jnci/djad225 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Irizarry T, Dabbs AD, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res. 2015;17(6):e4255. doi: 10.2196/jmir.4255 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Aljabri D, Dumitrascu A, Burton MC, et al. Patient portal adoption and use by hospitalized cancer patients: a retrospective study of its impact on adverse events, utilization, and patient satisfaction. BMC Med Inform Decis Mak. 2018;18(1):70. doi: 10.1186/s12911-018-0644-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Coughlin SS, Caplan L, Young L. A Review of Web Portal Use by Oncology Patients. J Cancer Treat Diagn. 2018;2(6): 10.29245/2578-2967/2018/6.1154. doi:10.29245/2578-2967/2018/6.1154 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Fatehi A, Gonzalez A, Ring DC, Queralt M. Psychosocial Factors Are Associated with Electronic Portal Registration. Clin Orthop. 2020;478(9):2079–2084. doi: 10.1097/CORR.0000000000001278 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Gerber DE, Laccetti AL, Chen B, et al. Predictors and Intensity of Online Access to Electronic Medical Records Among Patients With Cancer. J Oncol Pract. 2014;10(5):e307–e312. doi: 10.1200/JOP.2013.001347 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Singh S, Polavarapu M, Arsene C. Changes in patient portal adoption due to the emergence of COVID-19 pandemic. Inform Health Soc Care. 2023;48(2):125–138. doi: 10.1080/17538157.2022.2070069 [DOI] [PubMed] [Google Scholar]
  • 16.Turner K, Hong YR, Yadav S, Huo J, Mainous AG III. Patient portal utilization: before and after stage 2 electronic health record meaningful use. J Am Med Inform Assoc. 2019;26(10):960–967. doi: 10.1093/jamia/ocz030 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Ancker JS, Barrón Y, Rockoff ML, et al. Use of an Electronic Patient Portal Among Disadvantaged Populations. J Gen Intern Med. 2011;26(10):1117–1123. doi: 10.1007/s11606-011-1749-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Hoogenbosch B, Postma J, de M van Ginkel JM, Tiemessen NA, van Delden JJ, van Os-Medendorp H. Use and the Users of a Patient Portal: Cross-Sectional Study. J Med Internet Res. 2018;20(9):e9418. doi: 10.2196/jmir.9418 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Zaidi M, Amante DJ, Anderson E, et al. Association Between Patient Portal Use and Perceived Patient-Centered Communication Among Adults With Cancer: Cross-sectional Survey Study. JMIR Cancer. 2022;8(3):e34745. doi: 10.2196/34745 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Avdagovska M, Ballermann M, Olson K, Graham T, Menon D, Stafinski T. Patient Portal Implementation and Uptake: Qualitative Comparative Case Study. J Med Internet Res. 2020;22(7):e18973. doi: 10.2196/18973 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Fujioka JK, Bickford J, Gritke J, et al. Implementation Strategies to Improve Engagement With a Multi-Institutional Patient Portal: Multimethod Study. J Med Internet Res. 2021;23(10):e28924. doi: 10.2196/28924 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Neuner J, Fedders M, Caravella M, Bradford L, Schapira M. Meaningful Use and the Patient Portal: Patient Enrollment, Use, and Satisfaction With Patient Portals at a Later-Adopting Center. Am J Med Qual. 2015;30(2):105–113. doi: 10.1177/1062860614523488 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Issa TZ, Lee Y, Toci GR, et al. The role of socioeconomic factors as barriers to patient reported outcome measure completion following lumbar spine fusion. Spine J. 2023;23(10):1531–1539. doi: 10.1016/j.spinee.2023.05.003 [DOI] [PubMed] [Google Scholar]
  • 24.Gebert P, Hage AM, Blohmer JU, Roehle R, Karsten MM. Longitudinal assessment of real-world patient adherence: a 12-month electronic patient-reported outcomes follow-up of women with early breast cancer undergoing treatment. Support Care Cancer. 2024;32(6):344. doi: 10.1007/s00520-024-08547-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Wiegel J, Seppen B, van der Leeden M, van der Esch M, de Vries R, Bos W. Adherence to Telemonitoring by Electronic Patient-Reported Outcome Measures in Patients with Chronic Diseases: A Systematic Review. Int J Environ Res Public Health. 2021;18(19):10161. doi: 10.3390/ijerph181910161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Smith AW, DiMartino L, Garcia SF, et al. Systematic symptom management in the IMPACT Consortium: rationale and design for 3 effectiveness-implementation trials. JNCI Cancer Spectr. 2023;7(6):pkad073. doi: 10.1093/jncics/pkad073 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Garcia SF, Smith JD, Kallen M, et al. Protocol for a type 2 hybrid effectiveness-implementation study expanding, implementing and evaluating electronic health record-integrated patient-reported symptom monitoring in a multisite cancer centre. BMJ Open. 2022;12(5):e059563. doi: 10.1136/bmjopen-2021-059563 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Cella D, Garcia SF, Cahue S, et al. Implementation and evaluation of an expanded electronic health record-integrated bilingual electronic symptom management program across a multi-site Comprehensive Cancer Center: The NU IMPACT protocol. Contemp Clin Trials. 2023;128:107171. doi: 10.1016/j.cct.2023.107171 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Cella D, Lancki N, Kuharic M, et al. Web-Based Cancer Symptom Self-Management System: A Randomized Clinical Trial. JAMA Netw Open. 2025;8(5):e258353. doi: 10.1001/jamanetworkopen.2025.8353 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Basch E, Reeve BB, Mitchell SA, et al. Development of the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JNCI J Natl Cancer Inst. 2014;106(9):dju244. doi: 10.1093/jnci/dju244 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D. Item Banks for Measuring Emotional Distress From the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, Anxiety, and Anger. Assessment. 2011;18(3):263–283. doi: 10.1177/1073191111411667 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Cella D, Choi S, Garcia S, et al. Setting standards for severity of common symptoms in oncology using the PROMIS item banks and expert judgment. Qual Life Res. 2014;23(10):2651–2661. doi: 10.1007/s11136-014-0732-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Chew LD, Griffin JM, Partin MR, et al. Validation of Screening Questions for Limited Health Literacy in a Large VA Outpatient Population. J Gen Intern Med. 2008;23(5):561–566. doi: 10.1007/s11606-008-0520-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Wallace LS, Rogers ES, Roskos SE, Holiday DB, Weiss BD. Brief Report: screening items to identify patients with limited health literacy skills. J Gen Intern Med. 2006;21(8):874–877. doi: 10.1111/j.1525-1497.2006.00532.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Barr PJ, Thompson R, Walsh T, Grande SW, Ozanne EM, Elwyn G. The Psychometric Properties of CollaboRATE: A Fast and Frugal Patient-Reported Measure of the Shared Decision-Making Process. J Med Internet Res. 2014;16(1):e3085. doi: 10.2196/jmir.3085 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Elwyn G, Barr PJ, Grande SW, Thompson R, Walsh T, Ozanne EM. Developing CollaboRATE: A fast and frugal patient-reported measure of shared decision making in clinical encounters. Patient Educ Couns. 2013;93(1):102–107. doi: 10.1016/j.pec.2013.05.009 [DOI] [PubMed] [Google Scholar]
  • 37.R Core Team. R: A Language and Environment for Statistical Computing. Published online 2023. https://www.R-project.org/
  • 38.Graetz I, Gordon N, Fung V, Hamity C, Reed ME. The Digital Divide and Patient Portals: Internet Access Explained Differences in Patient Portal Use for Secure Messaging by Age, Race, and Income. Med Care. 2016;54(8):772. doi: 10.1097/MLR.0000000000000560 [DOI] [PubMed] [Google Scholar]
  • 39.Onyeaka H, Ajayi KV, Muoghalu C, et al. Access to online patient portals among individuals with depression and anxiety. Psychiatry Res Commun. 2022;2(4):100073. doi: 10.1016/j.psycom.2022.100073 [DOI] [Google Scholar]
  • 40.Sisodia RC, Dankers C, Orav J, et al. Factors Associated With Increased Collection of Patient-Reported Outcomes Within a Large Health Care System. JAMA Netw Open. 2020;3(4):e202764. doi: 10.1001/jamanetworkopen.2020.2764 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Lyleroehr MJ, Webster KA, Perry LM, et al. A mixed methods evaluation of patient perspectives on the implementation of an electronic health record-integrated patient-reported symptom and needs monitoring program in cancer care. J Patient-Rep Outcomes. 2024;8(1):66. doi: 10.1186/s41687-024-00742-8 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

PV Appendix Figure 1
NIHMS2118871-supplement-PV_Appendix_Figure_1.pdf (8.2KB, pdf)
PV Appendix Tables 1-3
NIHMS2118871-supplement-PV_Appendix_Tables_1-3.docx (35.7KB, docx)
PV Data Sharing Statement
NIHMS2118871-supplement-PV_Data_Sharing_Statement.pdf (130.6KB, pdf)

RESOURCES