Implementing the NYU Electronic Patient Visit Assessment (ePVA)^© for head and neck cancer in rural and urban populations: a study protocol for a type 1 hybrid effectiveness-implementation clinical trial

Abstract

Background

Aggressive treatment with multimodal therapies such as surgery, chemotherapy, and radiation therapy has improved survival in head and neck cancer (HNC) but at a human cost of a substantial symptom burden and impact on quality of life. We developed the NYU Electronic Patient Visit Assessment (ePVA)^© for HNC as a digital patient-reported symptom monitoring system that enables early symptom detection and real-time interventions at the point of care. With this study protocol, we aim to test the effectiveness of the ePVA in improving HNC outcomes in real-world settings and to identify implementation strategies optimizing its effectiveness.

Methods

We will conduct a longitudinal mixed-methods hybrid type I study at four National Cancer Institute-designated Comprehensive Cancer Centers serving diverse populations in rural and urban settings (New York University, the University of Kansas Cancer Center, Fox Chase Cancer Center, and Baylor College of Medicine) guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Patient eligibility criteria include having histologically diagnosed HNC and undergoing radiation therapy with or without chemotherapy for curative intent. We will also interview clinicians caring for patients with HNC at the participating institutions regarding facilitators and barriers to implementing the ePVA. The accrual goal is 270 patients. Aim 1 is to determine the effect of the ePVA on HNC symptoms in a two-arm (usual care vs. ePVA + usual care) trial. The study’s primary outcomes are patients’ self-reported social function, senses of taste and smell, and swallowing, measured by the European Organization for Research and Treatment of Cancer QLQ-C30 and QLQ-H&N35. For Aim 2, we will interview patients (n = 40) as well as clinicians (n = 30) caring for patients with HNC at the participating institutions regarding facilitators and barriers to implementing the ePVA. In Aim 3, we will integrate Aims 1 and 2 data to identify strategies that optimize the use of the ePVA.

Discussion

The overarching goal of this research is to advance cancer care by identifying implementation standards for effective, widespread use of the ePVA that apply to all patient-reported outcomes in cancer care.

Trial registration

ClinicalTrials.gov NCT06030011. Registered on 8 September 2023.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-025-09268-w.

Background

Over 72,000 patients in the United States (US) [1] and 562,328 individuals worldwide are diagnosed with head and neck cancer (HNC) [2, 3]. Most patients with HNC are treated aggressively with multimodal therapies (i.e., surgery, chemotherapy, radiation therapy). This aggressive treatment has improved survival but at a human cost of substantial symptom burden (e.g., decreased social function, inability to taste food, difficulty swallowing), pain, decreased health-related quality of life (HRQoL), and extensive acute care use (e.g., feeding tube placements, emergency room visits, and hospitalizations) [4–10].

Groundbreaking research shows that patient-reported outcome (PRO) monitoring during cancer care is associated with improved symptom control, decreased acute care use, and longer survival [11–13]. As such, the Centers for Medicare & Medicaid Services (CMS) has recommended collecting PROs in routine cancer care [14–16]. Yet, a closer investigation of the literature finds the effects of PROs on cancer outcomes vary depending on implementation strategies, sometimes yielding small effect sizes and limited user engagement [17–21]. Variations in PRO collection may be attributed to a lack of implementation standards, inequities in user access, and limited integration with electronic health record (EHR) systems [19–24], creating barriers to widespread implementation. Research is critically needed to overcome these barriers and ensure the success of collecting PROs in routine cancer care.

We developed the New York University (NYU) Electronic Patient Visit Assessment (ePVA)^© for HNC, based on the Theory of Unpleasant Symptoms [25], as a valid, reliable PRO for early detection of uncontrolled symptoms, using branching logic and binomial items to tailor the measure to the patient experience [26–28]. The ePVA has since evolved into a digital patient-reported symptom monitoring system, providing actionable information at the point of care that enables clinicians to provide real-time interventions. A pilot randomized clinical trial among 32 patients with HNC undergoing radiation therapy (RT) with or without chemotherapy revealed that those assigned to the ePVA + usual care arm reported significantly less severe HNC symptoms and decreased acute care use than those assigned to the usual care arm [29].

This protocol builds on the pilot study by testing the effectiveness of the ePVA as a digital patient-reported monitoring system with patient responses emailed to the clinical team at the point of care versus routine collection of PROs in cancer care to improve HNC outcomes in real-world settings and identify implementation strategies optimizing its effectiveness. To achieve this goal, we will conduct a longitudinal mixed-methods hybrid type I study [30] at four National Cancer Institute (NCI)-designated comprehensive cancer centers serving diverse populations in rural and urban settings. We will also explore integrating the ePVA with EHR systems using Fast Healthcare Interoperability Resources (FHIR) and the Substitutable Medical Applications, Reusable Technologies (SMART) standards to transmit actionable information for symptom management at the point of care.

Study hypothesis

We hypothesize that participants assigned to the ePVA + usual care arm (n = 135) will have better swallowing, taste and smell, and social function (primary outcomes) than participants assigned to usual care (n = 135) at 4 weeks after completing RT.

Specific aims

To test the aforementioned hypothesis, the specific aims are as follows:

Among 270 patients with HNC undergoing RT with or without chemotherapy as follows:

• Aim 1 (quantitative aim): Determine the effect of the ePVA on HNC symptoms (primary outcomes), pain, HRQoL, and acute care use (secondary outcomes) in a two-arm (usual care vs. ePVA + usual care) multi-site randomized clinical trial.

• Aim 2 (qualitative aim): Examine HNC patients’ and clinicians’ perspectives on patient, clinician, and organization facilitators and barriers to the ePVA’s reach, adoption, effectiveness, and maintenance.

• Aim 3 (integrative aim): Identify implementation strategies optimizing the ePVA’s reach, adoption, effectiveness, and maintenance.

Exploratory aim

Explore the usability of the integration of the ePVA with EHR systems.

Methods/design

The reporting of this protocol conforms to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist, and the Standards for Reporting Qualitative Research guided the reporting of this protocol [31–35]. The selection of the study design and methods was guided by the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance), an implementation science structure for the evaluation of interventions [36–40].

Trial management structure

The lead research site (i.e., coordinating center) is The University of Texas Health Science Center at Houston (UTHealth Houston). The principal investigator (PI), project manager, and research coordinator are located at the lead research site. The PI, Dr. Van Cleave, is responsible for and oversees the project’s operations, ensuring rigorous research implementation and compliance with human subjects’ policies and regulations. The UTHealth Houston project manager and research coordinator assist Dr. Van Cleave in overseeing the operations of the research project and have constructed a research manual to ensure study fidelity across the participating sites.

The steering committee consists of clinical site PIs and study co-investigators. The clinical site PIs oversee the day-to-day research operations and work closely with research coordinators at their institutions. These day-to-day operations consist of recruitment, enrollment, data collection, data storage, and endpoint adjudication. The study co-investigators oversee other research operations including interviews with patients and clinicians, the integration of the ePVA with the electronic health records, and the data analysis.

The project manager, the UTHealth Houston research coordinator, and biostatistician meet weekly with each participating institution’s research coordinators using secure video conferencing to monitor the project’s recruitment and enrollment, ensure each site maintains fidelity to the study protocol, and verify data integrity (i.e., data completeness and accuracy). In addition, the PI and project manager hold monthly meetings with the steering committee (i.e., study co-investigators) to review the study site’s challenges and problem-solve. All decisions are recorded, including the research team member who makes the decision. The PI and the project manager also conduct semiannual site visits.

Study participants

The accrual goal for Aims 1, 2, and 3 is 270 patient participants and 30 clinician participants. The accrual goal for the exploratory aim is 40 patient participants and 30 clinician participants. The accrual goals are based on sample size calculations for Aims 1, 2, and 3 and the team’s experience from their pilot studies [26, 27].

Patient participants — Aims 1, 2, and 3 (N = 270 and ~70 patients from each site)

Patient participants’ eligibility criteria are as follows: (a) ≥ 18 years old, (b) histologically diagnosed HNC, (c) comprehends written English and/or Spanish, and (d) undergoing curative RT with or without chemotherapy. Patient participants will be excluded if they have any medical condition that could limit the patient’s ability to provide informed consent and complete the questionnaires.

Clinician participants — Aims 1, 2, and 3 (N = 30 and ~7 clinicians from each site)

Clinician participants’ eligibility criteria are as follows: (a) Physicians, nurses, dentists, social workers, nutritionists, speech/swallow therapists, research assistants, and clinical administrators caring for patients with HNC at the participating institutions, (b) ≥ 18 years old, and (c) comprehends written English or Spanish.

Patient participants — exploratory aim (N = 40 and 10 participants from each site)

Patient participants not included in the randomized study will be recruited using the same eligibility criteria for Aims 1, 2, and 3. Enrollment for the exploratory aim will begin after enrollment for Aims 1, 2, and 3 is closed.

Clinician participants — exploratory aim (N = 30 and ~8 participants from each site)

Clinician participants from each study site will be recruited using the same inclusion and exclusion criteria for Aims 1, 2, and 3.

Recruitment and retention

The study team employs multiple methods for the recruitment of participants. The site PIs meet with the research coordinators each week to review potential participants and identify patient appointments that are optimal times to educate patients on the subject. Other methods include attending weekly HNC conferences and obtaining Institutional Review Board (IRB) approval to pre-screen patients via the EHR. Each clinical site’s research coordinators store participant data in a HIPAA-compliant, secure, and central REDCap (Research Electronic Data Capture) database at UTHealth Houston [41, 42]. These data consist of the number of potential participants approached, their eligibility, the number of enrolled participants, reasons for refusal to participate, the date of enrollment, dates for data collection time points, completed data collection time points, and protocol deviations. The lead research site uses the REDCap database to monitor the status of each participant, including participants with missed data collection time points or who have dropped out of the study, as well as the reasons for their withdrawal. The lead research site aggregates these statistics and emails them to the site PIs and research coordinators.

Retention methods include daily reminder emails to patients who have not completed data collection time points followed by a telephone call. Because HNC patients become highly symptomatic during treatment, we purposefully designed our research to ensure a low respondent burden, including easy-to-use weblinks to online surveys and ePVA completion time of approximately 10 min. In Aims 1, 2, and 3, monetary incentives to patient participants are not provided to evaluate patient engagement with the ePVA in real-world settings outside of a clinical trial. In the exploratory aim, patient participants will receive US $10 for testing the ePVA integration with the electronic health record.

Study settings

This multi-site study is conducted at four NCI-designated comprehensive cancer centers in the US — NYU (New York University, New York City, NY, US), KUCC (the University of Kansas Cancer Center, Kansas City, KS, US), FCCC (Fox Chase Cancer Center, Philadelphia, PA, US), and BCM (Baylor College of Medicine, Houston, TX, US). The initial plan was to conduct the study at NYU, KUCC, and FCCC. BCM was added as a clinical site in an amendment to the protocol dated March 14, 2025, due to lower-than-expected enrollment at the initial sites and to expand the reach of the clinical trial to an additional region in the US.

Clinical trial design (Aims 1, 2, and 3)

The trial design is a hybrid type 1 study using a convergent mixed-methods approach. This hybrid type 1 design tests the effects of the ePVA on HNC outcomes to determine the superiority of the intervention over usual care using a randomized clinical trial with parallel groups while observing and gathering information on the implementation of the ePVA in real-world clinical settings. The convergent mixed-methods approach consists of the parallel conduct of Aim 1 (quantitative data) and Aim 2 (qualitative data), followed by data integration (Aim 3) through merging the two data sets.

Exploratory aim design

The study team will use the knowledge gained from Aims 1, 2, and 3 to inform the build of a prototype of the ePVA web-based app integrated with EHR systems using FHIR technology and then explore the prototype’s usability.

Description of control — usual care (Aims 1, 2, and 3)

Usual care encompasses pre-treatment, on-treatment, and posttreatment care. Pre-treatment care includes education of patients on the treatment plan. On-treatment care includes daily monitoring by a nurse in the RT department, weekly on-treatment visits with the RT advanced practice providers and/or radiation oncologists, and weekly visits with medical oncology advanced practice providers and/or medical oncologists for patients receiving RT plus chemotherapy. Posttreatment care includes follow-up visits with HNC clinicians (i.e., surgeon, medical oncologist, oral maxillofacial surgeons, and radiation oncologist) to assess the patient’s cancer status and identify the patient’s acute and long-term symptoms. The clinician’s decisions on radiologic exams and referrals to appropriate specialist services, including dental care, are based on the patient’s cancer status and symptoms. All National Comprehensive Cancer Network (NCCN) guideline-concordant therapies are allowed. Standard-of-care treatments may be administered before, during, or after study participation at the treating physician’s discretion.

Participants randomized to the usual care condition will complete all questionnaires except the ePVA on digital devices weeks 1, 3, and 5 during RT and then 4, 12, 24, and 48 weeks after RT. The design for the usual care arm is intended to provide a parallel experience to that of the intervention group in using digital PROs, except that participant responses are not reported to the HNC team.

Description of the intervention — ePVA + usual care (Aims 1, 2, and 3)

Participants randomized to the ePVA condition will receive usual care, as described above, plus the intervention. The intervention consists of the ePVA as a digital patient-reported monitoring system. Participants randomly assigned to the intervention will complete the ePVA on digital devices on weeks 1, 3, and 5 during RTradiation therapy and then 4, 12, 24, and 48 weeks after the completion of radiation therapy (see Fig. 1) plus the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 and QLQ-H&N35 questionnaires. The EORTC patient responses will not be reported to clinicians.

Fig. 1.

Schedule of enrollment, interventions, and assessments

The ePVA is comprised of 21 symptom categories of binomial (yes/no) questions to evaluate a patient’s perception of the presence or absence of a symptom. Some questions are conditional with branching logic leading to multidimensional questions to tailor the questionnaire to the needs of patients and clinicians and limit the respondent burden. The ePVA is used as a web application and will be administered through the REDCap database hosted at UTHealth Houston [41, 42]. Patients access the ePVA through a weblink and complete the survey questions on digital devices (e.g., smartphones, laptops, or devices provided by the clinic). After the patient completes the ePVA, the data are transmitted and stored on the UTHealth Houston database. Patient responses are sent through encrypted emails to the HNC clinical team (i.e., surgeons, radiation oncologists, medical oncologists, nurses, nurse practitioners, physician assistants, and/or speech/swallow therapists). The HNC team verifies that they have received and read the ePVA reports verbally or through email. The study team contacts the HNC team if the ePVA report has not been read within 24 h of the ePVA completion.

Outcome measures

Outcomes

The study’s primary outcomes are HNC symptoms (i.e., social function, senses of taste and smell, and swallowing) (see Table 1). Early detection and interventions for these symptoms may help prevent long-term functional deficits. The secondary outcomes are pain, HRQoL, and acute care services use (see Table 1).

Table 1.

Variables and measures with timepoints

Variables	Measures with timepoints
Key independent variable
•Group assignment to usual care arm or usual care + ePVA arm	•Group assignment will be operationalized as a binary variable: Usual care arm = 0 and ePVA + usual care arm = 1
Primary outcome
•HNC symptoms (social function, HNC senses of taste and smell, and swallowing) •Description: HNC symptoms of social function, senses of taste and smell, and swallowing are critical in preventing treatment delays and dose reductions that increase the likelihood of cancer recurrence [8, 43]. Early detection and interventions for these symptoms help prevent long-term function deficits [5, 6]	•HNC symptoms will be measured by the European Organization for Research and Treatment of Cancer (EORTC)© QLQ-C30 and QLQ-HN35 [44, 45]. The EORTC© QLQ-C30 consists of 30 items that form subscales to assess common cancer issues: global QoL/health, function (physical functioning, role functioning, emotional functioning, cognitive functioning, social functioning), and symptoms (fatigue, nausea, and vomiting, pain, dyspnea, insomnia, appetite loss, constipation, diarrhea, financial difficulties). The EORTC© QLQ-H&N35 consists of 35 items assessing symptoms and side effects of HNC and its treatment on a scale ranging from 0 to 100. The EORTC© QLQ-C30 and QLQ-H&N35 are commonly used valid and reliable measures of symptoms and HRQoL in HNC studies. Patients rate the extent to which they have experienced HNC symptoms, ranging from 1 (not at all) to 4 (very much). The scores are summed and then transformed to a scale ranging from 0 = best to 100 = worst •Timepoints: Baseline, every 2 weeks during RT, and weeks 4 (primary endpoint), 12, 24, and 48 after completing RT
Secondary outcome
•Pain •Description: Up to 84% of patients will experience pain during HNC treatment [46], and approximately 35% of patients with HNC will have chronic pain [47]	•The outcome will be measured by the EORTC© QLQ-H&N35. Patients rate the extent to which they have experienced HNC symptoms, ranging from 1 (not at all) to 4 (very much). The scores are summed and then transformed to a scale ranging from 0 = best to 100 = worst •Timepoints: Baseline, every 2 weeks during RT, and weeks 4 (primary endpoint), 12, 24, and 48 after completing RT
Secondary outcome
•HRQoL •Description: HRQoL, defined as the global and quality-of-life aspects related to health [44, 48], is predictive of a patient’s survival up to 10 years after diagnosis [49]	•The outcome will be measured by the EORTC QLQ-C30© and QoL/health subscale. Patients rate their QoL and health on a 7-point Likert scale, ranging from 1 (very poor) to 7 (excellent). The scores are summed and then transformed to a scale ranging from 0 = worst to 100 = best •Timepoints: Baseline, every 2 weeks during RT, and weeks 4 (primary endpoint), 12, 24, and 48 after completing RT
Secondary outcome
•Acute care services use •Description: Acute care services use is the use of services delivered in acute care settings, such as feeding tube placements, emergency room visits, and hospitalizations, and provides critical insight into patients who become debilitated during treatment and require hospitalization for supportive care to avoid treatment reductions or delays and regain the ability to function independently [5, 6, 8]	•The date and reason for acute care use will be abstracted from participants’ EHRs and stored in the REDCap database. The number of acute care visits and the number of days in acute care will be summed and coded as number of acute care visits over the research period and number of days of using acute care (e.g., number of hospital days) over the research period •Timepoints: Baseline, every 2 weeks during RT, and weeks 4 (primary endpoint), 12, 24, and 48 after completing RT
Covariates
•Demographic variables: Age, sex, race, ethnicity, comorbidities, finances, history of alcohol and tobacco use, cancer location, cancer stage, zip code, and urban/rural classification provide contextual information on participants’ health status •Clinical management data: Clinical management data (pain medication changes and dates and reasons for supportive care visits of palliative care, nutrition, speech/swallow therapy, physical therapy) provide evidence of mechanistic pathways of the effect of the ePVA as a digital patient-reported symptom monitoring system	•Demographic variables will be collected at baseline using the REDCap system •Clinical management data across all time points will be abstracted from the EHR and stored in REDCap. The number of pain medication changes and the number of visits with palliative care, nutrition, speech/swallow therapy, and physical therapy visits will be summed and coded as number of pain medication changes and number of palliative care, nutrition, speech/swallow therapy, and physical therapy visits over the research period
Exploratory outcome
•Usability of the ePVA integrated with the EHR •Description: Patients and clinicians will evaluate the usability of the ePVA integrated with the EHR. The usability evaluation assesses user satisfaction that the technology meets five Nielsen’s principles: learnability, efficiency, memorability, errors, and satisfaction (https://www.nngroup.com/articles/usability-101-introduction-to-usability/). The underlying principle of usability is that the performance of tasks and procedures is structured logically and consistently for human–computer interaction to be effective [50–53]	•The outcome will be measured by the Post-Study System Usability Questionnaire [50–53], an 11-item Likert scale developed at IBM to assess user satisfaction with a novel technology’s usefulness, interface quality, and information quality. The responses range from 1 (strongly agree) to 7 (strongly disagree). The questionnaire has demonstrated reliability and validity in evaluating a user’s perception of the usefulness of information technology •Timepoint: Patient participants will complete the Post-Study System Usability Questionnaire immediately after using the ePVA integrated with the EHR. Clinicians will answer the questionnaire after 30 participants have used the ePVA integrated with the EHR

Informed consent

Informed consent—patient participants

Informed consent will be obtained at the four participating institutions — NYU, KUCC, FCCC, and BCM by Institutional Review Board approved study personnel. The study team will use in-person and eConsent procedures. In-person recruitment and consenting will take place in a private area, such as an exam room, to protect the participant’s privacy. The eConsent procedure contains all elements of informed consent required by federal regulations (45 CFR 46.116 and 21 CFR 50.25), using telephone calls, videoconferencing, REDCap, and encrypted email services.

Informed consent — clinician participants

Informed consent will be initiated by Institutional Review Board-approved study personnel with an email asking each potential clinician participant if they are interested in participating in an interview. After indicating interest, the clinician will meet with the study team to review the consent form; explain the study procedures, risks, and benefits; and inform the clinician that their participation is voluntary and will not jeopardize their employment status.

If interested in enrolling in the study, participants will complete the consent via paper or via eConsent, and a unique research identifier will be assigned to each participant to be used as the participant ID. A signed copy of the consent form will be given to each participant.

Protection against risk

This research, which involves participants answering surveys or interview questions, represents minimal risk in that the probability and magnitude of harm or discomfort anticipated in the study are not greater than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests. If unforeseen issues arise, such as the participant becoming anxious when asked about symptoms, the research team notifies the site PI immediately. A social worker will also be available to speak with patient participants. Steps to ensure participant safety include site research coordinators calling participants who have not responded to the questionnaire and contact with participants occurring during the HNC clinical care team’s office hours for rapid notification of participant reports of adverse events.

Monitoring of adverse events is conducted during data collection visits (e.g., in person, telephone, video conferences). The site PIs adjudicate decisions on adverse events. The site PI notifies the primary PI, Janet H. Van Cleave, of adverse events within 5 days of the event. The PI will notify the Data and Safety Monitoring Committee of the adverse event within 5 days. The study will be halted when three grade 3 adverse events determined to be “probably related” to completion of the ePVA are reported to Janet Van Cleave, PI.

Interim analysis and stopping rules

We will conduct a preliminary effectiveness analysis when the outcome data are available from 50% of the planned accrual of patient participants for Aims 1, 2, and 3. We will stop the study if the usual care arm is substantially “worse” than the ePVA arm for any of the three primary endpoints. We will define “worse” using the O’Brien–Fleming boundaries [54] that will protect the type I error rate of the study.

Data sources

For Aims 1, 2, and 3, the data sources and materials will be the ePVA, EORTC QLQ-C30 and QLQ-H&N35 questionnaires, and clinician surveys administered through REDCap, qualitative patient and clinician interviews, acute care services use data abstracted from the EHR, and clinical management data abstracted from the EHR. For the exploratory aim, the data sources will be the ePVA integrated with the EHR, audio recordings of participants’ comments while completing the ePVA, and patient and clinician surveys administered through REDCap. Data integrity is reviewed during weekly team meetings. This review includes analysis of participant accrual by study site, participant retention, missing data, and confirmation that clinicians received the ePVA reports. Weekly emails are sent to sites to remind site research coordinators of upcoming participant visits and inquire about missing data. Other data integrity monitoring methods include intra-rater reliability of randomly selected charts by the UTHealth Houston Coordinating Center during the semiannual site monitoring visits to verify data accuracy. A copy of the intervention manual and data collection forms may be obtained from the corresponding author.

Data collection and storage for Aims 1, 2, and 3

Patient participant study questionnaires administered through REDCap (Aims 1 and 3)

Data will be collected using the REDCap database housed at the lead research site, UTHealth Houston. The participants can complete the questionnaires through a weblink emailed to them via REDCap or complete the questionnaires on digital devices with touchscreen technology during oncology visits. The participants may also answer questionnaires on paper if digital devices are unavailable or have technology malfunctions. Research coordinators will manually enter the patient responses into REDCap. The accuracy of the manual entry of data is verified by a second research coordinator. All paper questionnaires will be stored in a locked file cabinet in a locked room. The method of completing the ePVA is recorded in the REDCap database.

Patient and clinician participant interviews: facilitators and barriers to implementation of the ePVA (Aims 2 and 3)

The study team will conduct in-depth interviews with patient participants enrolled in Aim 1 and clinicians caring for patients with HNC across the four participating institutions to explore patient, clinician, and organization facilitators and barriers to the reach, effectiveness, adoption, implementation, and maintenance of the ePVA. For these interviews, the team will use a purposeful multistage sampling strategy to select information-rich participants for interviews to capture major variations and similarities of the phenomenon under study [55]. The first stage will focus on recruiting diverse participants to ensure the representation of perspectives across race, ethnicity, gender, and site of residence (i.e., rural vs. urban) to obtain a broad view of the study phenomena. The subsequent sampling stages will focus on recruiting participants who provide information on commonalities that emerged in previous interviews to obtain a narrower view of the study phenomena [55]. The interviews will be conducted in person in private areas in the clinical setting or via video conferencing using semi-structured interview guides with 4–5 open-ended questions, followed by probes. All interviews are expected to last 30 min, will be recorded and stored on secure password-protected databases, transcribed verbatim, and checked for accuracy.

Clinician participant surveys: engagement with ePVA (Aims 1 and 3)

The research team will survey clinicians involved in the care of patient participants every 6 months to evaluate clinician engagement. The clinician surveys will consist of quantitative questions beginning with a branching logic question asking whether the clinician uses the ePVA reports (yes/no). If the answer is “yes,” the questionnaire will direct the clinician to questions about how they use the data and what data are most valuable. If the clinician answers “no,” then the questionnaire will direct the clinician to a multiple-response question asking why the clinician does not use the ePVA. All clinicians will answer questions regarding whether they would recommend a similar system to other clinicians. We anticipate querying 30 clinicians every 6 months and a response rate of 75%.

Data abstracted from EHR: acute care use data and clinical management data (Aims 1 and 3)

A research coordinator at the clinical site will review the EHR clinical notes for each participant enrolled in Aims 1 and 3 and record in REDCap the dates and reasons for acute care use (e.g., feeding tube placement, emergency room visits, hospitalizations) and clinical management data (e.g., pain medication changes, dates and reasons for supportive care visits, such as nutrition, speech/swallow therapy, physical therapy). The REDCap database design consists of standardized checkboxes for uniform data across all sites.

Data collection and storage for exploratory aim

Patient participant study questionnaires administered through EHR

Using information from Aims 1, 2, and 3, the study team will build a prototype of integrating the ePVA web application with the EHR system (i.e., Epic) using FHIR technology. The patient participants will complete the ePVA in Epic before their oncology visit. The ePVA patient responses will be stored in each study site’s Epic Caché database and evaluated for completeness, accuracy, and validity. Research coordinators will sit with patients while they complete the ePVA, asking them to “think aloud” about the problems they encounter to capture the patient participants’ problem-solving process [56]. These “think-aloud” sessions will be audio-recorded and stored on secure password-protected servers, transcribed verbatim, and checked for accuracy. The research assistants will also observe the clinicians’ workflow and record their observations in field notes to inform the refinement of the ePVA prototype.

Patient and clinician surveys: usability of the ePVA integration with electronic health records

The usability of the ePVA integration with the EHR will be measured by the Post-Study System Usability Questionnaire [50–53], a Likert scale developed at IBM to evaluate user satisfaction with a technology system. Patient participants will answer the questionnaire once after completing the ePVA, and clinicians will answer the survey on study closure to assess their overall perception of the system’s usability.

Data analysis

Intended use of study data

Data collected under this protocol will be used to test the effectiveness of the ePVA as a digital patient-reported monitoring system for patients with HNC in real-world settings and identify implementation strategies that optimize the effectiveness of the ePVA in diverse rural and urban settings.

Sample size justification: Aims 1, 2, and 3

The study is designed to detect effects for three primary endpoints: social function, senses of taste and smell, and swallowing (see Table 2). We expect 135 participants per arm (N = 270). This sample size has 85% power to detect effects using a T-test for primary endpoints even after accounting for 10%–20% loss to follow-up that was experienced in preliminary studies. A two-sided type I error rate was set to 1.67% using a Bonferroni correction to account for the three primary endpoints (5% typical alpha divided by 3 = 1.67%).

Table 2.

Sample Size Calculation for the Primary Outcomes Based on Pilot Study Effect Sizes [29]

Sample size justification: exploratory aim

Approximately, 40 patients (10 from each site) and 30 clinicians (i.e., ~8 from each site) will assess the usability of the ePVA integrated with the EHR. We will judge the exploratory aim successful if (a) ≥ 70% of patients and clinicians agree they are satisfied with the ePVA integrated with EHR and (b) 7 of 10 patients at each participating institution complete the ePVA successfully (i.e., patient responses are visible on the clinicians’ same day visit EHR (i.e., Epic) screen). For patients within sites, we will have a 5% type I error rate (one-sided) and 88% power if the population completion rate expected in future studies is 40% (discouraging) versus 80% (encouraging) using a binomial exact test. This aim is exploratory, so we do not adjust for multiple hypothesis testing.

Randomization

For Aim 1, patients will be randomly assigned at the individual level to usual care or ePVA + usual care between days 1 and 5 after RT has started to avoid screen failures. This parallel randomization scheme will be generated by the biostatistician using a permuted block design and uploaded to REDCap to ensure that the study team is blinded to the sequence until the study arm is assigned. The blocks are stratified according to study site and HNC treatment to ensure balance among important potential confounders. Each study site’s research coordinator conducts the randomization after confirming the treatment plan with the site PI and records the arm in the REDCap database (Table 3).

Table 3.

Randomization Scheme

^aStudy site and HNC Treatment may affect study outcomes. Therefore, each study site will randomize participants to usual care or ePVA+usual care based on whether participants undergo radiation therapy with chemotherapy versus radiation therapy without chemotherapy and surgery before starting radiation therapy versus no surgery before starting radiation therapy

Blinding

Blinding to the intervention at the patient and clinician level is not possible as both patients and clinicians will know that the patient answered the ePVA. The research staff who abstract data from the EHR (e.g., for acute care services use) are not blinded to the participants’ group allocation. To ensure the accuracy of data abstraction, the PI or project manager will verify the intra-rater reliability and data entry accuracy during site visits. For intra-rater reliability, UTHealth Houston coordinating center team members will randomly select up to 20% of data entered within the past 6 months and record if the data entry is accurate. If < 90% of the data entries are accurate, the research team member will correct the erroneous data and continue randomly selecting up to 20% of data entries within the past 6 months until verifying > 90% of data entries are accurate. The data analyst is blinded to the study assignment.

Aim 1 (quantitative aim)

The three primary endpoints will be the EORTC QLQ-C30 subscale for social function and the EORTC QLQ-H&N35 scales for senses (i.e., taste and smell) and swallowing. The primary measurement time will be at 4 weeks after RT is completed. T-tests for the primary analysis between arms will be used. Summary statistics of potentially confounding variables between randomization arms will also be examined to ensure that those confounders are balanced between arms. If confounders are found to differ between arms at the p < 0.10 level, the confounders will be controlled by including them as covariates in linear regressions of outcomes. Similar analyses are planned for secondary EORTC QLQ-C30, QLQ-H&N35, and acute care use endpoints.

Intention to treat will be the primary analysis paradigm. In secondary analyses, we will perform as-treated (i.e., categorizing participants by treatment actually received) and per-protocol (i.e., including only those who complied with protocol-defined requirements for their arm) analyses.

We will examine longitudinal trends of symptom scales using random-effects regressions with random intercepts for study participants. We will further evaluate multilevel models that account for clustering within each site via an additional random effect. Study arm (i.e., usual care vs. ePVA + usual care) and study wave (i.e., data collection time point) will be included as categorical variables (i.e., binary indicators). Interactions between study arm and study wave will be investigated. We will examine generalized mixed linear models of acute care use with appropriate link and family functions; offset terms will account for follow-up time. Care management data will be included as covariates.

Missing data

For the primary analysis, we will perform a complete case analysis to exclude those with missing data. While we do not anticipate much missing data, if there is substantial missing data, the multiple imputation methods of Raghunathan and colleagues with 25 imputed datasets as a sensitivity analysis will be used [57]. Summary statistics of those who do and do not have missing data will also be examined to characterize missingness patterns. For participants who withdraw from the study, researchers will use data collected before withdrawal.

Aim 2 (qualitative aim)

Transcripts of patients’ and clinicians’ interviews will be analyzed using directed content analysis [58] in ATLAS.ti. Directed content analysis is a deductive approach using prior research or theory to determine the initial coding categories. The initial coding will be guided by findings from our analysis of qualitative data collected during our preliminary research, such as on patient resiliency and clinician focus on diagnosis and treatment. The initial codes will be grouped into categories. Concepts in one or more categories will be linked to create theoretical relationships [59]. New codes will be applied to data that cannot be coded with the previous codes established during our preliminary research. An auditor will review and examine the initial coding to ensure its consistency. The final code key will be reapplied to all transcripts [58]. Detailed notes during the coding process, including memos of any personal bias, will be maintained throughout the analysis. Any disagreements among the study team will be discussed until consensus is achieved. To provide an audit trail and ensure the dependability of coding decisions, decisions and rationale for the decisions will be documented in ATLAS.ti. The credibility of the analysis will be evaluated through member checks to confirm findings with participants and the HNC clinical study team [60].

Aim 3 (integrative aim)

The goals of integrating quantitative and qualitative data are to (a) interpret and compare data for complementarity, convergence, and divergence of the study findings; (b) identify the implementation strategies that optimize the reach, adoption, effectiveness, and maintenance of the ePVA; and (c) uncover explanatory processes, mechanistic pathways, and disparities in digital access. The quantitative and qualitative data will be integrated and analyzed using a joint display [61]. The rows will represent the RE-AIM framework dimensions that guided the study design. The columns will display the quantitative and qualitative data side by side. The analysis of the joint display will include an appraisal of concordant and divergent findings. An analysis of the concordant findings will consist of data queries regarding any shared bias between methods that resulted in concordance. An analysis of divergent findings will query the data for any lapse in scientific rigor or potential sources of bias in study components that may have led to the divergence. If needed, additional data will be collected to examine convergence or divergence. If the divergence of the study findings remains unexplained, additional hypotheses regarding the nature of the divergence will be formed to guide future follow-up studies.

Exploratory aim: explore the usability of the integration of the ePVA with EHR systems

The study team will explore the usability of the ePVA integrated with the EHR using quantitative and qualitative approaches. The Post-Study System Usability Questionnaire [50] will be assessed using means and frequencies of patient and clinician responses. The patient participants’ comments during “think-aloud” sessions will be evaluated using directed content analysis [58] to identify key categories related to usability. The study team, in collaboration with each site’s information technology team, will use data from the main categories to iteratively refine the ePVA prototype to improve the user experience.

Data linkage and management

The research team will collect only the minimum level of data necessary to conduct the study analysis. A secure linking log composed of identified patient data and the participant’s study ID is maintained separately from the patient-level data collected at data time points in the REDCap database housed at UTHealth Houston. For the statistical analysis, the study ID will be used to merge patient-level data collected at baseline, during RT, and then 4, 12, 24, and 48 weeks after the completion of RT. Access to the linking log is restricted to research personnel who have been approved by the institutional review board. The data and linking log will be stored in REDCap for 6 years and will be destroyed by erasing (deleting) the digital files and shredding the paper files.

Ethical considerations

The protocol is approved by the UTHealth Houston Committee for the Protection of Human Subjects (HSC-SN-23–0784 version 1.9). Each institution follows procedures approved by the UTHealth Houston Committee for the Protection of Human Subjects as the single primary IRB for the study. An executed data use agreement is established as part of the subaward process with each institution.

A Data and Safety Monitoring Committee (DSMC) independent of sponsor and competing interests is responsible for safeguarding the interests of trial participants, assessing the safety and efficacy of interventions during the trial, and monitoring the clinical trial’s overall conduct, including mitigating any conflicts of interest among study team members. The DSMC is composed of three members who have expertise in HNC rehabilitation, statistics, and the conduct of clinical trials in the HNC population. The DSMC meets with the PI, project manager, and study biostatistician every 6 months. The PI communicates protocol modifications and DSMC recommendations to site PIs at monthly co-investigator meetings. Amendments are submitted to the single IRB, relevant changes are communicated to participants (with re-consent if necessary), and major modifications are updated on trial registries (e.g., ClinicalTrials.gov). A copy of the DSMC plan may be obtained from the corresponding author.

Dissemination

In compliance with the National Institutes of Health’s Public Access Policy, final peer-reviewed journal manuscripts will be submitted to the digital archive PubMed Central upon acceptance for publication to ensure that the public can access the published results. In compliance with the National Institutes of Health’s 2023 Data Management and Sharing Policy, data will be shared with researchers after approval by the principal investigator and co-investigators, and a data use agreement is fully executed.

Discussion

Patients undergoing HNC treatment have one of the highest symptom burdens among all cancer types treated in the outpatient setting [6, 62, 63]. Despite symptom prevalence, up to 50% of HNC patients’ symptoms go undetected during busy oncology visits and are undocumented in the EHR [64–66]. The use of PROs during cancer care is associated with improved symptom control, decreased acute care use, and longer survival [11–13]. Yet, a closer investigation of the literature finds the effects of PROs on cancer outcomes vary depending on implementation strategies, sometimes yielding small effect sizes and limited user engagement [17, 20, 21, 67]. Reasons for variations in PRO implementation include lack of implementation standards, inequities in user access, and limited integration with EHR systems [19–24], which create barriers to widespread implementation of PROs. This study aims to identify implementation strategies that optimize the effectiveness of the ePVA as a digital PRO monitoring system using longitudinal mixed-methods in a hybrid type I study design [30] and the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, Maintenance) [36–40] to improve HNC outcomes in real-world settings.

An important reason for variations in PRO implementation is the presence of barriers to access and use of technology [68, 69]. The majority of Americans (91%) own smartphones. Yet, statistics show a lower percentage of ownership among Americans who earn less than US $30,000 (84%), are 65 and older (79%), have a high school education or less (85%), or live in a rural area (88%) [70]. Other data show that even with access, the use of digital health technology differs among age and race categories, which has become known as the “digital divide.” To advance knowledge in facilitators and barriers to access and use of technology, we are recruiting diverse urban and rural populations and using quantitative and qualitative approaches on the effect of technology on the ePVA’s reach, adoption, effectiveness, and maintenance in routine cancer care.

Advances in technology, such as FHIR and the SMART standards [71–73], have created a unique and powerful opportunity to build low-cost, equitable access to PROs in HNC. These advances enable the seamless integration of digital technology across multiple applications and end-user devices through tokens (assigned numbers), thereby eliminating the need to share protected health information across health systems. These technological advances will provide the basis for integrating the ePVA with EHR systems in routine health care to advance a cancer care delivery system that delivers high-quality healthcare for rural and urban populations [74].

Strengths and limitations

A main strength of this research is the collaboration among four NCI-designated comprehensive cancer centers serving rural and urban populations to identify implementation strategies that optimize the effect of the ePVA. Another strength is that all sites use the Epic electronic health record system, facilitating the integration of the ePVA with EHR systems. A challenge in designing this study was selecting the randomization scheme for Aim 1. Randomization at the individual level is more likely to result in a balanced population across study groups; however, it can also lead to the same clinician treating patients in both arms. The study team explored the use of a stepped wedge cluster-randomization scheme but found it also has multiple limitations. For example, after the site transitions from control to intervention, the same clinician will treat patients in both study groups given the study’s longitudinal design, which is similar to randomization at the individual level. Furthermore, the small number of sites that serve as clusters (i.e., NYU, FCCC, KUCC, BCM) may result in treatment effects overwhelmed by site-specific effects, leading to a potential imbalance of population characteristics across arms. Thus, the team decided on randomization at the individual level as the best option for Aim 1. Another limitation is that the intervention has not been fully integrated into regular practice flows during the clinical trial but is being tested as part of the exploratory aim for future real-world implementation.

Conclusion

The overarching goal of this research is to advance cancer care by identifying implementation standards for effective, widespread use of the ePVA that apply to all patient-reported outcomes in cancer care. Thus, this study is an opportunity to engage patients and clinicians in optimizing the ePVA to help patients with HNC live longer, healthier lives.

Trial status

Protocol version: Version 1.9 and date 10/8/25. The study recruitment began June 25, 2024, and is estimated to end March 31, 2028. One-hundred and two participants have enrolled in the study.

Abbreviations

ePVA

New York University Electronic Patient Visit Assessment (ePVA)©

HNC

Head and neck cancer

HRQoL

Health-related quality of life

PRO

Patient-reported outcome

CMS

Centers for Medicare & Medicaid Services

EHR

Electronic health record systems

NYU

New York University

NCI

National Cancer Institute

UTHealth Houston

The University of Texas Health Science Center at Houston

REDCap

Research Electronic Data Capture

KUCC

The University of Kansas Cancer Center

FCCC

Fox Chase Cancer Center

BCM

Baylor College of Medicine

EORTC

European Organization for Research and Treatment of Cancer

FHIR

Fast Healthcare Interoperability Resources

NCCN

National Comprehensive Cancer Network

SMART

Substitutable Medical Applications, Reusable Technologies

SPIRIT

Standard Protocol Items: Recommendations for Interventional Trials

RE-AIM

Reach, Effectiveness, Adoption, Implementation, and Maintenance

DSMC

Data and Safety Monitoring Committee

US

United States

RT

Radiation therapy

Authors’ contributions

JVC is principal investigator; she conceived the study and led the proposal and protocol development. AB, DS, KSH, ZL, SBJ, VJM, CEL, JRB, and BLE, GVT, and MT contributed to study design and development of the protocol. All authors read and approved the final manuscript.

Funding

Janet H. Van Cleave, Abraham A. Brody, Dena Schulman-Green, Zujun Li, Stephen B. Johnson, Vincent J. Major, Christopher E. Lominska, Jessica Bauman, Alexander N. Hanania, and Brian Egleston have received NIH/NCI R01CA282149. Funding source address: National Cancer Institute at the National Institutes of Health, 9609 Medical Center Drive, Rockville, MD 20850, USA. The funding source had no role in the design of this study and will not have any role during execution, analyses, interpretation of the data, or decision to submit results.

Data availability

In compliance with the National Institutes of Health 2023 Data Management and Sharing Policy, data will be shared with researchers after approval by the principal investigator and co-investigators, and a data use agreement is fully executed.

Declarations

Ethics approval and consent to participate

The protocol is approved by The University of Texas Health Science Center at Houston Committee for the Protection of Human Subjects (HSC-SN-23-0784 version 1.9, dated 10/8/25). The initial IRB approval and model consent to participate are submitted with this manuscript.

Consent for publication

Not applicable.

Competing interests

The copyrights in the ePVA are owned by New York University (NYU). If NYU receives income from licensing the ePVA, then authors Janet H. Van Cleave and Kenneth S. Hu may receive a portion of the license income. The other authors declare that they have no competing interests.