Introduction
Patient reported outcomes (PROs) capture how patients feel, function, live their lives, and survive.1,2 They include outcomes such as symptoms, functional status, well-being, and health-related quality of life reported directly by the patient without interpretation by a clinician or anyone else.3,4 Standardized and validated PRO measures (PROMs) are used to assess PROs. Increasingly, PROMs are being collected routinely to monitor how individual patients are doing and to inform their care.5–9
Vulnerable and underserved populations have been defined as populations who have been systematically denied a full opportunity to participate in aspects of economic, social, and civic life.10,11 Evidence indicates that using PROs in clinical practice can improve patient care and outcomes and may be even more valuable for vulnerable and underserved populations in the United States. For example, some studies have shown that PROs improved health-related quality of life, reduced emergency department visits, and increased survival for patients generally, and improved these outcomes even more for patients who were older, less educated, or Black.12,13 Similarly, another study reported that patients from minority racial groups with less education or less computer usage reported more benefits from completing PRO questionnaires in their routine care.14
However, numerous, multi-level barriers prevent broad implementation of PROs in clinical practice,15 and evidence suggests that vulnerable and underserved populations face unique challenges that hinder PRO use. These challenges include patient-level factors such as lack of access to technology, low technology literacy, cultural differences, language barriers, privacy concerns, and stigma.16–20 Therefore, there is a need for unique resources and tailored support to promote PRO use in the care of vulnerable and underserved populations in the United States. Specifically, there is a need to determine how best to address the full range of barriers.
To address these challenges, we convened an expert advisory group of stakeholders from various backgrounds and perspectives. We engaged the Advisory Group to identify the key barriers and potential solutions for PRO implementation in vulnerable and underserved cancer populations. Based on the input of the Advisory Group, we identified a framework of solutions that align with established implementation processes.
Below, we describe the process and outcomes of the Advisory Group. The framework of solutions can be used by various stakeholders who are interested in improving the use of PROs in cancer care for vulnerable and underserved populations, including funders, clinicians, researchers, and health system administrators.
Advisory Group Formation
The Advisory Group was organized by the PROTEUS Consortium (Patient-Reported Outcomes Tools: Engaging Users & Stakeholders21) in collaboration with Pfizer. PROTEUS is a consortium of over 50 patient, clinician, research, health system, industry, government, policy, and regulatory groups from the United States and internationally. PROTEUS helps navigate the use of PROs in clinical trials and clinical practice.
We solicited expressions-of-interest for participation in the Advisory Group from experts with interest in and/or experience using PROs in vulnerable and underserved populations in the United States. We advertised widely through PROTEUS Consortium members and other relevant organizations to identify a broad group of members representing the needs of vulnerable and underserved populations. Interested individuals completed a brief application regarding their background and expertise, as well as why they wanted to serve on the Advisory Group. For the purposes of the Advisory Group, we defined vulnerable and underserved populations as: “Populations sharing a particular characteristic, as well as geographic communities, that have been systematically denied a full opportunity to participate in aspects of economic, social, and civic life, including: Black/African American populations and other persons of color; American Indian / Alaskan Native and other Indigenous populations; people whose first language is not English; older adults; members of religious minorities; LGBTQ+ populations; individuals with disabilities; people who live in rural areas; populations impacted by persistent poverty or inequality; people with limited health literacy and/or numeracy; and populations who may face barriers that make it difficult to get health coverage and basic health care services for other reasons.10,11
Three members of the PROTEUS leadership team (CS, MB, AS) reviewed a total of 86 expressions-of-interest from which 24 individuals were invited and able to serve on the Advisory Group. Individuals were selected based on their reach to underserved and vulnerable populations; expertise in PROs; and their role in the healthcare system. They were also selected to ensure that the Advisory Group was comprised of multi-disciplinary experts who had experience working across the categories of vulnerable and underserved populations (e.g., rural populations, racial minorities). The final Advisory Group included 29 members (see Acknowledgments): the PROTEUS leadership (i.e., the 4 named authors), the 24 invited individuals, and a Pfizer representative. The selected individuals served a broad range of vulnerable and underserved populations and had an interest in or experience using PROs. Multiple disciplines were represented, including clinicians, researchers, pharmacists, and program managers. Participants came from healthcare systems across the country, including 12 states.
Advisory Group Meeting: Process and Outcomes
The Advisory Group met in-person for a full-day meeting in May 2023 to identify barriers and solutions to address the unique PRO implementation considerations in institutions caring for vulnerable and underserved cancer populations. The Advisory Group focused specifically on cancer care in the United States and concentrated on the use of individual cancer patient PROs to inform monitoring and management and not on other uses of PROs in the clinical setting (such as use of PROMs at the aggregate level for quality evaluations or pay-for-performance initiatives).
We used a structured process that included brainstorming, break-out groups, and prioritization exercises to enumerate and prioritize key barriers and develop solutions for overcoming them. Specifically, the meeting was organized in four parts (Figure 1): (1) providing key background information to inform subsequent discussion; (2) identifying barriers and challenges to implementing PROs in vulnerable and underserved populations; (3) prioritizing the top barriers; and (4) developing solutions to address the prioritized barriers.
Figure 1.
Process to engage the Advisory Group
Part 1. Providing key background information to the Advisory Group
The first part of the Advisory Group meeting provided background information on key topics to promote relevant and productive discussions. PROTEUS team leaders presented background on PROTEUS, the rationale for establishing the Advisory Group, and the goal of developing solutions to advance the use of PROs in vulnerable and underserved populations. In addition, three orienting presentations highlighted a patient’s experience overcoming cancer and the importance of partnering with patients; the current literature on implementing PROs in routine clinical care for diverse and underrepresented patients in the U.S.;17 and the known barriers to integrating PROs in clinical care generally.15 Table 1 summarizes these barriers, which span multiple levels – patient, clinician, administrative, and system – and range from factors such as patients’ lacking access to technology, clinicians’ time and resource constraints, administrative concerns around costs, and systems’ technical capacities.
Table 1.
Known multi-level barriers to implementing PROs in clinical care*
| Level | Barrier | Example |
|---|---|---|
| Patient | Accessibility | Not all patients have access to or choose to use technology |
| Lack of buy-in | Patients may not complete PROs if it is unclear how these measures are relevant to them and their care | |
| Burden reporting PROs | PRO collection systems, particularly those that are web-based or electronic, can be complex to navigate | |
| Clinician | Technological and logistical challenges | Challenges such as difficulty finding the PRO system, and remembering when to check the system |
| Disruption to workflow | Integrating PROs into clinical care in any systematic way is likely to disrupt preexisting workflows | |
| Time and resource constraints | Competing with collection of other quality metric data that are required and tied to reimbursement | |
| Uncertainty with integrating data into clinical care | Clinicians may be hesitant to use PRO data if they are uncomfortable in how to interpret the information | |
| Administrative | Cost | Costs associated with training clinical teams and staff and having staff on hand to support PRO completion |
| Lack of shared values/purpose | Stakeholder groups may disagree on the purpose and value of collecting and integrating PROs into clinical care | |
| Uncertainty in assessing impact of PRO collection on quality | Need to review and evaluate new PRO systems performance may require additional resources and new approaches | |
| Legal/regulatory concerns | Can raise issues such as who is responsible for medical decisions based on PRO data with risks for patient health | |
| System | No “one-size fits-all” approach | PRO systems must be made based on the unique goals, resources, and context of healthcare setting |
| Technical capacity | Not all PRO systems may be integrated easily with existing electronic medical records | |
| Lack of in-house expertise | Lack of expertise to inform the design, implementation, or evaluation of PRO systems |
Adapted from Chapter 2 of the PROTEUS Guide to Implementing Patient-Reported Outcomes in Clinical Practice: A Synthesis of Resources15
Part 2. Identifying barriers to PRO implementation in vulnerable and underserved cancer populations
The second part of the meeting focused on identifying key barriers and challenges to implementing PROs in vulnerable and underserved populations. To accomplish this, the Advisory Group members brainstormed in 4 break-out groups, each with 7–8 participants. Topics discussed in the break-out groups included (1) additional barriers that were not covered by the multi-level barriers presented in the first part of the meeting; (2) their top five barriers and challenges for using PROs in vulnerable and underserved populations; and (3) any specific barriers that applied to specific vulnerable and underserved populations (e.g., rural patients, hearing impaired patients). Each small group included members with a mix of expertise and perspectives, and a PROTEUS leader facilitated the discussion.
After the break-out groups finished their tasks, we reconvened as a large group. One representative from each break-out group reported on the top barriers prioritized by their group and any other key points. The PROTEUS leaders reviewed the barriers reported by the break-out groups and generated a list of 17 unique barriers for using PROs in vulnerable and underserved populations (Table 2). The barriers apply at the patient, clinician, administrator, and system level. Some barriers also expanded on those that were already known. For example, in addition to the resources needed to train clinicians and staff on how to use PRO systems and how to use PRO data for making clinical decisions, the Advisory Group described how PRO use in vulnerable and underserved populations requires resources for systems and clinicians to meet patients’ culture, language, literacy, and numeracy needs.
Table 2.
Barriers to the implementation of PROs in vulnerable and underserved cancer populations
| Barrier | Number of votes* |
|---|---|
| Systems’ commitment among competing priorities as demonstrated through resources and staffing | 17 |
| Systems and clinicians’ ability to address patients’ culture, language, literacy and numeracy | 12 |
| Investment required to collect data among vulnerable populations | 9 |
| Patient-level technology capability (broadband access, willingness/capability to use) | 8 |
| Clinician resistance / lacking appreciation of value | 7 |
| Patient not seeing value if not seeing PROs used | 5 |
| Trust and respect | 4 |
| Availability of PRO measures in multiple language and literacy levels | 4 |
| Lack of reimbursement to systems trickles to clinicians and then patients | 4 |
| Concerns about technology security and data privacy | 3 |
| Actionability (not asking about things that can’t be addressed) | 3 |
| PROs not patient-centered because not patient informed | 3 |
| Sustainability in a dynamic environment | 2 |
| Responsibility to act on data | 1 |
| Inequitable impact of PRO data benefits | 1 |
| Lack of engagement of these populations in healthcare generally | 1 |
| Time and transportation | 0 |
To prioritize the 17 barriers, which were listed on flip-chart pages, Advisory Group members participated via dot voting22,23 where they were given four sticker dots and asked to put their dots next to the barriers they thought were most important and actionable.
Part 3. Prioritizing barriers to PRO implementation in vulnerable and underserved cancer populations
To prioritize the 17 barriers, which were listed on flip-chart pages, Advisory Group members participated via dot voting22,23 where they were given four sticker dots and asked to put their dots next to the barriers they thought were most important and actionable. The four barriers identified by the Advisory Group members as most important and actionable (Table 2) were: (1) health system-level commitment among competing priorities as demonstrated through resources and staffing (17 votes); (2) systems’ and clinicians’ ability to address patients’ needs related to culture, language, literacy, and numeracy (12 votes); (3) investment required to collect data among vulnerable populations (9 votes); and (4) patient-level technology capability (broadband access, willingness/capability to use) (8 votes). These top barriers spanned patient, clinician, administrator, and system levels.
Part 4. Developing solutions to address prioritized barriers
The last part of the meeting focused on developing solutions to address the top four prioritized barriers. First, the Advisory Group members met in their break-out groups and brainstormed relevant solutions that ranged from small to large monetary investments. Then the Group developed a list of unique solutions based on the breakout group discussions. Finally, each Advisory Group member voted on their top solution. Overall, the Advisory Group identified 47 different potential solutions to address the top barriers.
Following the meeting, the PROTEUS leaders reviewed and categorized the solutions into categories and sub-categories (Table 3). The solutions spanned the following four categories that each consisted of two to three sub-categories: (1) research; (2) education and engagement; (3) information technology or technological resources; and (4) incentives, mandates, and marketing. Nearly half of all solutions (48%) fell into the education and engagement category, 21% in information technology or technological resources, 15% in research, and 15% in incentives, mandates, and marketing.
Table 3.
Framework of solutions that address prioritized barriers
| Solutions category | Solution sub-categories | Solutions |
|---|---|---|
| Research | Identify value of PROs (monetarily and otherwise) | Collecting data to demonstrate cost savings, improvement or changes in administrative or quality measures |
| Return on investment via engagement, retention, outcomes | ||
| Show value of between-visit monitoring | ||
| Understand the resources and methods that are most efficacious | ||
| Make PROMs relevant and applicable | Know who your population is | |
| Identify and address gaps in use of PROMs by race, ethnicity, language, literacy, numeracy, and technology to ensure PROMs are suited for everyone | ||
| Explore what patients think the PROs mean and how patients think their PRO data are being used | ||
| Education and engagement | Create and promote shared resource repository/toolkit/best practices for different populations | Summary of the PROTEUS-Practice Guide15 |
| Leadership education slide set | ||
| One-page “business case for PROs” | ||
| Descriptions for everyone’s role (e.g., navigators, receptionists) | ||
| Disseminate existing tools and guidance | ||
| Directory/repository of resources that are translated | ||
| Toolkit with information to address knowledge gaps | ||
| Audio cards that explain PROM program | ||
| Videos/animations on: How to set up a system; How to engage with the portal | ||
| Culturally tailored in-house videos to encourage willingness to use PROM program | ||
| Partner with patient advocates, community organizations, community health workers | Identify and work with champions | |
| Engage community advocates/lay navigators to help with PROM completion | ||
| Symposium/community building/learning collaboratives among community hospitals (matching/partnering); both one-time and longitudinal options | ||
| Partner with patient advocacy groups (including Patient Advisory Councils) and community organizations at a local level to promote PRO system design and use and make sure patient advocacy groups see benefits of efforts | ||
| Establish a network that: (1) supports local development and implementation of community health worker or patient navigator model; and (2) has a centralized coordination eConsult resource for the community health workers and/or patient navigators | ||
| Develop and provide training and educational opportunities | Webinar teaching/training | |
| Promoting better understanding of humility (think of everyone as a collective with differing needs) – avoiding “otherness” | ||
| How to Do PROMs in vulnerable and underserved | ||
| Standardized training/certification/teach-the-teachers curriculum | ||
| Co-creation of communication and sensitivity training with patient, caregiver, and/or community organizations | ||
| Create educational videos to communicate the value of PROs | ||
| Engage post-docs and fellows to work on this | ||
| Information technology or technological resources | Advance technological resources | Address burden through, for example, computer adaptive tests |
| Apps and low-cost innovative information technology solutions / novel data collection technologies | ||
| Interpretation tools/interpreters | ||
| Low-tech tools and devices to reach folks who cannot use portals at an institutional level | ||
| Loan out hot spots and technology | ||
| Connect patients to technologies and services available in their community (e.g., free cell phones) | ||
| Promote data sharing | Partner with electronic medical record systems used in community settings | |
| Un-siloed, accelerated data sharing (like what occurred with COVID) | ||
| Repository and sharing strategy of electronic medical record system users to promote shareability across institutions | ||
| Learn from Health Resources and Services Administration Uniform Data System | ||
| Incentives, mandates, and marketing | Provide incentives | Paying patients and patient advocates for time they spend advocating for PROs |
| Insurance coverage where PRO monitoring is included with prescribed treatment plan | ||
| Incentivize PROM completion (gift cards, reduced copays, other ideas from patients) | ||
| Consider PRO use as part of ranking and credentialing processes | ||
| Create mandates | Mandates from institutions, journals, grants, professional organizations | |
| Utilize marketing | PRO program as a marketing branding tool | |
| Incorporate patient preferences and priorities throughout |
Discussion
This Commentary reports the results of an Advisory Group formed to improve the use of PROs in vulnerable and underserved cancer populations in the U.S. Specifically, the Advisory Group of multidisciplinary experts identified and prioritized the barriers to PRO implementation and proposed potential solutions. The main outcome of the meeting was a framework of solutions. This project demonstrates “designing for implementation,” which means that the solutions are tailored to the needs, preferences, and capacities of the intended users and settings.24 This approach increases the likelihood of successful adoption and sustainability. The findings have several implications for practice, research, education, and policy on the use of PROs in vulnerable and underserved cancer populations.
First, the findings highlight the need for a comprehensive and multi-level approach to address the complex and interrelated barriers that hinder PRO use in this context. The solutions identified by the Advisory Group cover various actions needed to enhance adoption, implementation, and sustainability of PROs in vulnerable and underserved populations, such as developing stakeholder interrelationships and collaborations to facilitate PRO integration, and training and educating stakeholders on the benefits and methods of PRO use. The framework of solutions aligns with established implementation solutions25,26 based on the Expert Recommendations for Implementing Change (ERIC) project. The ERIC project’s solutions represent a refined compilation of methods or techniques to enhance the adoption, implementation, and sustainability of interventions. That the solutions produced by the Advisory Group align with the ERIC recommendations provides additional evidence that they are well-suited to improve the use of PROs.
Second, the findings suggest that there is a gap between the current evidence and practice on PRO use in vulnerable and underserved populations. Although some studies have shown that using PROs in clinical practice can improve patient care and outcomes for these populations,12–14 there may still be a lack of widespread implementation of PROs in institutions that serve them. Moreover, there is a need for more research on the effectiveness, feasibility, acceptability, and sustainability of the proposed solutions for overcoming the barriers to PRO use in this context. Therefore, future research should focus on testing and evaluating the priority solutions identified by the Advisory Group, as well as exploring other innovative and tailored solutions that may enhance PRO use in cancer care for vulnerable and underserved populations.
Third, the findings indicate that there is a need for more awareness and advocacy on the importance and value of using PROs in cancer care for vulnerable and underserved populations. The Advisory Group members expressed their enthusiasm and commitment to using PROs in their work, but also acknowledged the challenges and difficulties they face in doing so. They emphasized the need for more support and resources from funders, policymakers, administrators, and leaders to promote PRO use in this context. Therefore, future efforts should aim to disseminate and implement the framework of solutions developed by the Advisory Group, as well as to engage and educate various stakeholders on the benefits and methods of using PROs in cancer care for vulnerable and underserved populations.
There are limitations to make note of when considering these barriers and solutions. For instance, the Advisory Group only had a few cancer patient participants from vulnerable and underserved populations, who are the primary beneficiaries and users of PROs in cancer care. Therefore, future work should seek to further engage and involve patients from vulnerable and underserved populations in the design, implementation, and evaluation of PRO use in cancer care. Another limitation is that the proposed solutions may require short-term funding that may not be sustained in the long run, or that may depend on external sources that may not be reliable or consistent. Therefore, the findings may not account for the potential trade-offs or challenges of implementing and maintaining the solutions in resource-limited settings. This point relates to one of the proposed solutions, which emphasizes the need to assess the value and return on investment of PROs. This approach could also include understanding the scalability of the proposed solutions, and identifying alternative or complementary solutions that could optimize the use of PROs in cancer care for vulnerable and underserved populations within constrained resources and ensure long-term sustainability. A final limitation is that the work does not address other aspects or uses of PROs in cancer care for vulnerable and underserved populations, such as using PROs at the aggregate level for quality improvement or pay-for-performance initiatives or using PROs for shared decision making or patient empowerment. Therefore, the findings may not capture the full potential or impact of PRO use in this context.
Our expert Advisory Group and its meeting had several strengths that contributed to the development of solutions for improving PRO use in cancer care for vulnerable and underserved populations. First, we used multiple channels to recruit a large and diverse pool of stakeholders from various fields and sectors related to PROs and cancer care, ensuring a representative and balanced sample of participants. Second, we invited and engaged stakeholders from different groups and perspectives, such as patients, caregivers, clinicians, researchers, advocates, policy makers, and health system administrators, capturing a wide range of experiences and opinions that reflect the complexity and diversity of the problem. Third, we designed and implemented a structured and interactive meeting process with various methods and techniques to facilitate the discussion and participation of the stakeholders. We believe that these strengths enabled us to generate a valuable and inclusive list of solutions for using PROs in cancer care for vulnerable and underserved populations.
Conclusion
The Advisory Group identified and prioritized barriers to implementing PROs in U.S. institutions providing cancer care to vulnerable and underserved populations and developed customized solutions to address these barriers. The developed solutions provide a framework to promote improved PRO implementation in the care of vulnerable and underserved cancer populations. Actively disseminating these solutions is a critical next step to their implementation.
Acknowledgments
The PROTEUS Consortium is funded through unrestricted support from Pfizer. The Advisory Group for Patient-Reported Outcome Implementation in Vulnerable and Underserved Populations is a collaboration between PROTEUS and Pfizer Global Medical Grants. We are grateful to the Advisory Group members for their valuable time and insights, which enabled us to develop this framework of solutions. The Advisory Group members are: Andrea Pusic, MD (PROVE Center at Brigham Health); Andrew Vickers, PhD, Memorial Sloan Kettering Cancer Center); Angelique Richardson, MD, PhD (University of California in San Diego); Anne Marie Rainey, MSN RN CHC CPHQ (Clearview Cancer Institute); Bethany Sibbitt, PharmD (Kettering Health and Cedarville University School of Pharmacy); Betty Hamilton, MD (Cleveland Clinic); Billie Baldwin, MA, MSW, LCSW-C (MedStar Franklin Square Medical Center); Cardinale Smith, MD, PhD (Mount Sinai Health System); Emilie Smith, PharmD, BCOP (Pfizer); Eric Boakye, PhD (Henry Ford Health System); Esteban Barreto, PhD, MA (Massachusetts General Hospital; Harvard Medical School); Kriti Mittal, MD, MS (University of Massachusetts); Larissa Meyer, MD, MPH, FACOG, FACS (The University of Texas MD Anderson Cancer Center); Lauren Shapiro, MD, MS (University of California - San Francisco); Leah Owens, DNP, RN, BMTCN, OCN (Tennessee Oncology); Maimah Karmo, BA (Tigerlily Foundation); Mary Vargo, MD (MetroHealth Medical Center; Case Western Reserve University); Nadine Jackson McCleary, MD, MPH (Dana-Farber Cancer Institute); Rachel Peterson, PhD, NCSP (Kennedy Krieger Institute, Johns Hopkins University School of Medicine); Rebekah Angove, PhD (Patient Advocate Foundation); Ronald Chen, MD, MPH (University of Kansas Medical Center); Tara Kaufmann, MD, MSCE (University of Texas at Austin); Theresa Gillespie, PhD, MA, BSN, FAAN (Emory University School of Medicine and Winship Cancer Institute); Vikas Mehta, MD, MPH (Montefiore Medical Center/Albert Einstein College of Medicine); William Lee, RPh, MPA, FASCP (Carilion Clinic).
References
- 1.FDA 2020. Clinical Outcome Assessment (COA): Frequently Asked Questions. Available at: https://www.fda.gov/about-fda/clinical-outcome-assessment-coa-frequently-asked-questions#COADefinition. Last accessed: February 8, 2022.
- 2.Haywood KL, de Wit M, Staniszewska S, Morel T, Salek S. Chapter 9 (Part II) Developing Patient-Reported and Relevant Outcome Measures. In: Facey K, Hansen HP, Single ANV (eds). Patient Involvement in Health Technology Assessment. Adis Springer Nature. 2017. DOI 10.1007/978-981-10-4068-9. [DOI] [Google Scholar]
- 3.FDA. Guidance for industry. Patient-reported outcomes measures: use in medical product development to support labeling claims. 2009: 65132–65133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Acquadro C, Berzon R, Dubois D et al. Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value in Health 2003; 6 (5): 522–531. [DOI] [PubMed] [Google Scholar]
- 5.Greenhalgh J The applications of PROs in clinical practice: what are they, do they work, and why? Quality of Life Research 2009; 18 (1): 115–123. [DOI] [PubMed] [Google Scholar]
- 6.Snyder CF, Aaronson NK, Choucair AK et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Quality of Life Research 2012; 21 (8): 1305–1314. [DOI] [PubMed] [Google Scholar]
- 7.Snyder CF, Brundage MD, Rivera YM et al. A PRO-cision Medicine Methods Toolkit to Address the Challenges of Personalizing Cancer Care Using Patient-Reported Outcomes: Introduction to the Supplement. Medical Care 2019; 57: S2–S7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Stover AM, Tompkins SC, Hammelef K et al. Using Stakeholder Engagement to Overcome Barriers to Implementing Patient-reported Outcomes (PROs) in Cancer Care Delivery. Medical Care 2019; 57 (S92): S99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lavallee DC, Chenok KE, Love RM et al. Incorporating Patient-Reported Outcomes Into Health Care To Engage Patients And Enhance Care. Health Affairs 2016; 35 (4): 575–582. [DOI] [PubMed] [Google Scholar]
- 10.U.S. Department of Health and Human Services (HHS) and the January 20, 2021, Executive Order on Advancing Racial Equity and Support for Underserved Communities Through the Federal Government Available at: https://www.whitehouse.gov/briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government/
- 11.“Serving Vulnerable and Underserved Populations.” U.S. Centers for Medicare & Medicaid Services Available at: https://marketplace.cms.gov/technical-assistance-resources/training-materials/vulnerable-and-underserved-populations.pdf [Google Scholar]
- 12.Basch E, Deal AM, Kris MG, et al. : Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 34:557–565, 2016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jensen RE, Snyder CF: PRO-cision Medicine: Personalizing Patient Care Using Patient-Reported Outcomes. J Clin Oncol 34:527–529, 2016 [DOI] [PubMed] [Google Scholar]
- 14.Snyder CF, Herman JM, White SM, et al. : When using patient-reported outcomes in clinical practice, the measure matters: a randomized controlled trial. J Oncol Pract 10:e299–306, 2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.The PROTEUS Guide to Implementing Patient-reported Outcomes in Clinical Practice A synthesis of resources. Prepared by Crossnohere N, Brundage M, Snyder C, and the Advisory Group, 2023. Available at: https://theproteusconsortium.org/proteus-practice/proteus-practice-guide/.
- 16.Meirte J, Hellemans N, Anthonissen M, et al. : Benefits and Disadvantages of Electronic Patient-reported Outcome Measures: Systematic Review. JMIR Perioper Med 3:e15588, 2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hyland CJ, Guo R, Dhawan R, et al. : Implementing patient-reported outcomes in routine clinical care for diverse and underrepresented patients in the United States. J Patient Rep Outcomes 6:20, 2022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Basch E, Stover AM, Schrag D, et al. : Clinical Utility and User Perceptions of a Digital System for Electronic Patient-Reported Symptom Monitoring During Routine Cancer Care: Findings From the PRO-TECT Trial. JCO Clinical Cancer Informatics 947–957, 2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Griffin J et al. Demographic differences in patient reported outcome (PROS) response rates of cancer-related symptoms by patient portal, in-clinic tablet and integrated voice response. 29th Annual Conference of the International Society for Quality of Life Research. Qual Life Res 31:9–169, 2022 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Aiyegbusi OL: Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Qual Life Res 29:325–333, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.The PROTEUS Consortium. Available at: https://theproteusconsortium.org/
- 22.Brown T, Wyatt J. Design thinking for social innovation. Development Outreach. 2010;12(1):29–43. [Google Scholar]
- 23.Bridges J, Lynch T, Schuster A et al. A review of paper-based advance care planning aids. BMC Palliat Care 17, 54 (2018). 10.1186/s12904-018-0298-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Brownson RC., Colditz GA, Proctor EK Proctor (eds), Dissemination and Implementation Research in Health: Translating Science to Practice, 2nd edn (New York, 2017; online edn, Oxford Academic, 23 Nov. 2017), 10.1093/oso/9780190683214.001.0001, accessed 12 Sept. 2023. [DOI] [Google Scholar]
- 25.Waltz TJ, Powell BJ, Matthieu MM et al. Use of concept mapping to characterize relationships among implementation solutions and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Sci 10, 109 (2015). 10.1186/s13012-015-0295-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Powell BJ, Waltz TJ, Chinman MJ et al. A refined compilation of implementation solutions: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Sci 10, 21 (2015). 10.1186/s13012-015-0209-1 [DOI] [PMC free article] [PubMed] [Google Scholar]