A PRO-cision Medicine Methods Toolkit to Address the Challenges of Personalizing Cancer Care Using Patient-Reported Outcomes

Abstract

Patients are increasingly being asked to complete standardized, validated questionnaires with regard to their symptoms, functioning, and well-being [ie, patient-reported outcomes (PROs)] as part of routine care. These PROs can be used to inform patients’ care and management, which we refer to as “PRO-cision Medicine.” For PRO-cision Medicine to be most effective, clinicians and patients need to be able to understand what the PRO scores mean and how to act on the PRO results. The papers in this supplement to Medical Care describe various methods that have been used to address these issues. Specifically, the supplement includes 14 papers: 6 describe different methods for interpreting PROs and 8 describe how different PRO systems have addressed interpreting PRO scores and/or acting on PRO results. As such, this “Methods Toolkit” can inform clinicians and researchers aiming to implement routine PRO reporting into clinical practice by providing methodological fundamentals and real-world examples to promote personalized patient care.

BACKGROUND AND RATIONALE

When treating cancer patients, their functioning and well-being is of utmost importance. How cancer patients feel and function are best measured using patient-reported outcomes (PROs), specifically, patients’ direct reports on outcomes such as symptoms, functioning, and health-related quality of life.^1,2 Traditionally, PROs have been measured in clinical trials and other research studies to compare the impact of different treatment options from the patient’s perspective.^3–7 More recently, there has been interest in using PROs to monitor the progress of individual patients and inform their management.^8–13 This approach of using PROs involves having a patient complete a standardized questionnaire, providing that patient’s results to his/her clinical team, and using the PRO data—along with other clinical information (laboratory values, imaging studies)—to manage that patient’s care.

The use of PROs in clinical practice has demonstrated benefits, including promoting patient-clinician communication,^14–17 assisting with problem detection,^11–13,17 influencing management,¹⁶ and improving outcomes, such as symptom control, health-related quality of life, and functioning.^14,18–20 Recent studies have shown a survival benefit associated with the intervention.^21,22 PRO-cision Medicine²³ is the concept of using patients’ own reports of their functioning and well-being to personalize their care.

Although evidence supports the effectiveness of PRO-cision Medicine, there are barriers to broad implementation. To promote the use of PROs, a panel of experts at a recent meeting prioritized: (1) helping patients and clinicians interpret the PRO scores, and (2) helping patients and clinicians act on the PRO results.²⁴ This supplement addresses those key issues.

Issue #1: Interpreting PRO Scores

The issues associated with interpreting PRO scores have been well documented. Problems with interpretation stem from the multitude of PRO instruments and lack of standardization in how these PRO instruments are scored and scaled. On some PRO instruments, higher scores represent better outcomes; on other PRO instruments, higher scores indicate worse outcomes; and on still other PRO instruments, higher scores indicate “more” of the outcome (such that higher scores are better for function domains but worse for symptoms). Beyond the directionality of scoring, PRO instrument scaling also differs, further complicating interpretation. For example, some instruments’ scores are normed to a general population average of 50, whereas others are linearly transformed to a 0–100 scale, and others are simply summed. Both patients and clinicians have reported that this variability is confusing, with quotes such as, “Of course I have no idea if this is a good score or a bad score,” “Until you address the scaling issues it isn’t very useful…,” “A score of say, 50, meant one thing on one graph and something different on another one, which I thought was strange,” and at the most basic level, “I don’t know what the numbers mean.”²⁵

To address the issue of score interpretation, we previously conducted a 3-part research study to identify the formats for displaying PRO data that were most accurately interpreted and rated as clearest.^26–28 Following the completion of this research project, our Stakeholder Advisory Board suggested that we had created an evidence base sufficient to inform development of best-practice recommendations for the display of PRO data to promote understanding and use. We thus undertook a modified-Delphi consensus process to develop stakeholder-driven, evidence-based standards for presenting PROs in practice.²⁹

The recommendations that emerged from the Delphi consensus panel include the value of: (1) providing descriptive y-axis labels (eg, none, mild, moderate, severe); (2) indicating scores that are possibly concerning; and (3) providing scores for reference populations. However, the Delphi panel also noted that the information needed to implement these recommendations is not available for many PRO instruments. For example, while the descriptive labels along the y-axis help add meaning to the numeric scores, the score ranges that would be associated with each category are unknown for most PRO instruments. Similarly, many PRO instruments do not have established threshold values to indicate which scores may be concerning. There is little comparison data from reference populations, and questions remain with regard to the appropriate comparators.

Issue #2: Acting on PRO Results

In addition to the challenges associated with understanding what scores mean, there is also the question of how issues identified by PRO measures should be addressed. That is, once a score has been identified as possibly concerning, what should clinicians (and patients) do to respond? As noted above, PRO measures may assess a number of domains ranging from symptoms, functional status, well-being, and health-related quality of life. Although clinicians are trained to manage specific somatic symptoms, based in part on established evidence for effective interventions, issues such as social function (ability to participate in work and hobbies) receive less attention. Clinicians may be reluctant to use PRO assessments if they do not feel comfortable responding to the results. In addition, with the increasing focus on patient empowerment and self-management, patients may also want guidance to self-manage issues identified by PRO measures.

Evidence suggests that providing guidance on how to respond to PRO-identified issues can contribute to the successful use of PROs in clinical practice. One of the earliest randomized controlled trials evaluating a PRO intervention provided problem-specific resource and management suggestions and found significant improvements in identifying problems, better management of them, and improved patient outcomes in the intervention group.³⁰ Later studies suggested that providing such recommendations for managing PRO issues is an important component of the intervention’s success.³¹ A number of groups have undertaken projects to develop suggestions for responding to PRO issues.^32,33 However, each of these projects has used different methods, and guidance is needed for others seeking to implement PROs in routine patient care.

To address the issues with interpreting PRO scores and acting on PRO results, we invited a series of papers from experts with experience developing methods for interpreting and/or acting on PROs in clinical practice. This resulting supplement can serve as a “toolkit” of different methods to which researchers and clinicians can refer when implementing PRO use in routine care.

METHODS

To develop the supplement, the principal investigators (PIs: C.S., M.B., A.W.) first convened a Steering Group with expertise in methods for interpreting and/or acting on PRO scores (see the Acknowledgments section). During a conference call and follow-up emails, the PIs and Steering Group identified additional experts who could also contribute papers to the supplement. Experts on the final list were invited to participate and contribute a paper to the series. An honorarium was provided to each Steering Group member and to each author team, with the author team determining the appropriate division of the funds.

To ensure coordination across papers in the supplement, the PIs held a kick-off conference call with the paper authors in December 2017. Subsequently, each author team submitted a draft abstract in January 2018, and then circulated its draft paper in May 2018. In June 2018, the PIs, Steering Group, and at least one representative from each author team met in person in Baltimore, MD. During this meeting, each paper was presented and discussed by the group as whole. Revised papers were reviewed by the PIs before being submitted to Medical Care for external peer review.

Notably, this supplement is designed to be a “methods toolkit,” with the individual papers describing different approaches to aid PRO score interpretation and/or develop guidance for acting on PRO results. Because of this focus on methods, many of the papers do not include data or results, although in some cases previous publications are referenced that have applied the methods. Although the supplement focuses on cancer, some papers also describe methods used in other conditions that would be applicable to cancer. Similarly, many of the methods applied in the cancer context could also be applied in other disease areas. Finally, the supplement focuses on PRO data in individual patient care, with the goal to describe methods that can help with interpretation/action at the individual patient level. Below, we provide an overview of the papers in the supplement.

A PRO-cision MEDICINE METHODS TOOLKIT

The supplement includes 14 papers: 6 describe different methods for interpreting PROs, and 8 describe how different PRO systems have addressed issues related to interpreting PRO scores and/or acting on the PRO results.

The 6 papers describing methods to aid interpretation cover a range of approaches. The first paper, by Shi et al,³⁴ describes quantitative methods for identifying cutpoints on questionnaires, while the second paper, by Cook et al,³⁵ describes a qualitative approach, “bookmarking,” for setting cutpoints. Another innovative approach for PRO score interpretation, as described by Browne et al,³⁶ is using modern psychometric methods, in this case the Rasch model, to benchmark the performance associated with different scores on a scale—and how this information could be used both to inform patient’s treatment decision-making, as well as to monitor their progress. As such, these 3 papers explain different methods, or “tools,” that can be used to determine what score ranges represent different levels, such as none, mild, moderate, or severe. The fourth paper, by Jensen et al,³⁷ describes how reference values can aid interpretation of score meaning—and reviews methods for collecting and applying the data for these reference values. Another approach for aiding interpretation is discussion between patients and providers, what the Oliver et al³⁸ paper refers to as “feedforward” of the patient’s perspective to his/her clinical team to inform that particular patient’s management. They also describe “feedback” population analytics based on data aggregated across patients to inform treatment decision-making, predictive modeling, and patient-centered care. Three case studies describe PRO systems at different levels of maturity in implementing feedforward and feedback. Finally, the paper by King et al³⁹ provides a thoughtful review of different metrics commonly used for group-level PRO interpretation [eg, minimal important difference, definitions of responders (ie, improved, stable, worsened)], and the extent to which they can aid in the interpretation of individual patient PRO scores.

Eight papers describe different systems that have been developed to collect PROs for patient monitoring and management and the methods they have used to aid interpretation of PRO scores and/or guide action on the PRO results. Notably, the different systems vary in terms of their characteristics, including their purposes for data collection; the types of patients who are targeted for PRO completion; which PRO questionnaires are used; how, when, and where the PROs are collected; and how, when, and where the PROs are reported. The characteristics of these PRO systems, as well as the 3 case studies described by Oliver et al,³⁸ are shown in Table 1. This diversity across the PRO systems’ designs, as well as the different approaches the systems have used to aid interpretation of the PRO scores and/or act on the PRO results, suggest a range of options for addressing these issues.

Table 1.

Characteristics of PRO Systems Described in PRO-cision Medicine Methods Toolkit Paper Series

PRO System	Purpose for Data Collection	Patients Targeted for Assessment	Mode of Administration	Frequency of Collection (Tied to Visits?)	PRO Questionnaires Collected	How/When/Where/to Whom Results are Reported
PatientViewpoint40	Monitoring	Breast and prostate cancer patients undergoing treatment, with oncologist visits at least monthly	Web: invited to complete at home, but opportunity to complete in-clinic if not performed before visit	Tied to visit frequency	Varied, but options included the EORTC QLQ-C30 and BR23, Supportive Care Needs Survey-Short Form-34, various PROMIS domains, Expanded Prostate Cancer Index Composite-Short Form	Patients see graphic display of results after completion Clinicians can access graphic displays via PatientViewpoint, or plain text numeric tables in electronic health record
KLIK41	Monitoring, screening, research	KLIK started in pediatric oncology and pediatric rheumatology care, but is now used in > 100 different patient groups, including children, parents/caregivers, and adult patients	Web: data are primarily collected at home via KLIK on a computer or mobile device; data can also be collected in the clinic through a tablet or computer	Tied to visits, but frequency varies, with for example, some generic HRQOL measures being collected every 3 months and other, more psychosocial measures, collected annually	7 categories of questionnaires are available (1)Generic HRQOL (2)Disease-specific HRQOL (3)Daily functioning (4)Cognitive functioning (5)Symptoms (6)Psychological screening (7)Transition	Results are immediately available to patients and clinicians on the KLIK website. Results are shown in several ways (1) literal representation of the individual items, (2) summary scores, (3) graphically
eRAPID (electronic patient self-Reporting of Adverse events: Patient Information and aDvice)42	Monitoring and supporting management of symptom toxicity during systemic cancer treatment	Colorectal, gynecological, and breast cancer patients during chemotherapy (with internet access at home)	Web: patients are encouraged to complete PROs at home but assessments can be completed on any internet enabled device	Patients are encouraged to complete weekly for 18 weeks during standard chemotherapy regimens to support clinical reviews before each cycle; however, patients can complete PROs more often if they choose, or if they want advice for particular symptoms	Approximately 12–15 items covering the most common toxicities for standard chemotherapy regimens; based on CTCAE grading (items developed locally); option for drop-down and free text to add and rate the severity of other issues	Following PRO completion eRAPID provides patients with immediate severity tailored advice to guide self-management or hospital contact. Patients can access and review their symptom reports and advice at any time (including longitudinal graphical representation of symptom data) Oncologists and oncology nurses can view PRO symptom severity scores in the electronic patient record in tabular or graphical form and can view scores over different time periods or single completions; they are encouraged to review and utilize PRO data during consultations with patients Email notifications can be sent to specified staff to alert them of clinically severe symptom scores
Symptom Care at Home43	Symptom screening, monitoring, clinical management, treatment decision-making	Patients undergoing active treatment (chemotherapy) and advanced disease/end-of-life, and their caregivers	Telephone: IVR, with plans to move to IVR with mobile app and web-based options	Daily (not tied to clinic visit frequency)	Single item 0–10 ratings tailored to population 11 symptoms for patients on chemotherapy 11 symptoms for patients at end-of-life 5 indicators for family caregiver well-being	Daily, immediately post call data are available to assigned provider(s), for example, oncology nurse practitioner, oncologist, oncology nurse, hospice nurse Alerts can go to multiple people
Beacon44	Monitoring and treatment decision-making	Patients with various cancers, as well as patients with diabetes and palliative care patients	In clinic: via iPad	At each routine medical visit	Single item linear analog self-assessment scales	Clinicians and patients see results via an iPad just before the medical visit Clinician prints and scans the results into the patient’s medical record
Ontario Provincial PRO Collection45	Screening	All patients attending for an appointment at a regional cancer center (including many partner hospital sites)	In clinic: via kiosks	Tied to visit frequency	Edmonton Symptom Assessment Scale Eastern Cooperative Oncology Group patient-reported functional status Expanded Prostate Cancer Index Composite-Clinical Practice	By the clinical care team on the day of the visit
PROMPT-Care (Patient Reported Outcomes for Personalised Treatment and Care)46	Screening, monitoring, toxicity management, treatment decision making, survivorship, palliative care	All cancer patients (currently only available in English, though pilot testing on a small scale in Arabic and Chinese)	Web: via an emailed link; can also be completed in the clinic on a tablet	Monthly (not tied to clinic visit frequency)	Distress Thermometer and Checklist; Edmonton Symptom Assessment Scale; Supportive Care Needs Survey-Screening Tool 9	PROMPT-Care clinical reports are available in the patient’s electronic medical record; an email alert is sent to a designated cancer center email address whenever a patient’s score on any assessment item has breached the predetermined score on 2 consecutive assessments; the patient receives an email immediately after completing an assessment, with links to self-management resources tailored to their “above-threshold” scores
Stover et al47: Three studies collecting PROs at visits or between visits	In all 3 studies: monitoring/toxicity management/palliative care during active systemic therapy (chemotherapy or oral biologics)	In all 3 studies enrolled patients are adults with advanced cancer	In all 3 studies PROs collected electronically For the single-arm intervention study: collected in clinic For the 2 randomized trials: collected at home between visits via the web, or patients in 1 trial had the option of IVR	For the single-arm intervention study: collected at 2 visits over a 3-month period For the 2 randomized trials: filled out weekly between visits	For the single-arm intervention study: PRO-CTCAE, Edmonton Symptom Assessment Scale-r, Functional Assessment of Cancer Therapy—sexual function, chemotherapy-induced peripheral neuropathy For 1 randomized trial, items written by authors covering common symptoms and adverse events For 1 randomized trial, PRO-CTCAE, plus questions on physical function, eating/drinking, and falls	For the single-arm intervention study: PRO responses were available immediately on a clinician-facing dashboard that was viewed by the care provider on a tablet or computer during the clinical visit; in addition to discussing results together during the visit, clinicians could view static symptom management algorithms and patients received automatically generated advice for symptoms exceeding a threshold For the 2 randomized trials: PRO responses triggered automated email alerts to nurses for frequent, severe, or worsening symptoms in the last 7 days; nurses contacted patients within 72 hours to assess symptom grade and make treatment recommendations; clinical decision support was included with every alert
Concord Multiple Sclerosis (MS) specialty care program38	Screening, monitoring, treatment decision-making, quality improvement and research	Multiple sclerosis outpatients attending clinic visits	In clinic: on paper	Once at baseline, then monitoring quarterly, tied to clinic visits	CES-D, GAD-7, PROMIS Fatigue-MS, PROMIS Cognitive Abilities, Primary Care-Post-Traumatic Stress Disorder, Mood Disorder Questionnaire, Adult Self-Report Scale for Attention Deficit and Hyperactivity Disorder, Epworth, selfefficacy, MS pain effects scale, multiple sclerosis patientdetermined disease steps (PDDS) Most are multiple items, some (PDSS) are single item	Results reported in electronic health record, visible to clinician, clinician must print out and share with patient during the clinic visit Results shared at time of encounter
Dartmouth-Hitchcock Spine Center38	Screening, monitoring, treatment decision-making, population health, quality improvement and research	Spine and multiple ambulatory care populations, including primary care	Electronic: internet portal connected to electronic health record or tablet computer at clinic before clinic visits	Electronic queuing based on visit type and other variables; electronic notification and reminders sent via patient portal; can be tied to clinic visits or independent of visits	Short Form-36, Oswestry Disability Index, audit, review of systems, Health Habits, Bothersomeness Index, work disability, expectations Most are multiple item instruments	Clinician and patient can review results at clinic visits and display trends over time Results used at time of encounter Predictive risk calculator used to aid treatment decision making
Swedish Rheumatology Quality Register38	Screening, monitoring, treatment decision-making, population health, improvement and research, self-management between visits, quality improvement and research	Rheumatoid arthritis outpatients	Web: via internet portal connected to electronic health record both from home between visits and at time of visit	Prompted previsit or at visit and optional entry between visits for self-management	EuroQol-5D and Health Assessment Questionnaire Both use multiple items	Clinician and patient can view results online and at clinic visits Results used at time of encounter to guide care intensity via “open-tight” clinic model (lower intensity wellness-based trajectory for those with a profile suggesting a stable course, higher intensity for those with PRO profile suggesting high complexity, or higher acuity profile)

CES-D indicates Center for Epidemiologic Studies-Depression; CTCAE, Common Terminology Criteria for Adverse Events; EORTC QLQ-C30 and BR23, European Organization for the Research and Treatment of Cancer Quality of Life Questionnaire-Core 30 and 23-item breast cancer module; GAD-7, Generalized Anxiety Disorder-7; HRQOL, health-related quality of life; IVR, interactive voice response; PRO, patient-reported outcome; PRO-CTCAE, patient-reported outcome version of the Common Terminology Criteria for Adverse Events; PROMIS, Patient-Reported Outcomes Measurement Information System.

First, Blackford et al⁴⁰ describe how the PatientViewpoint web system determines which scores to highlight as possibly concerning, either in absolute terms or a significant worsening, as well as how guidance was developed for acting on these possibly concerning scores. In addition, the paper describes methodologic research the authors pursued to use needs assessments and patients’ reports of their most bothersome issues to identify possibly concerning scores. Haverman et al⁴¹ describe the KLIK system developed for children with chronic disease and their parents in the Netherlands, which is now also being used in adults. They highlight how KLIK scores can be reported in various formats, including literal representation of the individual items, sum scores, and different graphic displays. From the United Kingdom, Absolom et al⁴² describe the electronic patient self-Reporting of Adverse events: Patient Information and aDvice (eRAPID) system, which focuses specifically on chemotherapy adverse events and provides immediate severity tailored feedback for self-management or advice to contact the provider. The paper describes the methods they used to enhance patients’ and clinicians’ engagement with the symptom reports to promote their use in clinical practice. The “Symptom Care at Home” PRO system in the Mooney et al⁴³ paper is unique in that it uses daily phone symptom collection. The interactive voice response system provides automated self-management coaching for patients and alerts to the oncology team about poorly controlled symptoms. In contrast to the other PRO systems, which primarily collect multi-item PROs, Zahrieh et al⁴⁴ discuss their Beacon system’s focus on single item data collection to highlight the patient’ s single biggest concern. In an example of population-wide PRO data collection, Barbera et al⁴⁵ describe how Ontario has implemented routine PRO reporting across the province over the past decade. They report the various pragmatic methods and evolving approaches used to make the PRO system more useful in practice. In an example from Australia, Girgis et al⁴⁶ describe the Patient-Reported Outcomes for Personalised Treatment and Care eHealth system (PROMPT-Care), which is fully integrated in hospital electronic oncology information systems. The PROMPT-Care developers emphasized selection of PRO measures that are brief and clinically actionable, and providing care pathways that fit in the clinical workflow to address identified issues. Finally, Stover et al⁴⁷ describe different methods for alerting clinicians about concerning symptom questionnaire responses from different research studies evaluating the use of PROs in clinical practice. The percentage of PRO reports that trigger an alert varied widely, depending on the guidelines used to determine which scores would generate an alert, as well as due to the different contexts in which the research studies were conducted.

SUMMARY

The use of PROs in clinical practice has the potential to promote patient-centered, personalized care. However, until issues related to interpreting PRO scores and acting on PRO results are addressed, the impact of PROs in practice will be limited. Together, the 14 papers in this supplement provide a range of options or “tools” that clinicians and researchers can apply to the use of PROs in clinical practice. The methodologic papers suggest alternative approaches, which can be used alone or in combination to aid PRO score interpretation. In addition, the descriptions of the different methods used by the various PRO systems illustrate the diversity of approaches that can be used, both for interpreting PRO scores and for acting on the PRO results. As such, this supplement “Methods Toolkit” can inform clinicians and researchers aiming to implement routine PRO reporting by providing methodological fundamentals and real-world examples of how to interpret PRO scores and act on PRO results.

Patients value routine PRO data collection more highly when their clinicians actually use the data to inform their care.²⁵ Implementing systems that arm patients and clinicians with the tools to understand the PRO scores, and then act on them, promotes the effective use of PROs in clinical practice. Seeing their data used in their care will encourage patients to complete the questionnaires. More complete data has the potential to not only improve patients’ own care, but also provides more complete datasets for secondary analyses in patient-centered and comparative effectiveness research.⁴⁸ In all these ways, we can promote personalized patient care based on patients’ reports of their functioning and well-being —“PRO-cision Medicine.”