PATIENT-REPORTED OUTCOMES (PROs): PUTTING THE PATIENT PERSPECTIVE IN PATIENT-CENTERED OUTCOMES RESEARCH

Abstract

Patient-centered outcomes research (PCOR) aims to improve care quality and patient outcomes by providing information that patients, clinicians, and family members need regarding treatment alternatives, and emphasizing patient input to inform the research process. PCOR capitalizes on available data sources and generates new evidence to provide timely and relevant information and can be conducted using prospective data collection, disease registries, electronic medical records, aggregated results from prior research, and administrative claims. Given PCOR’s emphasis on the patient perspective, methods to incorporate patient-reported outcomes (PROs) are critical. PROs are defined by the U.S. Food & Drug Administration as “Any report coming directly from patients… about a health condition and its treatment.” However, PROs have not routinely been collected in a way that facilitates their use in PCOR. Electronic medical records, disease registries, and administrative data have only rarely collected, or been linked to, PROs. Recent technological developments facilitate the electronic collection of PROs and linkage of PRO data, offering new opportunities for putting the patient perspective in PCOR. This paper describes the importance of and methods for using PROs for PCOR. We (1) define PROs; (2) identify how PROs can be used in PCOR, and the critical role of electronic data methods for facilitating the use of PRO data in PCOR; (3) outline the challenges and key unanswered questions that need to be addressed for the routine use of PROs in PCOR; and (4) discuss policy and research interventions to accelerate the integration of PROs with clinical data.

INTRODUCTION

Comparative effectiveness research (CER) “inform[s] health-care decisions by providing evidence on the effectiveness, benefits, and harms of different treatment options.”¹ Patient-centered outcomes research (PCOR) stresses the importance of research “informed by the perspectives, interests and values of patients” throughout the research process.² Both CER and PCOR capitalize on primary data collection and secondary data sources to provide timely and relevant information to decision-makers. While PCOR and CER comprise largely overlapping sets of activities – both aim to provide practical evidence to support real-world decision-making – PCOR particularly emphasizes the patient perspective. Given the central role of the patient perspective in PCOR, methods to incorporate patient-reported outcomes (PROs) are critical to its successful conduct. However, PROs have not routinely been collected in a way that facilitates their use in PCOR. Electronic medical records, disease registries, and administrative data have only rarely collected, or been linked to, PROs. Recent technological developments that facilitate the electronic collection of PROs and the linkage of PRO data with other clinical data offer new opportunities for putting the patient perspective in PCOR.

In this paper, we describe the importance of and methods for using PROs for PCOR by (1) defining PROs; (2) identifying how PROs can be used in PCOR; (3) outlining the challenges and key unanswered questions for routine use of PROs in PCOR; and (4) discussing policy and research interventions to accelerate the integration of PROs with clinical data.

DEFINING PROs

PROs are defined by the Food & Drug Administration (FDA) and National Quality Forum (NQF) as “… a report that comes directly from the patient (i.e., study subject) about the status of a patient’s health condition without amendment or interpretation of the patient’s response by a clinician or anyone else.”^3–4 Considering the meaning of each word in “PRO” is instructive. First, while the “P” in PRO stands for “patient,” it could refer to any person reporting on his/her own health condition or treatment. For example, the definition above includes “study subjects,” and could also apply to healthy individuals. Use of the term “patient” rather than “person” conveys the health-related nature of PRO. Second, “reported” is a critical aspect of the PRO definition. As noted above, the standard is for PROs reported directly by the patient, “without amendment or interpretation by …anyone else.”^3–4 However, when patients cannot report for themselves (e.g., due to disability), proxy respondents can be used in an attempt to obtain the patient’s perspective. PROs are distinct from other patient outcomes such as physiological measures (e.g., hemoglobin A1c), clinician-reported measures (e.g., global impressions), and caregiver-reported measures.⁵ Finally, “outcome” in PRO is interpreted broadly to reflect a variety of information reported directly by the patient, including health-related quality of life, functional status, symptoms, and treatment adherence.⁵ The definition of PRO put forward by Academy Health, “health data that are provided by the patient through a formal and tested system of reporting,” does not explicitly emphasize the importance of reporting directly by a patient. Given the central role of direct reporting by the patient in most PRO definitions and the broad application of the FDA and NQF definition, this paper adopts the FDA and NQF definition of PROs.

IMPORTANCE OF ELECTRONIC DATA METHODS

The diverse use of PROs can be classified in three applications: population surveillance, individual patient-clinician interactions, and research studies.⁶ When combined with other clinical data, the PRO data resulting from these uses can be used for PCOR. Figure 1 displays how PRO data from these different applications can be linked with other clinical data sources (e.g., electronic medical records, disease registries) to create PCOR datasets that can inform patient choice, provider recommendations, and policy-maker decisions, and serve as key prognostic covariates. However, linking PRO data with other clinical data can be challenging,⁷ and electronic data methods play a critical role in facilitating this linkage of PRO data with other clinical information. Below, we describe how electronic data methods can be used to integrate PROs in PCOR, and then provide examples for each of the applications.

Figure 1.

The applications of PROs in PCOR build off their use in population surveillance, in research studies, and in clinical encounters for individual patients. These sources of PRO data can be linked with clinical data from a variety of sources to create PCOR datasets. The use of PROs in PCOR produces data to inform patients, providers, and policy-makers. In addition, PRO data may serve as a key prognostic covariate to address methodologic issues associated with observational data.

Electronic Data Methods to Integrate PROs in PCOR

Across all levels of PRO application, collection of PRO data is difficult – albeit not impossible – without electronic data methods. Electronic data capture improves feasibility of collection, decreases burden, and enables sophisticated survey administration.⁸ Take, for example, paper-and-pen PRO data collection. The paper form has to be given in-person or mailed, the patient must complete the questionnaire, and then return it (in-person or by mail). The data have to be entered into a database. Standards need to be established for addressing improper questionnaire completion (e.g., more than one response to the same question), and there is no practical way to recover missing data. Finally, the questionnaires must be scored before conducting the analyses of interest. This process is expensive, labor intensive, error-inducing, and involves a considerable time lag between questionnaire administration and result reporting.

In contrast, with electronic data collection, particularly via the Internet, the process is much simpler.^8–9 Patients can access questionnaires anyplace with Internet access, allowing flexibility to complete questionnaires both inside and outside of clinic visits. An estimated 77% of the US population had Internet access in 2010.¹⁰ As patients submit their completed questionnaires, the data are automatically entered into a database. Respondents can receive real-time, automatic alerts that provide guidance for improperly completed questions (e.g., not allowing more than one answer, reminders regarding missing responses). The survey scores can be calculated and reported immediately. Automatic skip patterns can be built-in, or more sophisticated techniques such as computer-adaptive testing (CAT) can be implemented. CAT uses the respondent’s prior answers to inform the next item administered from a large pool of items on the same topic, thereby providing reliable assessments more efficiently than fixed-item questionnaires.¹¹ For example, on a physical function questionnaire, if a patient responds that s/he can walk several blocks “none of the time,” the next question administered would not ask if s/he can run one mile. It would select a question more suitable for someone with lower physical functioning, such as performing self-care activities. Thus, electronic PRO collection improves data quality and collection efficiency across a wide range of patients, facilitating the use of PROs in PCOR.

Population Surveillance

At the population level, clinical information is collected using health surveys, disease registries, and administrative claims. Collection of PROs at the population level produces various opportunities to incorporate PROs in PCOR. For example, the United Kingdom’s National Health Service collects PROs before and after certain elective surgical procedures: hip replacement, knee replacement, varicose vein surgery, and hernia repair.^12–13 Given the large scale of this effort, the UK now has PRO data from hundreds of thousands of surgical patients. These PRO data can be used to compare care quality across providers or by patients to choose a provider. Additionally, the PRO data can help patients understand the likely impact of the procedure on patient-centered outcomes. For these applications, important issues such as risk adjustment need to be addressed.

Disease registries also offer opportunities to capture population-based clinical information, but the data in these registries generally come from medical record abstractions, making inclusion of standardized PROs challenging because PRO data are not routinely available in medical records. Similarly, administrative claims data offer a valuable resource for PCOR but currently lack a mechanism for including PRO data.

With increasing use of electronic data capture, new possibilities are emerging for collecting PROs for large populations. For example, the Commission on Cancer has developed an online platform for hospital-based cancer registries to report on their cancer cases.¹⁴ The Commission is also pilot-testing a Rapid Quality Reporting System and plans to expand it to include PROs. The linkage of the PRO data with the registry data will enable the use of PROs in a range of PCOR studies. Another example of using PROs at the population level is the Washington Heights/Inwood Informatics Infrastructure for Community-Centered Comparative Effectiveness Research (WICER) initiative.¹⁵ WICER is sampling the population of 5 New York ZIP codes annually for 3 years and collecting data on PROs, demographics, vital statistics, and neighborhood information. Survey data are linked with medical records in a data warehouse, thereby providing a rich data resource.

Individual Patient-Clinician Interactions

On the individual level, PROs can be used clinically to screen for conditions (e.g., depression), monitor patient progress over time, and in decision aids.^16–18 With the increasing use of electronic medical records (EMRs), there are more opportunities to link PROs with the EMR. Eighty-one percent of hospitals, and 41 percent of office-based physicians intended to take advantage of federal incentive payments for adoption and meaningful use of certified EMR technology in 2011–12.¹⁹ Thus, PRO data collected to inform individual patient’s care can be aggregated across patients and analyzed in conjunction with the other clinical information available in the EMR, creating a practical opportunity for PCOR.

Online web tools designed to collect PROs and link them with the EMR can promote the use of PROs for patient care and for PCOR. For example, Patient View point (www.PatientViewpoint.org) allows clinicians to assign PROs for patients to complete, just as they would with any laboratory test.^20–22 Patients receive an email asking them to complete the assigned PROs, which they can do anywhere with Internet access. Patient View point tracks scores over time in reports available through the website and in the patient’s EMR. While the primary purpose of this data collection is to improve individual patient care, the PRO data can be aggregated across patients, combined with clinical information from the cancer registry or medical records, and used to compare treatments or evaluate the quality of care.

Research Studies

PROs are most commonly used as outcome measures in clinical trials and observational studies.^23–32 For example, the Surgical Care and Outcomes Assessment Program (SCOAP) Comparative Effectiveness Research Translation Network (CERTN) is a prospective cohort study of intermittent claudication. ³³ The investigators have developed a Survey Center, which allows collection of data using multiple modes including web-based, to collect PRO and other data outside of the index hospitalization so that pre- and post-hospitalization information can be collected.

However, PRO data are generally used to answer a given study’s specific questions and are not readily accessible for secondary analyses. The ability to link these data with other clinical information across studies would improve their usefulness for PCOR. To that end, a draft effectiveness guidance document from the Center for Medical Technology Policy recommends that PROs be included in all late-phase and CER studies in oncology, and that these studies measure a core set of 12 symptoms to facilitate cross-study comparison.³⁴

Illustrating the critical role of electronic data capture in the collection of PRO data for research studies, the Patient-Reported Outcomes Measurement Information System (PROMIS) has developed an online data collection platform in conjunction with the PROMIS questionnaire content. PROMIS is “a system of highly reliable, precise measures of patient–reported health status for physical, mental, and social well–being.”^35–36 The PROMIS Assessment Center is a free, online research tool that allows investigators to administer both PROMIS and other PRO questionnaires to patients electronically. PROMIS measures were developed based on modern psychometric concepts, and Assessment Center was designed to make these features accessible to researchers. It allows for customized domain-specific questionnaires and multiple administration methods (both static and computer-adaptive). Domain scores are comparable regardless of length or administration method. This system is already in wide-spread use with over 1000 registered researchers.³⁷

An Application for Further Exploration: Use of PROs as Prognostic Variables

In addition to using PROs in the three applications described above, there has been recent discussion of using PROs as important stratification variables or covariates. Because many PCOR studies use observational designs, without patient randomization, confounding is an important methodologic issue. PROs may be useful as balancing variables given that there is substantial evidence that PROs are important prognostic variables³⁸ and improve the accuracy of survival predictions.³⁹ While there are methodologic concerns with this application of PROs in PCOR, given the unique prognostic contribution of PROs, this potential application is worthy of further exploration.

KEY CHALLENGES AND UNANSWERED QUESTIONS

A number of practical issues need to be confronted to integrate PROs in PCOR. Challenges include the proprietary nature of PRO measures, governance related to setting standards, selection of tools for clinical use, confidentiality issues, and coordination among organizations in all of the above.

While many PROs are available without charge in the public domain and others are available on the basis of “copy left” (i.e., copyrighted so that they can be made freely available for public use),⁴⁰ other PROs are copyrighted and may require payment for use, as is common for other kinds of intellectual property. Currently, EMR vendors, clinicians, and health care organizations are generally unaccustomed to this model for PROs. In the future, electronic methods should make it feasible to track use of PROs, and to charge for them as appropriate. For example, a clinician may order a PRO test, and the patient or his/her insurer will be billed for this service, as with laboratory tests.

For health care organizations, there are few precedents for the governance of PROs used on an institutional basis. In general, individual investigators or research groups select PRO tools for their studies. Important questions to consider as PROs are integrated in clinical systems include: Who has the authority to establish standards regarding the selection of PRO tools for clinical use? Who should establish score interpretation guidelines? In practice, these decisions may fall to technical members of the health information technology team, but it seems clear that practitioners should lead efforts to establish clinical standards.

In general, PRO information that exists in EMRs is available to any health care worker with a system log-on in the same way that a blood count or radiograph is available. Clinically, this is generally restricted to members of the patient’s care team, on a need-to-know basis. However, some confidentiality considerations are specific to PROs and may trump common practice for other data elements. For example, a patient who completes a satisfaction survey may not want his/her provider to see the results, for fear of retribution if the responses are critical.

Perhaps these data should only be available to individual providers on an aggregated basis. Just as psychiatric care is privileged due to the high risk for discrimination associated with psychiatric diagnoses, some patients may want to restrict access to their self-reported information on sexual functioning. The balance between confidentiality and clinical utility of specific PROs must be considered carefully and systematically. We believe that organizations and other standard-setting bodies should establish procedures for considering the adoption of specific PRO for use within EMRs, and then apply these procedures to evaluate each candidate PRO.

Once these governance and data policy decisions have been made, to whom should they apply? It seems logical for large health systems or organizations sharing specific electronic data tools to use the same standards. Common standards could also be applied across different organizations using, for example, a common EMR (e.g., EPIC). Most small organizations do not have the capacity to handle the technical work and decision-making independently. Accountable care organizations (ACOs) and regional health information exchanges may play an important role in centralizing decision-making and establishing standards for broad application.

There are a number of unanswered questions related to using PROs in PCOR, including how to select the PRO measure and interpret the results. Thousands of PROs have been developed and published. Which of these are appropriate for clinical use? The Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID), a curated, categorized library of PRO measures provides a reference source for available PROs.⁴¹ Also, in some cases, particular PROs have long been in use (e.g., the International Prostate Symptom Score).⁴² Clinicians in these specialties have become accustomed to using these PROs as clinical tools, but their tacit knowledge regarding these tools has not generally been codified for a wider audience. This is a necessary step for broader dissemination.

For most PROs, broad use would be aided by establishing criteria for psychometric properties and test characteristics for clinical use. For example, how many points on a given scale represent a meaningful or clinically important difference,⁴³ and what cutoff score represents a level of dysfunction that requires a clinician’s attention.⁴⁴ In evaluating treatment effectiveness, missing PRO data are often informative, e.g., sicker patients are less likely to complete their PRO questionnaires. Thus, it must be determined how missing data are to be handled in analysis. Organizations should consider this question prospectively and establish standard/default procedures in consultation with experts in PROs and missing data. These procedures would then be applied to specific PROs, with due consideration to established evidence about the performance of the PRO related to missing data.

An additional issue is the potential to link PRO data to other data sources. In some cases, patients with records in administrative data sources such as insurance billing or pharmacy benefits managers can be identified and invited to respond to survey questions. This requires patient identifiers, but the operation can be performed via a third party in a secure, HIPAA compliant manner.⁷ Although issues remain related to patient consenting procedures and implementation, this represents a novel mechanism to produce enhanced sources of information.

POLICY (AND RESEARCH) INTERVENTIONS TO PROMOTE PROs IN PCOR

To accelerate the adoption of PROs in PCOR, interventions are needed that can be aimed at stakeholders across the health care system, including patients, family members, patient organizations, clinicians, clinical managers, health care organizations, manufacturers of medications and devices, payers, employers, communities, states, and researchers themselves.

Various policy interventions could be applied to stimulate the adoption of PROs for PCOR. Research policy could require the collection of PROs in specific prospective studies, registries, and cohort studies. Electronic data collection methods make this requirement less onerous and less costly than it may have been previously. FDA regulations could require the collection of these data in Phase III and post-marketing surveillance studies. The Patient-Centered Outcomes Research Institute (PCORI) and National Institutes of Health could encourage the creation of data crosswalks (sometimes referred to as backdoors) between large national studies to permit comparison of results across studies. A critical advantage of PROMIS is the ability to compare scores from any PROMIS measure normed to the general U.S. population, regardless of whether a fixed-item form or CAT is used.³⁶

Coverage/payment policy could create financial incentives to increase PRO assessments. For example, Medicare could reimburse physicians or hospitals for PRO data collection when these data are shown to add value. Another fruitful area is including PROs in cases of “coverage with evidence development” (e.g., when Medicare agrees to pay for a promising but still unproven treatment subject to collection of registry data).

In other cases, organizations such as the Joint Commission and NQF might require PRO data collection as a condition for accreditation or compliance, similar to the Joint Commission’s requirement that data on pain be adopted as a fifth vital sign.⁴⁵ Health systems or hospitals could also bundle PROs together as part of the default or recommended panel of tests for a given condition. This is often done for clinical tests that might otherwise be ordered incorrectly, such as thyroid function or iron studies. For example, the HAQ (a PRO for rheumatoid arthritis [RA]) might be part of a default set of measures collected for RA patients.

As PROs are increasingly included in clinical trials, methods are needed to support literature synthesis and meta-analysis of PRO data. A fledgling effort is already underway by a wing of the Cochrane Collaborations dealing with PROs.

CONCLUSION

In summary, PROs have the potential to improve the quality and patient-centeredness of medical care in a variety of ways. They can be used at the individual patient level to improve interactions between patients and clinicians. They can also be used in research studies to identify benefits and harms of interventions. Finally, they have a role to play in policy-making and population surveillance, including contributing to guideline development, informing coverage and reimbursement decisions, evaluating care quality, and identifying the impacts of policy options. The PRO data collected for these purposes can in turn be used for PCOR. However, this integration of PROs in PCOR requires linking PRO data with clinical information from a range of data sources. Electronic data methods play a pivotal role in executing this linkage.

This is an optimistic vision, and there is a great deal of research to be done before PROs will be fully embraced by all stakeholders. This will take time, as for all new technologies, as well as evidence for utility and value added. Looking ahead, research and policy initiatives are needed to facilitate the routine use of PROs in policy, research, and practice, and to enable the linkage of PRO data with other clinical data to enable PCOR.⁴⁶ In doing so, it will be important to address the ethical, methodologic, and governance considerations that currently prevent optimal application of PROs in PCOR. Funders such as PCORI and the Agency for Healthcare Research and Quality should support research investigating solutions to those challenges. Addressing these barriers will be critical to putting the patient perspective in patient-centered outcomes research.

TABLE 1.

Challenges and solutions for using PRO in PCOR

Challenge	Proposed Solutions for Policy, Research, and Practice
Comprehensive, uniform adoption of PRO	Funding agency, FDA, payor, accreditor requirement for PRO data collection for specified conditions Reimbursement and billing for PRO data collection Research to establish process-outcome linkage for specified conditions Bundle PROs with clinical measures for specified conditions
Selection of PRO	Consensus on recommended PROs for specified conditions and situations Research on psychometric properties, natural history, heterogeneity
Proprietary measures	Studies to compare proprietary to public domain measures Begin with PROs currently in use, in public domain, or copy-left
Clinical interpretation	Research on clinical interpretation Engage clinicians and patients/family members
Literature synthesis/meta analysis	Research on meta analysis methods for PROs
Confidentiality	Establish standard procedures to evaluate PROs Research on stakeholder preferences Obtain input from patients and clinicians
Coordination among organizations	Create data sharing consortia
Data linkage	Research studies on data linkage
Missing data	Establish guidance for handling Research studies on impact of missing data; imputation