Measure Once, Cut Twice – Adding Patient-Reported Outcome Measures to the Electronic Health Record for Comparative Effectiveness Research

Abstract

Objective

This paper presents the current state of patient-reported outcome measures, and explains new opportunities for leveraging the recent adoption of electronic health records to expand the application of patient-reported outcomes in both clinical care and comparative effectiveness research.

Study Design and Setting

Historic developments of patient-reported outcome, electronic health record, and comparative effectiveness research are analyzed in two dimensions: patient-centeredness and digitization. We pose the question: “What needs to be standardized around the collection of patient-reported outcomes in electronic health records for comparative effectiveness research?”

Results

We identified three converging trends: the progression of patient-reported outcomes toward greater patient centeredness and electronic adaptation; the evolution of electronic health records into personalized and fully digitized solutions; the shift toward patient-oriented comparative effectiveness research. Related to this convergence, we propose an architecture for patient-reported outcome standardization that could serve as a first step toward a more comprehensive integration of patient-reported outcomes with electronic health record for both practice and research.

Conclusion

The science of patient-reported outcome measurement has matured sufficiently to be integrated routinely into electronic health records and other e-health solutions to collect data on an ongoing basis for clinical care and comparative effectiveness research. Further efforts and ideally coordinated efforts from various stakeholders are needed to refine the details of the proposed framework for standardization.

Introduction

In 2010, controversy about the desirability of comparative effectiveness research briefly obscured what many of us knew: there is a great need for evidence to guide decision making by patients and clinicians.[1,2] The passage of the federal health reform legislation, on the backdrop of a financial downturn, left a greater need than ever for comparative effectiveness research.[3] In recent years, three developments have converged: measurement of patient-reported outcomes, evolution of medical records, and comparative effectiveness research. Figure 1 depicts the co-evolution and convergence of patient-reported outcomes measurement, medical records and comparative effectiveness research over the past half century. The vertical axis indicates increasing patient-centeredness, and the horizontal axis indicates increased electronization.

Figure 1.

The convergence of patient-reported outcome (PRO) measures, electronic health records (EHR) development, and comparative effectiveness research (CER) movement. Note the increased adoption of electronic solutions for patient-reported outcome measures, the electronic conversion of medical records and emphasis on patient-centered solutions; and, the shift in emphasis of comparative effectiveness research to more patient-centered outcomes research (PCOR).

HSM=Health Status Measurement

HRQOL=Health Related Quality of Life

PRO = Patient Reported Outcome

PROMIS = Patient-Reported Outcomes Measurement Information System

PCOR=Patient Centered Outcomes Research

PHR=Personal Health Record

CER=Comparative Effectiveness Research

EHR=Electronic Health Record

Circles indicate measurement of PROs

Triangles indicate the comparative effectiveness research field

Squares indicate the predominant forms of medical records

In this paper, we discuss the emergence of the three trends, and the collective opportunity they present for enabling and enriching patient-centered outcomes research. We complement this discussion by examples from our recent experience at our institution in all three areas. Our institution has decided to abandon its current, homegrown electronic patient record in favor of adopting a commercially available electronic health records system. Fortuitously, the latest release of this electronic health record system includes a tethered patient portal that is equipped to collect patient-reported outcomes from patients. This development has propelled us towards the practical aspects of implementing patient-reported outcomes for clinical care and research. The adoption process has revealed a series of choices and decisions to be made. Our discussion of these issues in this paper is punctuated by examples from our working through of that process. In particular, we have recognized the need for standardization, guided by the input of the many relevant stakeholders. The paper culminates in the recommendation of an architecture for the ideal integration patient-reported outcomes within the electronic health record for comparative effectiveness research. We believe that judicious decision-making about architecture and standards will allow collection of patient-reported outcome data at a single point in time to serve two or more purposes – thus reversing the proverb “measure twice, cut once” to “measure once – cut twice”.

Patient Reported Outcomes

Patient outcomes can be measured from several perspectives: clinical, patient and societal.[4] Initial measures of health status were often measured from the point of view of a clinician or caregiver such as the Karnofsky Performance Status Scale[5] and Katz Activities of Daily Living.[6] Measurement from the patient perspective began to be used widely in health care research in the 1970s and 1980s, and in clinical research soon thereafter. Measurement of these concepts, referred to initially as health status and health-related quality of life, saw major advances in the 1990s.[7] In 2001, the term patient-reported outcome was introduced as a byproduct of a harmonization effort to encourage uptake by the US Food and Drug Administration (FDA).[4] Use of this term was solidified by its adoption in the subsequent FDA guidance.[8] The Patient-Reported Outcomes Measurement Information System was initiated in 2004 as part of the US National Institutes of Health (NIH) Roadmap program.[9] The goal of Patient-Reported Outcomes Measurement Information System was to develop the ability to measure patient-reported outcomes with greater precision and fewer questions using a set of questions tailored to the individual’s level of health.[10]

Electronic Health Records

The origins of the modern medical record date back to the 1970s[11], when medical records were still maintained on paper. Initial attempts to digitize these records and provide computerized decision support were limited to a few academic settings.[12] In the 1980s electronic medical records were mainly developed for administrative purposes and lacked a user-friendly interface; however, in 1990s the first Windows-based medical records were released.[12] In the 2000s electronic medical records began to include non-clinical health information, leading to the broader term "electronic health record". Electronic health records soon began to integrate patient portals (see Table 1). The process was accelerated by the Health Information Technology for Economic and Clinical Health Act and eligibility for stimulus funds related to the “meaningful use” of data among electronic health record adopters.[13] Although a separate movement of standalone personal health records was also initiated in this period, the electronic health record-tethered personal health records (i.e., electronic health record-based patient portals) was the most resilient architecture. (To limit the use of abbreviations, this document will use the word electronic health record interchangeably with electronic health record-tethered personal health records and electronic health record itself). The electronic health record-based portals permit patients to retrieve their records and to enter additional information.[14] In addition, a number of standalone webtools were designed specifically to handle patient-reported outcome measures, such as PatientViewpoint, developed by our group.[15,16]

Table 1.

List of selected EHR systems with considerable market share and their integrated/built-in patient portals (note: standalone PHRs are not included)

EHR	Patient Portal	Type	Customizable
Allscripts	[Sunrise] Patient Portal	Commercial	Yes – through Microsoft HealthVault capabilities
Cerner	Cerner Patient Portal	Commercial	Yes – limited to certain variables / no custom questionnaire integration
eClinicalWorks	Patient Portal	Commercial	Yes – limited to certain variables / no custom questionnaire integration
Epic	MyChart	Commercial	Yes – includes custom questionnaire integration
GE Centricity	Patient Portal	Commercial	Yes – includes custom questionnaire integration
Meditech	Patient and Consumer Health Portal	Commercial	No
NextGen	Patient Portal	Commercial	Yes – limited to certain variables / no custom questionnaire integration
OpenMRS	PHR Module Enhancement	Non-profit Open-source	Yes – fully customizable with new questionnaire and additional functionalities
Practice Fusion	PHR	Commercial	No
Siemens MobileMD	Patient Portal	Commercial	No
VistA	My Health-e-Vet	Government Available to VA facilities	Yes – fully customizable with new questionnaires and additional functionalities
WebChart	WebChart PHR	Commercial	No

EHR=Electronic Health Record

PHR=Personal Health Record

VA=Veterans Affairs

Effectiveness Research

Early efforts in outcomes and effectiveness research were initiated in the 1980s.[17] Recognition of the need for data on the efficacy and effectiveness of health care interventions began in earnest in the 1990s.[18,19] Large effectiveness research projects were funded by AHRQ (e.g., Patient Outcomes Research Teams – PORTs) and foundations such as the Robert Wood Johnson Foundation.[20,17] For example, one of the PORT-II studies examined the effectiveness of hemodialysis versus peritoneal dialysis.[21] Shortly after, AHRQ’s effective health care program constituted research networks to generate evidence including the Evidence-based Practice Centers[22,23] and the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) centers.[24] Substantial recent funding accelerated these efforts. The 2009 American Recovery and Reinvestment Act (ARRA) provided $1.1 billion for comparative effectiveness research, of which a portion was disbursed to AHRQ to fund comparative effectiveness research.[25] The Affordable Care Act designated additional funding for comparative effectiveness research, in part through its creation of the Patient Centered Outcomes Research Institute (PCORI), which is slated to receive an additional $1.2 billion through 2019. Over time, the AHRQ has placed increasing emphasis on the centrality of stakeholder engagement in comparative effectiveness research, most importantly the involvement of patients. Moreover, PCORI has moved comparative effectiveness research towards a more explicitly patient-centered approach.[26]

Thus, the measurement of health status and health-related quality of life has become more patient-centered in philosophy and approach. Measurement has also become more standardized, a process accelerated by research, market forces, and informatics solutions. Paper-based medical records have been converted into electronic health records with customizable tethered patient-centered portals. Additionally, the practice of comparative effectiveness research has become more patient centered, with increased emphasis on stakeholder participation. Patient-reported outcomes, electronic health records, and comparative effectiveness research have converged in a more patient-centered and increasingly digital space providing the opportunity for standardization in collecting patient-reported information (Figure 1).

Just Add Patient-Reported Outcomes?

Conventional comparative effectiveness research employs data sources such as administrative/billing data, clinical research data, and medical records. These sources produce variable amounts of information about clinical outcomes and health care utilization, but generally lack the patient perspective. Mentions in clinical notes about health status tend to be ambiguous and non-standardized, such as “patient is doing well.”[27] There are several ways in which patient-reported outcome can be combined with other data sources to provide information about comparative effectiveness research.[28,29] The classical approach is to include patient-reported outcomes in clinical trials and observational studies or link them to administrative data. Patient-reported outcomes can further be combined with clinical data including electronic health records to provide a more complete picture for researchers and clinicians.

The conventional approach can be modified successfully, as exemplified by a comparative effectiveness research study being conducted to improve patient outcomes in end-stage renal disease.[30] The DEcIDE Patient Outcomes in end-stage renal disease Study was commissioned to compare three common treatment strategies for patients with end-stage renal disease, including 1) antihypertensive medication regimens, 2) intravenous iron therapy for anemia, and 3) early versus later dialysis initiation. Outcomes for each question include all-cause and cause-specific mortality and morbidity as well as patient-reported outcomes. Among numerous data sources employed to study these questions, two cohort studies provide a ‘backbone’ of data that contains information on patients’ routine medical care and patients’ self-reported health-related quality of life. Cohort data are derived from electronic medical records generated during the course of routine dialysis care, and capture patients’ demographic characteristics, laboratory measures, outpatient medications and medications administered in the dialysis facility, and patient-reported outcome measures (collected annually). Additional data complement these clinical data, including administrative data contributed by dialysis clinical providers at the initiation of their care and at the time of patients’ death (required by the Centers for Medicare and Medicaid Services), and Medicare billing records generated during patients’ outpatient and inpatient hospital care. Thus, this study has used various data sources to provide a complete picture of care.

With the evolution of medical records to electronic health records, and the incorporation of patient portals (see Table 1), new opportunities for efficiently gathering data for such comparative effectiveness research studies have emerged. The patient portals can be equipped to collect patient-reported outcomes directly from the patients (it should be noted that not all of them do yet). Some electronic health record systems incorporate a predefined set of standardized measures into their routine workflow, such as the Short Form (36) Health Survey (SF-36), Patient Health Questionnaire (PHQ-9) and brief, static Patient-Reported Outcomes Measurement Information System questionnaires. [31,32,10] Thus, the clinician/researcher can chose a patient-reported outcome measure from a list of pre-populated standardized patient-reported outcome questionnaires, administer it through the patient portal, and analyze the results for clinical care/comparative effectiveness research. In addition, multiple technological advancements have further pushed the patient-reported outcome integration into day-to-day clinical operations such as: standalone patient-reported outcome-based patient portals (e.g., PatientViewpoint)[15] and mobile patient portals (mPHRs).[33]

The electronic health record-based integration of patient-reported outcomes provides usable information to various stakeholders. Initially, patient-reported outcome data have been collected primarily for clinical care, as if they were lab tests. However, in parallel, efforts are underway to use the same infrastructure to collect data for clinical research (e.g., comparative effectiveness research). Thus, it is possible that patient-reported outcome data collected once can be useful for at least two purposes, if not more: clinical care and comparative effectiveness research/ patient-centered outcomes research. The electronic health record-based patient-reported outcome data may also be used for quality assessment, quality improvement, accountability measures, pay for performance, population indices, and public health solutions.[34,35]

The Need for Standardization

The key advantage of patient-reported outcome methods is the ability to capture the patient experience in a standardized, systematic way with established reliability and validity.[4,36,28] If successful, electronic health record-based patient-reported outcome standardization efforts could provide two benefits: (1) if pooled/aggregated across providers/centers, patient-reported outcome measures can generate research evidence for literature synthesis/meta-analysis; and, (2) if collected across multiple organizations, they can be employed to compare and evaluate quality measures / improvements among the providers. The latter will render valuable information to accountable-care organizations to increase patient-centric measures while offering high performance outcome and low cost of service.

To make such standardization a reality, harmonizing efforts are needed on multiple levels. First, much greater coordination is needed among the components of individual institutions. For example, in 2012 our institution had more than 12 different electronic health record systems across inpatient and outpatient units, specialties and subspecialties, pharmacy and laboratory, radiology and pathology, the emergency department and critical care, and others. Secondly, coordination is needed among various integrated providers, especially given the tendency for health care organizations to partner with small practices and networks of hospitals that may use different electronic health record/patient-portal platforms.

At the next level, standardization efforts should focus on provider groups, integrated delivery systems/networks, health maintenance organizations, healthcare delivery systems, metropolitan- or state-wide health information exchange groups, and regional health information organizations. At each level, various degrees of standardization may be required to ensure that patient-reported outcome measures are collected, shared, and retrieved in a meaningful way. The standardization efforts should ensure that similar patient-reported outcome measures are available across organizations, and the results are interpreted similarly.

De-facto standardization via market dominance by a few EHR vendors may play an important role in consolidating patient-reported outcome measures. For example, one commercially available electronic health record used by many academic medical centers is both comprehensive and customizable, and includes a patient portal with built-in patient-reported outcome questionnaires. There is concern that a dominant system could establish the “laws and the language” for how information is formatted and flows. In the best-case scenario, resembling Apple’s ecosystem of apps, the result could be a clear set of operating standards and expectations that innovate and adapt to market needs. However, an internally controlled system could also limit the development of new ideas.[37] This form of governance could exclude external contributions, or more likely, result in variations in the external contributions that result from multiple unique and disjointed installations of the same system.

Recommendations

An important goal of standardization is for the same patient-reported outcome data to serve two or more purposes. In making plans for implementation of patient-reported outcome data collection within the electronic health record, it will be important to anticipate the needs of clinicians, clinical managers, and health care researchers. A preliminary question that needs to be addressed is: “What needs to be standardized around the collection of patient-reported outcomes in electronic health records for comparative effectiveness research?”

We have attempted to answer this question by dividing the task of incorporating patient-reported outcome measurement in the electronic health record into several stages on the pathway of planning through implementation: 1) Selection; 2) Administration; 3) Reporting and Interpretation; 4) Analysis; and, 5) Access, Confidentiality, and Security. To focus the discussion, we will concentrate primarily on patient-reported outcome for research needs. We will assume that the context of use is the clinical use of electronic health records, primarily for ambulatory care, but also for inpatient care. We assume that patient-reported outcome data collection will generally be initiated by clinicians who order patient-reported outcomes for individual patients as if they were laboratory tests. These may be intended as screening tests, or as outcome measures to evaluate the trajectory of health or the effects of treatment. Table 2 shows a sample of the issues that we are confronting at our institution.

Table 2.

Areas for Standardization of PROs within EHRs for CER

Area	Key questions
Selection	What are domains to be measured?
	How to identify existing candidate tools?
	Evidence for measurement properties
Clinical use	What PROs are already used routinely?
Quality measurement	What are goals for PRO use?
Research	What are PRO research questions?
Inclusion of proprietary measures?	Only public domain?
	Only freely available?
	Track and charge for test?
	Is someone willing to pay for test?
Administration	Standardize instructions?
	Special instructions?
	Location of administration
	Mode of administration
Interpretation
What do scores mean?	What are clinically relevant cutoffs?
Connectivity	Common messaging and content?
	Common terminology?
Analysis	Collect metadata
	Protocols for missing data
Accessibility
Clinical data	Release automatically to EHR?
	Accessible to patient?
Research/Quality Data	Separate consent process?
	Release to EHR?
	Viewable by clinicians?

1. Selection

To select a patient-reported outcome measure, it is necessary to identify existing tools for measuring the outcome of interest in the target population. There are a few measures that are already in wide use, such as the International Prostate Symptom Score (IPSS)[38,39]. These measures are likely to be among the first to be implemented, since there is already clinician demand for them. In many other cases, identifying a patient-reported outcome measure is more complex. To aid selection, there are catalogs of patient-reported outcome measures, such as the patient-reported outcome & Quality of Life Instruments Database (PROQOLID), a searchable, curated library of measures, with references and the individual(s) to contact for each.[40]

A patient-reported outcome should meet a number of methodological standards; in particular there should be adequate evidence for the reliability, validity, responsiveness and interpretability of the measure in the intended population. Review of these issues is beyond the scope of this paper, but a number of guidelines have been published recently.[41–48]

An additional consideration is whether the selected patient-reported outcome measure is proprietary, and more practically, whether payment is required for using a questionnaire. Some of the most widely used patient-reported outcome general health measures, such as the SF-36[31] and the Health Utilities Index (HUI-3),[49] are copyrighted, and there is a charge for each use, or another payment scheme such as a license for unlimited use within a specified time period.

However, health care organizations and electronic health record vendors are not yet accustomed to this patient-reported outcome business model. Some may elect to use only patient-reported outcomes that are available in the public domain. This may be more because added steps slow the development process, rather than the charge per se. One commercially available electronic health record is using the publically available RAND-36[50] and patient health questionnaire (PHQ-9)[32] in their first version of patient-reported outcome assessment within their patient portal. Another interesting development is the practice known as “Copyleft,” whereby a patient-reported outcome measure is copyrighted to be made freely available for public use.[51] However, as patient-reported outcomes demonstrate their value in health care, it should be possible to track their use within an electronic health record. It would then be possible to bill a patient, a payer, or a research grant for the use of this service.

2. Administration

Some laboratory tests are accompanied by instructions for administration, such as “fasting blood glucose” or “a.m. cortisol.” It may be desirable to impose or at least recommend standards for administering patient-reported outcomes. These may be general, such as "complete within a specific time window" or "before a clinic visit", or "complete without assistance." This kind of standardization is required to reduce measurement error introduced by the context in which the data are collection. It would be most helpful if data are to be aggregated and compared across different treatment settings or organizations.

Many questions will arise while implementing patient-reported outcome data collection methods: Can reminders be sent via text as well as emailed? Can responses be given via app on a phone as well as via PC? Is there provision for in-clinic data collection in a quiet and confidential setting? The task for organizations will be to proactively seek answers and develop solutions to address these questions. At our institution there are small-scale efforts to develop and test all of these mechanisms. The next step will be decide which of them to implement system wide.

3. Reporting and Interpretation

A common complaint voiced by clinicians about patient-reported outcome measures is that it is difficult to understand what the scores mean. For clinical laboratory tests in many electronic health records, values significantly out of the normal range are automatically highlighted in a notable color such as yellow. What is the equivalent for a specific patient-reported outcome scale? What cutoff score on a pain scale represents a level of dysfunction that should warrant an inquiry?[52] Although these questions seem more pertinent to clinical practice than to research, the solutions are likely to affect what clinicians do, and hence patient outcomes. This will also have an effect on the ability to conduct research that compares the impact of different treatments or other items of interest.

In addition, what messaging and content standards (e.g., Health Level Seven International (HL7) standards for interoperability) are currently applicable to patient-reported outcome measures? Are there common terminology and vocabulary systems (e.g., International Statistical Classification of Disease, Ninth Revision (ICD-9), Systematized Nomenclature of Medicine (SNOMED)) that can be used in the patient-reported outcome domain? Again, the task here for governing bodies will be to proactively seek answers to these questions.

4. Analysis

The analytic methods applied to patient-reported outcome data will vary based on the proposed research questions. A few considerations, however, may be important for data collection to support different analytic goals, including estimation and risk adjustment. For example, it may be helpful to collect metadata elements such as location where completed (home vs. in-clinic), mode of administration (face-to-face vs. phone or computer), or completion with assistance (completed by surrogate, translated by interpreter, or family member). The availability of these data would help researchers to generate evidence about the reliability and validity of patient-reported outcomes to measure specific questions of interest. Such metadata can also help to clarify the unit of analysis, and assist with risk adjustment or imputation of missing data.

5. Access, Confidentiality, and Security

Within some electronic health record patient portals there is an option for individual clinicians to “release” a specific piece of information from the patient portal to the electronic health record, where it is then available to all health care staff members with an individual logon to the system. As with all clinical information, it is general policy that actual accessing of information is on a need-to-know basis, restricted to members of the care team.

Some data access and confidentiality considerations are unique to patient-reported outcomes. For example, some data that are collected for quality measurement rather than clinical care, such as patient satisfaction surveys, should probably be made available to individual providers only on an aggregate basis, or perhaps only to clinical managers.

A number of critical questions should be answered: When data are collected via an electronic health record, should they be made available in the clinical record? Would a separate consent procedure be required? Should patient-reported outcome results be shared with patients? How do different technical architectures affect the privacy and confidentiality of patient-reported outcome results?

Governance

Who should decide on these and other standardization issues? We believe that the need for governance underlies them all. There is little precedent within health care organizations for the governance of patient-reported outcome use within electronic health records and no consensus on determining standards for patient-reported outcome tools to be incorporated into electronic health records. It must be acknowledged there is also a downside to standardization; there are advantages to tailored use of specifically selected questionnaires to meet institutional needs. Therefore, standardization should be conducted with care.

In practice, most standards regarding electronic health records are operationalized by information technology professionals. However, few of these individuals have the training and experience to establish patient-reported outcome standards that have both scientific and clinical ramifications. In addition, the meaningful use[13] requirements (i.e., stage I and II) for electronic health records are currently lacking any policy or guidance on patient-reported outcome implementation. Incorporating patient-reported outcome requirements in future versions of meaningful use policies will ensure an aligned effort between EHR standardization champions and patient-reported outcome harmonization experts.

One option would be to designate an expert group to guide the electronic health record-based patient-reported outcome standardization policy. At our institution, we have established a multidisciplinary “taskforce on the systematic collection of patient reported outcomes” composed of clinicians and researchers across the schools of medicine, nursing, and public health, as well as information technology leaders and technicians. The group has tapped the PROQOLID database to identify patient-reported outcome measures corresponding to specific domains of interest (e.g., dementia). The group has also surveyed leading researchers and clinicians to identify patient-reported outcome measures that are already in routine clinical use. At a higher level, it would be desirable to cultivate national consortia with broad expertise and experience with patient-reported outcomes to provide guidance. It can be debated whether a Clinical Laboratory Improvement Amendments (CLIA)-like regulatory organization (which regulates clinical laboratories) should be established to enforce standardization.

Conceptual System Architecture

We propose a conceptual system architecture to help guide standardization that will enable high-quality comparative effectiveness research and the ability to "measure" once within electronic health records, and "cut" at least twice for multiple uses of the resulting data (figure 2).

Figure 2.

An idealized conceptual system architecture for an interoperable city/state/region/nation-wide HIE-based PRO platform that can be used for various analytics including CER. Orange: Typical HIE data sources; Dark green: Population health data sources; Light green: PRO data repositories; Light gray/dashed border: sample systems; Arrows with middle circle: PRO messaging standardization required. For the sake of simplicity the diagram does not show existing relationships among non-HIE components (e.g., connection between lab systems and EHRs) and alternative access points for CER researchers (e.g., CER researcher accessing isolated EHR-PRO data directly). Abbreviations: EHR=Electronic Health Record, PRO=Patient Reported Outcome, HIE=Health Information Exchange, CAT=Computerized Adaptive Test

Electronic data repositories of patient-reported outcome measures are collected and stored in silo systems such as: electronic health record-based patient portals (a.k.a., tethered-personal health records; e.g., MyChart); standalone [mobile] patient portals (a.k.a., [m-]personal health records; e.g., Microsoft HealthVault); standalone patient-reported outcome portals (e.g., PatientViewpoint); digitized version of local paper-based patient-reported outcome research studies (e.g., local Patient-Reported Outcomes Measurement Information System Computer Adaptive Tests (CATs)); and, multicenter provider/payer patient-reported outcome research systems.

These dispersed patient-reported outcome repositories are currently not connected under Health Information Exchange initiatives. Health information exchanges collect, aggregate, and share certain clinical data across multiple electronic health records/providers; however, they often ignore the data captured in electronic health record-based patient portals (i.e., where patient-reported outcome measures are collected) due to the lack of standardization mechanisms and lack of clinical importance to providers. In addition, health information exchanges do not traditionally collect the data captured in non-electronic health record data repositories such as standalone patient portals, research data repositories, and non-clinical population health datasets (e.g., social work).

In an ideal/conceptual system architecture, health information exchanges can develop separate or embedded technical layers that collect, map, aggregate, store, retrieve, and analyze different sources of scattered patient-reported outcome data repositories. In this conceptual framework other patient-reported variables, both subjective and objective, that are captured in patient portals can also be shared for other research/clinical purposes. This general storage/analytic layer could be called the population health layer which will have a patient-reported outcome module to handle the patient-reported outcome data in a standardized way.

Summary

We are at an opportune moment in the history of comparative effectiveness research/patient-centered outcomes research, when patient-reported outcome measurement has matured sufficiently to be used "off the shelf" in clinical and health care research, and electronic health records provide a routine mechanism for collecting patient-reported outcome measurements. The convergence of electronic platform and patient-reported outcome technology provides the opportunity to build a system to collect comparative effectiveness research/patient-centered outcomes research information on an ongoing basis. Standardization is important for a host of reasons. The time to harmonize is now, as there are crucial questions that concern both clinical and healthcare research methods, implementation, and governance of how patient-reported outcomes will be used within electronic health records as a data source for comparative effectiveness research.

What is new?

key findings

• The fields of patient-reported outcome measurement, electronic health records, and comparative effectiveness research have converged to a space that is more patient centered and more electronically based.

what this adds to what is known

• New patient portals connected to the electronic health record can provide a unified platform for computerized patient-reported outcome measures

what is the implication, what should change now

• This is an opportune time to intervene and align various stakeholders to harmonize patient-reported outcome measures in electronic health record to accelerate comparative effectiveness research.

• Coordinated efforts are needed to complete electronic health record conversion in a way that allows efficient generation of knowledge.