Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs)

Abstract

In recent years, there has been an increased focus on placing patients at the center of health care research and evaluating clinical care in order to improve their experience and ensure that research is both robust and of maximum value for the use of medicinal products, therapy, or health services. This paper provides an overview of patients’ involvement in clinical research and service evaluation along with its benefits and limitations. We describe and discuss patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs), including the trends in current research. Both the patient-reported experiences measures (PREMs) and patient and public involvement (PPI) initiative for including patients in the research processes are also outlined. PROs provide reports from patients about their own health, quality of life, or functional status associated with the health care or treatment they have received. PROMs are tools and/or instruments used to report PROs. Patient report experiences through the use of PREMs, such as satisfaction scales, providing insight into the patients’ experience with their care or a health service. There is increasing international attention regarding the use of PREMS as a quality indicator of patient care and safety. This reflects the ongoing health service commitment of involving patients and the public within the wider context of the development and evaluation of health care service delivery and quality improvement.

Introduction

High-quality clinical care requires patients to provide information regarding how they are feeling, their symptoms, and any effects of prescribed treatment. The medical outcomes study was at the forefront of this concept as patient outcomes were examined and differences in care, clinicians, and communication styles were reported for both patient and clinical outcomes.1 A further extension of this concept into drug development research has only been evident in the past two decades.2,3 The pharmaceutical industry recognized the importance of considering patient-reported outcomes (PROs) along side biomarkers of health improvement.4 The separation between health outcomes and treatment outcomes became clearer when research into health services began to focus on improving the patients’ health-related quality of life, particularly when patients were undergoing optimal medical therapy.4 This, in turn, created the need for identifiable, valid, and reliable patient-reported measures (PROMs).

The increasing attention being given to patients involved in research studies has led to other initiatives such as Patient and Public Involvement in the United Kingdom.5 This initiative builds upon commissioned research by the Health Technology Assessment program that sought to identify the benefits and barriers to patient involvement in research and the associated research governance activities, such as human research ethics committees.6 Therefore, this paper provides an overview of patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in research and clinical practice. We also describe patient-reported experiences measures (PREMs) and initiatives for patient involvement in research such as the Patient and Public Involvement program (PPI).

Patient-Reported Outcomes (PROs)

A PRO is directly reported by the patient without interpretation of the patient’s response by a clinician or anyone else and pertains to the patient’s health, quality of life, or functional status associated with health care or treatment.7 These outcomes may be measured in absolute terms, such as a patient’s rating of the severity of pain. PROs can also be used to report changes from a previous measure such a new onset of nausea following administration of a new drug. Traditional survival, disease, and physiological outcomes may demonstrate the physiological benefits of treatment; however, the patient perspective provides a more holistic interpretation and a comprehensive assessment of the benefits of the treatment under investigation.8,9 For example, a new drug may demonstrate good clinical outcomes in terms of improving the length of survival for a particular patient group, while PROs may identify that patients are non- compliant with the drug regime due to reported adverse or side effects, complexity of the drug regime, and or a poor quality of life. The effectiveness of any therapeutic intervention, therefore, has many dimensions, including clinical effectiveness of the intervention and the benefit felt by patients as a direct result of having the intervention. The assessment by the benefit to the patient using a variety of instruments or tools may include functional status, service satisfaction, and quality of life.

The patient experience has played a part in clinical research for some time and continues to increase with the recognition that a whole system, such as a patient-centered approach, is necessary for comprehensive assessment of the impact of treatment and care. In clinical pharmacology trials, PROs are used as either primary outcome measures or to complement primary outcome measures.7 When clinical trials involve conditions in which there is no objective outcome measurement, such as the degree of morbidity or biomarkers for symptoms, and in which outcomes can only be observed subjectively to the patient in terms of impact, PROs can be used as primary outcome measures. PROs are also used to complement primary outcomes such as survival rates and biomarkers as they reflect components important to the patient and may include patient reports of symptoms and other indices such as quality of life.7 Therefore, PROs can be used as either a primary outcome or as a secondary outcome of a study, while a PROM is the measurement of the PRO, such as quality of life.

Patient-Reported Measures (PROMs)

PROMs are the tools or instruments used to measure PROs. These tools may measure the patient’s health status such as health-related quality of life. These tools are often (patient) self-completed questionnaires.10 PROMs may include instruments or tools that measure functional status, health related quality of life, symptom and symptom burden, personal experience of care, and health-related behaviors such as anxiety and depression.8,11 They can be either general in nature or disease-specific.8 Broader PROMs examine aspects that fit a variety of different conditions and allow comparison across these various medical conditions to assist in the evaluation and the implementation of new methods of providing care and equity of service delivery.8,12 Broad-based PROMs, such as the EuroQol EQ-5D, also enable cost-effectiveness analysis as part of a cost-utility analysis to examine the cost of a health-related intervention and the benefit it produces in terms of the number of years lived in full health.8,12

In contrast, disease-specific PROMs are designed to identify specific symptoms and their impact on the function of those specific conditions. Disease-specific PROMs have greater face validity and credibility than generic PROMs, but these comparisons cannot always be made across a variety of conditions. 8 Often, clinical studies use a combination of generic and disease-specific PROMs. For example, a study involving asthma patients may include a PROM of ‘asthma control’ (disease-specific) along with a generic PROM such as the EuroQol EQ-5D as a measure of quality of life.11

Development of PROMs in research

PROMs were initially developed for use in pharmacological and health service research8 and were largely restricted to England, Sweden, and parts of the US as a way to improve the clinical care of patients.11 In 1975, the medical profession in Sweden established the nation-wide use of PROMs using disease-specific clinical databases known as quality registers.8 By 2000, PROMs were introduced into some parts of the US with the aim of extending PROMs as a reimbursement mechanism for accountability within care organizations.8

The use of PROMs continues to expand beyond clinical research in recognition of its potential to transform health care, as well as improve quality and safety by placing the patients at the center of decision-making. This increasing usage of PROMs has culminated in PROs attaining greater credibility amongst regulatory bodies who aim to standardize their use and interpretation in clinical trials.13 For instance, both the US Food & Drug Administration and the European Medicines Agency have both released guidelines that mandate the use of PROMs to support labeling claims.2,13

Since 2009, it has become mandatory in the UK to use PROMs to report outcomes for certain elective surgical patients as a method of collecting information on the effectiveness of patient care within the NHS from the patient perspective. This mandatory adoption of PROMs has been driven by the UK government to enable comparison of health service8 and to identify strengths and weaknesses of health care delivery, drive quality improvement, inform commissioning, and promote choice.12,14 The Department of Health12 also released guidelines on the national standards (in the NHS) for the mandatory routine collection of PROMs involving some elective surgical procedures. 12 Building on this work, The Patient-Reported Outcomes group at Oxford University has produced evidence-based reports on PROMs for chronic conditions such as chronic obstructive pulmonary disorder (COPD)15 along with recommendations for selection of the best measurement instruments according to literature reviews. The formalization of, and broader emphasis on, PROs and PROMs has led to need for guidance on the development, use, measurement, and analysis PROs and the establishment of a conceptual foundation for PRO assessment to overcome issues with PROMs.

Overcoming issues in the development and selection of PROMs

Development and selection of appropriate PROMs, whether for clinical trial research or other uses such as quality improvement initiatives, requires consideration of several methodological issues including validity, sensitivity, reliability generalizability, and feasibility. One issue involves the identification and selection of valid, sensitive PROM instruments. A multitude of validated PROMs instruments have stemmed from earlier clinical research involving the patient perspective as a health care outcome.15 This work focuses on developing valid and responsive measurement instruments to produce empirical evidence regarding health from the patient perspective.16 The validity of PROM instruments is based on whether these tools represent or measure what they are intended to (construct validity) from the patient perspective.7 For example, a study investigating the impact of pulmonary rehabilitation for COPD patients on breathlessness would require the use a PROM of breathlessness.

The content validity of PROMs is equally important in selecting a PROM and is established through analysis of the instrument’s content (or items) and the concept that the test is designed to measure.17 Therefore, a precise and reliable instrument must be valid and responsive and/or sensitive to change when evaluating treatment differences in order to measure differences between groups, if they exist.7 Content validity is important in selecting or modifying existing PRO instruments, but there is a lack of consensus regarding the best practices for establishing and documenting validity within the research community.17

Within clinical research, a variety of instruments may be chosen to measure a specific PRO, such as dyspnea, making comparisons between PROMs challenging. Issues regarding comparisons of PROM instruments arise when there are different definitions of a particular PRO (eg, quality of life) and there is a choice to use a number of instruments such as generic (EQ-5D) and a disease-specific [St. George’s Respiratory Questionnaire (SGRQ)] instrument of a patient with COPD.7 Traditionally, the choices of PROMs are often based on professional judgment versus strong conceptual models creating issues with grouping and scoring items into domains. An unclear conceptual match between the PRO instrument and the intended claim threatens its content validity.17 This may result in problems with analysis and interpretation of study data as links and comparability between certain tools may come into question.

To overcome this issue, Rothman et al suggested beginning with a conceptual model and framework to guide the selection, analysis, and interpretation of PROs.18 Conceptual models should form the rationale and specification for selecting the appropriate PRO outcome(s) for a clinical trial.18 The PRO concept guides the development of a framework for measurement. For example, if the health behavior of a patient is central to the study, then a health belief model may be a relevant framework to assist with the selection of appropriate outcomes and outcome measures. Hence, all variables and their relationships to each other are given an operational meaning that guide the selection and approaches to analysis.19 This allows the focus to be maintained on interrelationships among PRO domains being measured, the content validity of each PRO instrument, and construct validity, reliability, and responsiveness of each PRO instrument when applied within a specified population being studied.17,18 The complexity of the PRO concept should be reflected in the decision to use either one-dimensional instruments (eg, rating of the perception of dyspnea) or complex tools such as health-related quality of life to measure a patient-reported multidimensional outcome.

Identification, evaluation, and recommendations of validated PROMs through national guidelines from organizations such as the Food & Drug Administration and the Oxford Patient-Reported Outcomes Group assist ininitiating and standardizing clinical research methods. In the US, the National Institutes of Health joined with several outcome scientists from across the country to collaboratively develop the Patient-Reported Outcomes Measurement Information System (PROMIS). This group of research scientists aim to create the next generation of PROMs and standardize and promote a common measurement system for PROs across clinical research.20,21

A second issue relates to the reliability, size, and content of existing PROMs. There is growing recognition that combining good measurement properties with shorter, more reliable instruments could result in better response rates by reducing the burden to responders.22 In order to produce these faster and more reliable instruments, work is being carried out not only to determine how well the PROM instruments work as a whole but also how well individual items on the PROM work based on how people respond to the individual questions. The PROMIS team uses item response theory (IRT) and computerized adaptive testing tools20,21 to assist in the development of measurement systems comprised of carefully calibrated questions that define and quantify a common concept and provide operational definitions of the trait. IRT is a probabilistic, mathematically based model used to describe the relationship between an individual’s response to questions about his or her health and an underlying variable measured by the instrument (eg, strength of attitude, intelligence). This analysis informs the tools development and helps to strengthen the content validity of the PRO instrument by identifying the most relevant and important item content in the instrument.17 The result is the production of shorter, more reliable measures which allow comparisons to be made between measurement instruments.

The aim of developing shorter and more reliable tools is to increase the rates of patient participation from target groups, particularly among underrepresented patient populations.8 While the production of shorter, more reliable measures may help improve the response rate by decreasing the time and effort required to complete them, a secondary aim would be to demonstrate the benefit and relevance of the item being measured, such as quality of life, to a previously difficult to attract target group. Rothman et al17 suggests that this will add to the content validity of the measure and can be achieved by involving patients directly with selecting the item content of the PROMs rather than presuming to know what is significant to them.

Higher response rates could also be obtained by widening the population base through the translational and cultural adaption of PROMs. Inclusion of under-represented patient groups would decrease the risk of bias and increase the generalizability of results. However, caution is required when utilizing PROMs in different populations (eg, pediatrics versus adults, South African versus European), as validation and normalization to each cultural group must be carried out prior to their administration as part of a research study. It has been suggested that translational and cultural adaptation of PROs should be directed by guidelines and standards.23

Lastly, the widespread use and feasibility of PROMs has been limited by the time-consuming and costly process of collection, analysis, and presentation of PROMs data in the conventional paper format. The internet opens up many opportunities to help improve the feasibility and cost-effectiveness of collecting and aggregating both PROMs and patient experience data. Greaves et al24 uses the term ‘cloud of patient experience’ to describe the collection of this type of information via various internet sources (eg, social networking sites, twitter, hospital review sites, etc). The potential to collect data in real-time to uncover poor clinical care and potential areas of excellence is an additional advantage over the infrequently administered paper-based tools (eg, hospital surveys).24 However, mailed surveys have the advantage of higher response rates.25 This may reflect the fact that internet users are generally younger, of higher economic status, and have less contact with health care,24,25 and therefore may not necessarily represent the target group. Offering both internet and paper-based PROMs options could minimize selection bias while improving the feasibility and cost-effectiveness of PROMs.

Patient-Reported Experience Measures (PREMs)

Patient-reported experience measures (PREMs) are tools and instruments that report patient satisfaction scores with a health service and are generic tools that are often used to capture the overall patient experience of health care. PREMs are often used in the wider population and in non-specific settings such an outpatient department. Patient experience tools, for example, may be used to monitor patient feedback and focus on the general experience such as customer service rather than an experience related to a specific disease.26,27 These instruments or tools have revealed positive associations between patient satisfaction and safety28,29 and are a reliable measure of how well a hospital is able to provide good quality service from a patient’s perspective.30 Internationally, PREMs are used to evaluate health care in terms of clinical effectiveness and economic efficiencies.27

Limitations and challenges of patient experience also need to be considered when developing policies based on patient experience and PREMs. In the US, the Centres for Medicare and Medicaid services and private insurers are basing reimbursement methods on patient satisfaction scores.28 Lyu et al28 argue that the reliability and adequacy of such policies that rely on patient satisfaction scores as metrics without consideration of the context may be questionable in some circumstances such as surgery. Patient satisfaction was found to be independent of hospital compliance with surgical process-of-care measures.28 The lack of correlation between patient satisfaction and how well surgical teams comply with best practice may be in part due to the fact that the processes are unknown to and unseen by the patient. For example, an anesthetized patient will not be able to comment on the quality of either the surgical or safety procedures within the operating theatre. However, they would be able to relate their health care processes outside of the theatre and familiar to them (eg, administrative processes, ward cleanliness, discharge practices) which would be beneficial to improving the overall quality of care.

In the UK, there is emphasis on measuring patient experience with both indicators and metrics. Metrics are precise measures of a known attribute26 such the number of falls occurring on a ward. Indicators are a type of metric that identifies issues requiring further investigation (eg, increase in number of falls) (NHS Institute for Innovation and Improvement/Public Health Observatories, 2007) and reflects how effectively an organization is performing on a set of metrics. Patient experience indicators are being embedded within NHS accountability frameworks, such as the NHS Outcomes 2012/13 framework.31 These measures increasingly used at Ministerial, policy, commissioning, and management levels as part of policy and legislation as the emphasis of the patient experience as a marker of patient care quality becomes more accepted and embedded in NHS contracts.26,32 The NHS National Quality Board produced a working definition of patient experience with eight indicators to guide the measurement.33 These indicators include: respect for patient-centered values, preferences, and expressed needs; coordination and integration of care; information, communication, and education; physical comfort; emotional support; welcoming and involvement of family and friends; transition and continuity; and access to care.33

Patient and Public Involvement (PPI)

The value of patient and public involvement (PPI) in the delivery and design of health care services has become increasingly recognized internationally. In the context of research within the UK, PPI actively involves patients and the public in the actual research process from conception through to analysis and dissemination of research findings. Thus, the role of patients and the public in PPI differs from research involving PROs and PROMs. PROs and PROMs involve research “about” or “to” patients in contrast to PPI, in which involvement in research is “with” or “by” the public.34 Actively involving consumers in the research process by obtaining their unique perspectives is thought to result in greater quality and clinical relevance of health research.6,35–37 A body of research is now evolving on the benefits and impact of PPI at different stages of the research process (eg, identification of research questions and research prioritization, research design, and protocol development, peer review of research proposals and funding decisions, medical device/intervention development, research advisory groups and other committees, data collection, data analysis and interpretation data), and on how to involve different members of the public (eg, older people, young people and parents, people with learning difficulties, mental health problems, or cancer).34 In 1999, the Department of Health introduced a policy directive to involve patients and the public in the National Health Service’s research and development process.5 This has been supported by INVOLVE, partially funded by the National Institute for Health Research (NIHR), which was set up in 1996 to promote, monitor, and evaluate PPI in research in the NHS, public health, and social care. Funding bodies such as the NIHR now require evidence of PPI in their research applications submitted for funding.

Conclusion

There is increasing recognition of the importance of involving patients and the public in in clinical research and within the wider context of development and evaluation of health care service delivery and quality improvement. PROS are reports provided by patients about their own health, quality of life, or functional status associated with the health care or treatment they have received.

PROMs are the tools and/or instruments that have been developed to ensure both a valid and reliable measurement of these patients-reported outcomes, such as quality of life measures. The inclusion of PROs through the use of PROMs with clinical outcomes in research and clinical practice provides a more complete understanding of the impact of an intervention, therapy, and/or service on the patient. PPI in the research process ensures that the research is relevant to patients and that the appropriate measures are selected.

Patient reported experiences through the use of PREMs, such as satisfaction scales, provide insight into the patients’ experience with their care or a health service. There is increasing international attention regarding the use of PREMs as a quality indicator of patient care and safety.