A Perspective on the use of patient-reported experience and patient-reported outcome measures in ambulatory healthcare

Abstract

Introduction:

Patient-reported experience measures (PREMs) are patient reports about their healthcare, whereas patient-reported outcome measures (PROMs) are reports about their functioning and well-being regarding physical, mental, and social health. We provide a perspective on using PREMs and PROMs in ambulatory healthcare.

Areas covered:

We conducted a narrative review of the literature about using PREMs and PROMs in research and clinical practice, identified challenges and possibilities for addressing them, and provided suggestions for future research and clinical practice.

Expert Opinion:

Substantial progress in using PREMs and PROMs has occurred during the last half-century. Collecting and reporting PREMs to clinicians in ambulatory care settings has improved communication with patients, diagnosis, and treatment, which may improve patients’ health. Optimal use requires appropriate data analysis, minimizing implementation barriers, and facilitating interpretation of PREMs and PROMs in clinical practice. Also, formal structures and processes that include patient and family input into care improvement are needed (e.g., patient and family advisory councils as partners in co-design and coproduction of quality improvement). PREMs and PROMs have been used primarily in more affluent countries (e.g., the United States, Australia, United Kingdom, Netherlands, Japan, and Portugal), but this is expected to increase in many countries.

1. Introduction

Measures that focus on whether clinicians order tests, prescribe medicine, and obtain positive intermediate outcomes (e.g., blood pressure control) are essential to high-quality ambulatory care. However, they do not provide information about patients’ experience with care or their perceived health, and they can encourage a focus on diagnostic study results rather than the patient. Patient-centeredness is increasingly emphasized as a quality-of-care objective [1].

Patient-reported experience measures (PREMs) and patient-reported outcome measures (PROMs) are increasingly used to assess the quality of ambulatory care. In contrast to patient satisfaction measures of ambulatory care that are influenced by values and expectations, PREMs focus on patients’ perceptions of their interactions with healthcare providers [2–5]. PROMs capture patients’ reports about their functioning and well-being in physical, mental, and social health domains [6]. PREMs and PROMs are patient-centered measures that provide crucial insights beyond traditional clinical measures and are essential to ensure that healthcare is safe and equitable [7]. Because they capture perceptions of care and how patients feel and function, these measures can inform treatment decisions, promote shared decision-making, and help identify areas for care improvement, resulting in beneficial changes within the ambulatory care setting and the broader healthcare system [8]. They can be helpful at all stages of patient-clinician encounters: (1) Identify/elicit the problem(s); (2) Discuss with the patient about planned action(s); (3) Enact action(s) and cocreate plan(s); (4) Action(s); and (5) Learn about the effects (IDEAL) [9].

Most of the work on PREMs and PROMs has been conducted in more affluent countries (i.e., those with high gross domestic product). Therefore, our review and expert perspective is based primarily on work done to date in some countries, particularly studies conducted in the United States (U.S.) and some studies in countries such as Australia, the United Kingdom, the Netherlands, Japan, and Portugal. As a result, the thoughts in this review may not generalize to countries with limited or no experience collecting PREMs and PROMs.

This article represents an expert perspective based on a subjective examination and critique of the literature (narrative review). Readers are encouraged to explore different opinions and examine literature other than that cited here. We aimed to promote critical thinking and discussion about using PREMs and PROMs.

Below, we provide thoughts on the current state of research and clinical practice about PREMs and PROMs, discuss the implementation challenges, and offer suggestions for the future. While there are potential benefits of using PREMs and PROMs in ambulatory care, many implementation challenges exist, such as data collection cost, administration burden, and ensuring sufficient representation of patients (e.g., age, race, ethnicity, language, chronic conditions, and payor). We provide opinions on the state of science regarding using PREMs and PROMs in research and clinical practice. We discuss efforts to overcome these challenges and offer suggestions and expectations for use in future research and clinical practice.

2. PREMs in ambulatory healthcare

PREMs are valuable tools for assessing and improving the quality of ambulatory healthcare. They provide a patient-centered perspective on care experiences, enabling a comprehensive evaluation of healthcare delivery. Applications of PREMs in ambulatory care include:

• Informing public reporting: Sharing PREM data with the public can increase transparency and accountability in healthcare [10].

• Assisting patient decision-making: By providing information on provider performance, PREMs can empower patients to make informed choices about their care [11].

• Comparing performance across providers and settings: PREMs can be used to benchmark performance and identify best practices [12].

• Driving pay-for-performance initiatives: Linking payment to performance based on PREMs can incentivize providers to prioritize patient experience [13–15].

• Guiding interventions: PREMs can help identify areas where targeted interventions can improve patient outcomes [16].

• Monitoring quality improvement efforts: Tracking PREMs over time can assess the impact of quality improvement initiatives [17].

• Evaluating the impact of interventions: PREMs can help measure the effectiveness of interventions to improve patient experience [18–19].

2.1. Research

PREMs are intended to complement other quality measures and are associated with better technical quality of care [20]. For example, significant positive associations have been observed between patient experience measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Medicare survey and receipt of evidence-based guideline-concordant cancer treatments, suggesting a link between positive experiences and better technical quality of care [21]. Better patient care experiences are associated with better adherence to recommended prevention and treatment processes, better clinical outcomes, and less use of healthcare resources [22].

Assessing Disparities in Patient Experiences with Healthcare with CAHPS Surveys

The most often used measures in the CAHPS Clinician and Group Survey are the overall provider rating and communication composite [18]. This survey is mainly used to evaluate interventions to improve patient experience and examine associations between various factors (e.g., organizational climate, physician empathy) and patient experience [19]. Notably, this survey has been used to assess disparities in patient experience across different provider groups.

The Agency for Healthcare Research and Quality (AHRQ) maintains large research databases for various CAHPS [23], and CAHPS Medicare data has been linked with the Surveillance, Epidemiology, and End Results (SEER) cancer registry [24]. These datasets have been used to assess disparities in care. For example, an analysis of 28,354 adult CAHPS health plan surveys found that racial/ethnic minority groups, except for Asians/Pacific Islanders, reported experience with care similar to non-Hispanic Whites [25]. An analysis of Medicare surveys completed by 1,326,410 non-Hispanic Whites and 40,672 Asians supported the measurement equivalence of the CAHPS Medicare survey measures [26]. Also, racial and ethnic disparities in patient experience (measured by adult Medicaid CAHPS) were found among nonelderly Medicaid managed care patients, primarily within the same plan [27].

2.2. Clinical Practice

Quality improvement (QI), as defined by AHRQ, involves systematic efforts to improve patient care delivery [28]. QI requires significant organizational commitment, resource allocation, and stakeholder engagement, including patients and families. Sustaining improvements over time remains a challenge despite organizational changes and monitoring efforts [29].

A qualitative study explored how CAHPS Clinician and Group Survey data were used in a patient-centered medical home transformation [30]. Practice leaders employed the data for various purposes, including:

• Focusing QI efforts for PCMH transformation

• Maintaining a focus on patient experience

• Monitoring site-level trends and changes

• Identifying, analyzing, and monitoring areas for improvement

• Providing provider-level performance feedback and coaching

A community-based health network implemented a shadow coaching program to enhance patient experience. Coaches encouraged positive provider behaviors and identified areas for improvement, primarily focusing on communication with patients. This intervention improved patient experience, but the gains diminished over time [31, 32]. Booster training was needed to regain the initial improvements [33]. Leveraging patient experience data and implementing targeted interventions can drive QI and enhance patient experience.

3. PROMs in ambulatory healthcare

Researchers have used PROM data to evaluate health outcomes associated with interventions and patient characteristics such as age, gender, and chronic conditions.

3.1. Research

Research on PROMs in ambulatory care is often observational because ongoing care processes are studied. For instance, CMS has administered the Medicare Health Outcomes Survey (HOS) to Medicare beneficiaries enrolled in managed care to monitor performance and stimulate quality improvement since 1998 [34–36]. A random sample of adults from each participating Medicare Advantage Organization with a minimum of 500 enrollees is asked to complete a baseline HOS each year. Baseline respondents are surveyed again two years later. HOS measures have been included in star ratings for Medicare Advantage quality bonus payments to plans since 2012. The HOS includes measures such as an 8-item Physical Functioning Activities of Daily Living scale that is reliable at assessing average or below physical function among Medicare recipients [37]. The National Cancer Institute linked the HOS data to the SEER cancer register [38]. This effort led to extensive research on factors associated with PROMs among adults with cancer in the U.S. [39–40]. For example, a recent study of older patients with esophageal cancer found that White patients had better self-reported physical and mental health than Hispanic patients’ pre-diagnosis but worse health than Hispanic and Asian patients’ post-diagnosis [41]. CMS has also mandated monitoring PROMs in specific conditions, such as those with end-stage renal disease and on dialysis [42].

Qaseem et al. [43] emphasized the need for data demonstrating relationships between PROMs and healthcare structure, process, and actionable outcomes by the accountable entity (e.g., group practices and health plans). Registry-based databases have been noted to create opportunities for patients and their families to get continuous real-time access to peer and professional support using curated, facilitated networks and for clinicians to access collaborative improvement networks [44]. These registries support various types of research, such as observational studies, n-of-1 experiments, augmentation of results from randomized controlled trials (RCTs), and identification of trial participants.

3.2. Clinical practice

PROMs have been less frequently used in clinical practice than PREMs. Evidence is mixed; however, several studies support using PROMs in ambulatory care. Wasson and James [45] suggested incorporating PROMs into ambulatory care can enhance physician attention to patient-centered concerns. Wasson et al. [46] found that the use of the Dartmouth Primary Care Cooperative Information Project (COOP) Charts increased the ordering of tests and procedures for women (52% vs. 35%; p < 0.01) but not for men (23% vs. 37%; p = 0.06).

Velikova et al. [47] randomly assigned 286 cancer patients of 28 oncologists to four groups: a PROMs-with-feedback group, a PROMs-only group, an attention-control group that completed PROMs, and a control group that did not complete PROMs. The attention-control and intervention groups had significantly better Functional Assessment of Cancer Therapy–General Questionnaire scores than the control group (p<0.01). However, there were no significant differences between the attention-control and intervention groups.

Gibbons et al. [48] conducted a meta-analysis of RCTs investigating the impact of providing PROM feedback to providers and patients. They concluded that feedback had little or no effect on general health perceptions, social functioning, pain, physical functioning, fatigue, and mental health. Based on 11 studies, the authors reported a standardized mean difference of 0.15 (< small effect size) favoring “qualify of life” outcomes of feedback interventions versus control groups. Only two of the studies in the meta-analysis reported positive effects of PROM feedback.

One study found that providing clinicians with symptom reports by patients with advanced solid tumors improved patient quality of life scores [49]. This yielded a standardized mean difference of 0.31 (small effect size) in EQ-5D-3L scores, favoring the intervention group. Another study found that providing feedback to patients and their mental health providers on ecological momentary emotional state assessments led to significant improvements in EQ-5D-3L scores [50]. The standardized mean difference was 0.73 (medium effect size). However, Gibbons et al. (2021) based this on the follow-up mean EQ-5D-3L scores from the Simons et al. [50] study: 0.32 (usual care) and 0.45 (intervention group). The difference in change from baseline to follow-up between the intervention and control groups was 0.10, and the regression estimated difference was only 0.08, representing a standardized mean difference of about 0.47 instead of 0.73, which was previously estimated.

3.3. Linkage of PREMs and PROMs

There are relatively few studies that have examined associations between patient experience with ambulatory care and health outcomes. However, one study demonstrated that better access to care was associated with significant improvements in self-reported physical and mental health among HIV patients, particularly among those with initially lower health scores [51]. Another study found that receiving necessary care was associated with a reduced risk of mortality among Black patients with colorectal cancer [52]. Marshall et al. [53] showed in cross-lagged analyses of the Medical Outcomes Study that baseline patient satisfaction with care was significantly related to mental health one year later. Doyle, Lennox, and Bell [54] concluded from their literature review that positive associations existed between patient experience and patient adherence to medical recommendations, health-promoting behaviors, and health (self-rated and “objectively” measured) outcomes.

PROMs in clinical practice have been found to improve clinician-patient communication and identify and manage symptoms [55]. An RCT showed frequent discussions of chronic nonspecific symptoms increased in the intervention group where PROMs were used [55]. Patients completed Patient-Reported Outcomes Measurement Information System (PROMIS) surveys before orthopedic surgery visits. Patients whose surgeons used the PROMIS measures during the visit were likelier to report that the provider spent enough time with them, rate the provider positively, and recommend the provider’s office to other patients on the CAHPS Clinician and Group Survey [56]. A study of veterans found that improved patient-reported communication with providers was associated with better SF-12 mental health summary scores [57]. However, an RCT of foot and ankle surgery patients found that patients who viewed and discussed their PROMIS scores with their surgeon were less likely to report “top box” positive responses to the CAHPS Clinician and Group Survey item about the provider explaining things in a way that was easy to understand and had less patient activation [58]. No associations between care evaluations and subsequent PROMs were reported in a study outside the ambulatory care setting [59].

Authors of a review of RCTs noted that systematic improvements in clinician-patient communication led to improved health outcomes (effect size = .11, p = 0.02) [60]. In another systematic review, more positive CAHPS communication scale reports were associated with better self-reported physical and mental health [61].

4. Conclusions

Based on the existing literature, we believe that using PREMs and PROMs in research and clinical practice poses no harm to patients [62], but their impact on patient health remains uncertain. Robust evidence supports the association between assessing PREMs and PROMs with improved patient-reported communication with healthcare providers and overall perceptions of ambulatory care. More information is needed about how and when PROMs and PREMs can best improve patient experience with healthcare, enhance patient awareness of symptoms and self-management, and improve health outcomes [63].

5. Expert Opinion on Future Directions for the Use of PREMs and PROMs in Ambulatory Healthcare

5.1. Research

The absence of consistent associations between using PREMs and PROMs and better downstream outcomes is partly due to difficulties interpreting observational data and the importance of adequate statistical modeling to identify potential causal effects [64–65]. This challenge is not unique to self-report data. For example, Kahn et al. [66] estimated the association between a composite quality of care measure derived from clinically detailed data abstracted from medical records and changes in the SF-12 physical component summary score. The ordinary least square regression estimate for the process of care was in the wrong direction and not statically significant: −1.41 (p = .188). To obtain an unbiased estimate of the influence of the process of care on change in the SF-12, it was necessary to use the structure of care (indicator variables for each physician organization) as an instrument for the process of care. When this was done, a better process of care was associated with a significantly smaller decline in SF-12 physical health over a 30-month observation window.

Inadequate data analysis can lead to misleading conclusions, as exemplified by the problematic conclusion by Fenton et al. [67] that good provider communication was associated with a greater mortality risk. Xu et al. [68] noted that medical care does not prevent or delay all deaths in their reanalysis of the Fenton et al. [67] data. The reanalysis showed that the CAHPS communication scale was not significantly associated with amenable mortality. A reanalysis of the data showed that only time spent by the provider with the patient was significantly associated with mortality, consistent with a study demonstrating that patients near the end of their lives often receive better patient-centered care and the fact that sicker patients receive more attention [69].

In another study, linear regression was used to assess whether the CAHPS Clinician and Group Survey global rating of providers was associated with changes in the PROMIS global physical and mental health scales [70]. No significant associations were found between global provider ratings collected on average 277 days after lumbar spine surgery and change in PROMIS global health scores from baseline to 1 year after surgery. However, the regression model controlled for baseline PROMIS scores and self-reported overall health and mental health items at follow-up, leading to potential confounding and uninterpretable results.

Another example of a flawed conclusion from inadequate analyses occurred outside of the context of ambulatory care. A principal component analysis of diverse measures, including patient experience, led to the erroneous conclusion that patient experience is a poor measure of hospital quality [71]. A reanalysis revealed that the indicators were multidimensional; the assumption of unidimensionality led to incorrect inferences about the validity of the patient experience measure [72]. Given these examples, journal editors need to be more vigilant about vetting unexpected and potentially misleading conclusions based on potentially biased or uninformed analyses of patient-reported measures

Most studies of differential item functioning (DIF) of PREMs and PROMs by patient characteristics such as gender, age, and race/ethnicity have supported measurement equivalence [73]. The widely used internal comparison method assumes DIF involves inconsistent group differences in item responses. This may not hold for patient experience items with a standard response scale, such as extreme or negative response tendency, governing all items equally. The authors of a recent study found that in contrast to the internal comparison method, DIF was detected when an external anchor (standardized vignettes describing provider-patient interactions) was used as the gold standard (Abel et al. [74]).

5.2. Clinical Practice

Clinicians need to have treatment options to address problematic PREM and PROM scores. Several barriers hinder the widespread adoption of PREMs and PROMs in ambulatory care, including data collection costs, skepticism, resistance to change, knowledge gaps, time constraints, and lack of managerial support [75]. The Dartmouth COOP chart system [76–77] and other initiatives, such as those proposed by Sipma et al. [75], have sought to address these challenges by simplifying data collection, increasing provider understanding, and securing management support. Greenhalgh et al. [78] argued that more attention should be given to the mechanism linking PROM assessment to outcomes to maximize its impact on clinical decisions. Others have offered practical suggestions for using PROMs to monitor symptoms [79], clinical practice [80], and palliative care [81]. Implementation science can be used to understand better system processes, resources, and capacities required to support research evidence [82].

Reevaluating the content of existing PREM and PROM survey instruments, such as the CAHPS Clinician and Group Surveys, can help identify and address gaps in patient-centered care measurement [83–84]. Interviews with high-performing physicians suggest potential gaps in the CAHPS Clinician and Group Surveys, such as nonverbal communication, greeting patients, and tracking patient information [85]. It could be fruitful to assess patients’ perceptions of specific verbal and nonverbal behaviors (greeting the patient, leaning forward in a chair, tracking personal information about the patient, asking about other family members) and developing new items to capture what these specific behaviors represent to patients (e.g., listening attentively, seems to care about me as a person, empathy). There is also interest in assessing patients’ perceptions of patient safety [86]. Open-ended questions provide valuable insights into patients’ experiences and preferences [87–90].

5.3. Use of PREMs and PROMs in Ambulatory Healthcare in Five Years

A recent review found that most PREM and PROM research originates from the US, with significant contributions from Europe and the UK [12]. PREM and PROM usage in ambulatory healthcare will likely grow and expand to more countries. Increasing calls for more patients and public involvement in research and clinical practice ensure that the measures reflect what is important to patients [83, 91]. There is accelerating momentum toward patients’ co-creation as equal partners [92].

Due to declining survey response rates, the number of modes of survey administration offered to patients has expanded. Web administration alone yielded a response rate of 50% to patients who were members of a large health maintenance organization [93]. Sequential mixed-mode data collection involves administering surveys in multiple modes and can improve response rates [94–95]. CMS recently added web administration to mixed-mode data collection for CAHPS Medicare surveys. However, internet use is less prevalent among racial and ethnic minorities, those who are less educated, and those with lower incomes [96]. Underrepresented groups, such as racial and ethnic minorities and low-income individuals, are often poorly represented in research and may experience worse health outcomes, as evidenced by lower PROM scores [97–99].

Evidence shows that the odds of response are lower for longer surveys [100], and shorter surveys have been suggested to improve response rates [83, 101]. One strategy to potentially enhance response rates is to send a brief follow-up survey to non-respondents. Computer-adaptive test (CAT) administration has been proven to be an efficient way of collecting PROM data [102]. Based on one’s response to the first item, an estimate of the PROM scale score is made, along with an estimate of the standard error of the score. The subsequent items administered are deemed most likely to provide additional information about the individual’s location on the scale. The value of a CAT is seen when there is a large item bank calibrated using item response theory to select the optimal items for each person [103]. An item bank can also be used to identify short-form instruments. While CAT can be effective for PROMs, its applicability to PREMs is limited due to the fewer items and focus on higher-level care units. Incentives, such as monetary rewards or gift cards, can improve survey response rates [93], but resource constraints and potential bias concerns often limit their use.

While a moderate number of patient surveys can yield adequate (e.g., 0.80) reliability, individual patient experiences can vary significantly over time [44, 68, 104–105]. The Medical Expenditure Panel Survey data [68] showed that among those in the quartile with the most positive experiences at one wave of data collection, 43% were in a lower quartile one year later. The weighted kappa (with squared [quadratic] weights) between round 2 and round 4 quartiles was only 0.49.

The primary goal of healthcare is maximizing patients’ health. CMS has implemented a national policy to use PROMs to evaluate total hip and knee arthroplasty [106]. The percentage of patients who achieve a “substantial clinical benefit” on the Hip dysfunction and Osteoarthritis Outcome Score for joint replacement and the Knee injury and Osteoarthritis Outcome Score for joint replacement is assessed. Voluntary reporting began in 2023, mandatory reporting will start in 2025, and outcomes will be tied to hospital payments in 2028. We project that there will be increased attention to evaluating change in PROMs at the individual patient level [107–109].

PREMs and PROMs will be used more frequently to achieve value-based healthcare [63]. Teisberg et al. [110] (p. 682) recommended that medical schools include “education on the principles and implementation of value-based healthcare throughout the undergraduate medical curriculum to prepare their graduates for the transformation to value-based healthcare as they enter the physician workforce.” Hyland et al. [98] noted that the cost-effectiveness of collecting PROMs in routine clinical practice is unknown because cost is rarely reported.

Nelson et al. [8] (p. 2) suggested that:

“Registry based learning systems could unite patients, clinicians, and researchers to strive for, and ultimately coproduce, optimal health, high value services, and new knowledge that can be rapidly deployed to benefit individual patients and the public. Today’s registries have brought us a long way in improving healthcare; tomorrow’s registries, as patient-centered learning systems, could bring us even further.”

Preference-based PROMs are designed to integrate across health domains to produce a single summary score for different health states anchored relative to ‘dead’ (score of 0) and ‘perfect health’ (score of 1). This single score can be the denominator in a cost/utility ratio that allows comparisons of the value of different possible interventions. For example, 3,234 adults with impaired glucose tolerance at risk for Type II diabetes were randomized to placebo, metformin, or lifestyle modification [111]. Study participants completed the self-administered version of the Quality of Well-Being Scale [112] before randomization and annually for the subsequent three years. The lifestyle modification was more expensive ($27,065 total cost in 2000 U.S. dollars) than metformin (total cost of $25,937), and both interventions yielded more quality-adjusted life years than the placebo. However, the lifestyle modification produced 0.05 more quality-adjusted life years and better value than metformin.

Continuing calls for assessing healthcare value are expected to stimulate more routine assessments of cost and outcomes. This will allow payers and providers to compare the value of different treatment options. However, the operationalization of value is more comprehensive than cost-effectiveness or cost-utility. Equity and ethical issues are increasingly included in decision-making [113]. A comparison of different approaches will be needed in the future.

We believe PREMs and PROMs can revolutionize healthcare by providing valuable insights into patient experiences and outcomes. However, there are notable implementation challenges. Future research should focus on developing robust data collection and analysis methodologies, exploring innovative approaches to engage patients in the measurement process, and addressing disparities in access to care. By overcoming these challenges and embracing the opportunities presented by PREMs and PROMs, it may be possible to produce healthcare systems that are truly patient-centered and value-driven. Readers may disagree with our assessment of the PREMs and PROMs literature, and we encourage the expression of different perspectives.

Article Highlights.

• PREMs and PROMs are used frequently in ambulatory healthcare research.

• Despite suggestions to the contrary, the use of PREMs and PROMs does not harm patients. Rigorous statistical analyses of observational studies are required to make correct inferences about the impact of PREMs and PROMs on health outcomes.

• The use of PREMs is widespread in clinical practice, but PROMs have been used less frequently.

• Collecting and reporting PREMs to clinicians improves communication and facilitates the identification of health problems and the treatment of patients.

• Collecting and reporting PROMs to clinicians may improve patient-reported health outcomes.

• Further efforts are needed to present patient-reported data in an actionable way for ambulatory healthcare providers. Including patient and family input about care experiences and needed improvement is essential.