Impetus of US hospital leaders to invest in patient-reported outcome measures (PROMs): a qualitative study

Abstract

Objectives

Though hospital leaders across the USA have invested significant resources in collection of patient-reported outcome measures (PROMs), there are very limited data on the impetus for hospital leadership to establish PROM programmes. In this qualitative study, we identify the drivers and motivators of PROM collection among hospital leaders in the USA.

Design

Exploratory qualitative study.

Setting

Thirty-seven hospital leaders representing seven different institutions with successful PROMs programs across twenty US states.

Methods

Semistructured interviews conducted with hospital leaders. Transcripts were analysed using thematic analysis.

Results

Leaders strongly believe that collecting PROMs is the ‘right thing to do’ and that the culture of the institution plays an important role in enabling PROMs. The study participants often believe that their institutions deliver superior care and that PROMs can be used to demonstrate the value of their services to payors and patients. Direct financial incentives are relatively weak motivators for collection of PROMs. Most hospital leaders have reservations about using PROMs in their current state as a meaningful performance metric.

Conclusion

These findings suggest that hospital leaders feel a strong moral imperative to collect PROMs, which is also supported by the culture of their institution. Although PROMs are used in negotiations with payors, direct financial return on investment is not a strong driver for the collection of PROMs. Understanding why leaders of major healthcare institutions invest in PROMs is critical to understanding the role that PROMs play in the US healthcare system.

Strengths and limitations of this study.

• The major strengths of this study include the broad sample of executives from across US healthcare services and the many in-depth qualitative interviews.

• Limitations include the fact that subjects were selected on the basis of published reports and recommendations from experts in the field and duration and volume of patient-reported outcome measure collection.

• Only seven US institutions serving 20 states were interviewed, which may limit generalisability.

Introduction

Patient-reported outcome measures (PROMs) are standardised and validated questionnaires assessing a patient’s health condition based on his or her own report and without amendment or interpretation by a clinician or anyone else¹ Traditionally, PROMs have been applied in research and clinical trials, but recently there has been increasing use in patient care and clinical decision making.^2–5 PROMs can help physicians choose the optimal procedures for patients,⁶ elicit sensitive information from patients that otherwise may be ignored,^7–9 and improve patient experience.^{10 11} A more controversial application is the use of PROMs to evaluate healthcare providers’ performance, especially in the context of tying outcomes directly to value-based payments.^{12 13}

Progress in this field has been hindered by data collection and analysis barriers.¹⁴ Perhaps even more importantly, acceptance of PROMs by clinicians and adequate training to collect and interpret PROMs data are critical to their meaningful adoption.^{15 16} Despite increasing clinical applications, PROMs still suffer from limited acceptance and adoption by practising physicians.⁸ This is partly due to the challenges of consistently collecting data and making it easily accessible to clinicians.^{17 18} Another critical obstacle is the significant time and resources required to engage healthcare personnel in using PROMs and generating the physician buy-in to drive meaningful adoption.^{19 20} In addition, some argue that PROMS have limited impact on quality of care and that there is insufficient evidence supporting routine outcome monitoring in terms of improving outcomes or management.^21–23

These obstacles do not appear to stop hospital leaders from investing in PROM programmes. In fact, 38% of USA healthcare providers are currently collecting PROMs in their organisation and 17% plan to do so within the next 3 years. This means that more than half of providers in the US may have PROMs data available for clinical use in the near future.²⁴

An important development in US healthcare in recent decades is the shift away from fee-for service reimbursement models towards value-based payment models.¹⁵ In this new paradigm, there is a strong need to measuring healthcare quality. Currently measured clinical outcomes such as mortality and hospital readmission are difficult to interpret, as there are many factors outside the control of the healthcare system. Other outcomes such as compliance with lab draws or medications are process measures and may fail to capture the reason patients seek care in the first place.²⁵ PROMs are thought to enable quantification outcomes that matter most to patients, thus making it a potential tool to identify value.^{15 20} Consequently, payors have identified PROMs to become increasingly more relevant in the next 10 years. Indeed, there are already various Center for Medicaid/Medicare programmes that mandate PROMs collection, such as US dialysis centres.^{26 27}

In this context, there is a need to understand what primarily drives healthcare executives and clinical leaders to invest in PROM programmes. We sought to better understand their motivations and goals by conducting a qualitative study and interviewing leaders in seven leading US healthcare organisations that have been systematically collecting PROMs. The findings will be relevant to both policy-makers who seek to promote patient-centred care and organisational leaders and clinicians who are planning to implement PROM programmes in their own healthcare systems.²⁸

Methods

Study design and sample

An exploratory qualitative study was conducted to identify the motivations of hospital leaders to establish and invest in PROM programmes. A combination of in-person or video-based interviews (Zoom, San Jose, CA) were conducted from February 2020 to June 2020 with 37 leaders of seven healthcare organisations represented in 20 different states across the USA. Inclusion criteria for PROMs programmes include (1) recommendations from PROMs experts and significant presence in academic literature (2) programmes with at least 3 years of consistent PROMs collection and (3) collection of at least 25,000 PROMs each year. The sample was intended to be representative of various geographic regions across the US and to include different types of healthcare organisations, including health systems, academic hospitals and specialised hospitals.

Convenience sampling and snowballing techniques were employed to recruit participants. Participants had to be either chief-level executive (e.g., chief executive officer (CEO), chief medical officer (CMO)), PROMs directors, clinician leaders (eg, chair or vice chair of departments) working specifically with implementation of PROMs, or directors of data warehouses and/or data analytic teams that are involved with PROMs.

A semistructured interview guide was developed from a combination of literature review and expert input from the Mass General Brigham PROMs group and a qualitative methods expert (online supplemental digital content 1). This interview guide was iteratively refined throughout the interviewing process with the guidance of qualitative research expert (BB). For example, early participants discussed leveraging PROMS as a marketing tool; we thus included a question about PROMS as marketing tools in subsequent interviews.

Interviews were conducted by CM and DM, both surgeons with qualitative interview experience. Participants were emailed to request an interview and to provide demographical data about their institution.

Data collection and analysis

All interviews were recorded, transcribed, coded line by line, and analysed with thematic analysis using NVivo V.12 (QSR International, Burlington, Massachusetts, USA). First, a codebook was developed using a priori codes based on the interview guide. The first 10 interviews were coded jointly, and inductive codes were generated based on emerging concepts. After these first 10 interviews, the codebook was finalised and applied to the remaining interviews, which were independently coded by four authors (CM, DM, CMS, AV). Coding conflicts were resolved in discussion with the qualitative research expert (BB). Interviews continued until theme saturation was achieved. This was determined through iterative discussion to the point at which subsequent interviews were no longer yielding novel concepts or themes.²⁹

Patient and public involvement

There was no patient or public involvement in this study.

Results

In-person or virtual interviews were conducted with 37 leaders of seven healthcare organisations with major PROMs programmes (table 1). Interviewed leaders included chief-level executives, PROMs directors, department chairs or division chiefs and director of relevant PROMs Data warehouses (table 2).

Table 1.

Characteristics of the seven participating institutions of the study

Institution	1	2	3	4	5	6	7
No of beds	2000	11 000	1000	3500	150	1000	200
No of PROMs collected in 2020	1 million	600 000	600 000	2 million	50 000	650 000	100 000
Type of organisation	Private Hospital	Hospital system	Academic hospital	Hospital system	Specialty Hospital	Academic hospital	Specialty hospital
Location	Mid-west/ South-east	West	West	Northeast	Northeast	Midwest	Northeast
Systematic collection of PROMs since	2015	2015	2012	2015	2017	2015	2017
Overall budget ($)	1 billion	25 billion	4 billion	10 billion	400 million	4 billion	1 Billion
Interviewee Titles	CLEs, dept. chairs, PROMs director	CLE, dept. chair, PROMs director	CLEs, dept. chairs, PROMs Director, DDW	CLEs, dept. chairs, DDW	CLEs, dept. chairs, PROMs directors	Dept. chairs, DDW	CLE, DDW

All numbers are approximate to ensure anonymity.

CLE, chief-level executive; DDW, director of data warehouse; PROMs, patient-reported outcome measures.

Table 2.

Interviewee titles

	n-count
Chief-level executive (eg, CEO, CMO, COO)	11
PROMs director	6
Department/division chief	9
Director of data warehouse/data analysis	11
Total	37

CEO, chief executive officer; CMO, chief medical officer; COO, chief operating officer; PROMs, patient-reported outcome measures.

Hospital leaders at institutions that have successful PROMs programmes identified two different motivations to invest in PROMs: culture-related drivers and value-related drivers (figure 1). Interestingly, our data also reveal doubt among hospital leaders regarding whether PROMs will be a compelling metric to measure clinician performance from a payer’s perspective. Below we discuss each of these themes and provide representative quotes.

Figure 1.

Why leading hospitals collect patient-reported outcome measures (PROMs).

Culture-related drivers

The right thing to do

Across different healthcare systems, interviewees frequently described the activity of collecting PROMs as something ‘right’ or ‘good’. Several executives specifically stated it was ‘the right thing to do’, one cited ‘a moral imperative’ at the institution level to do so. Interviewees often referred to the value-based healthcare framework of Professor Michael Porter at the Harvard Business School.^{30 31} One chief-level executive said, ‘if you are trying to figure out value in healthcare, value as outcomes over cost, […] the question is whether you can measure the outcomes that are relevant’. The executive pointed to PROMs as being these ‘outcomes of relevance’—something that can help assess whether a patient’s symptoms, function or quality of life have in fact improved following a treatment. One PROMs director described the focus on PROMs as a movement within the organisation, stating: ‘we believe in value. It is almost faith-based’.

Interviewees also felt that PROMs offered a tool to better capture patients’ perspectives after medical interventions and to amplify their voices. One quality director characterised the use of PROMs as being able to expand the definition of quality: ‘So, we thought, what do patients seek when they look forward to getting care? Do they get better function? Do they get relief from pain, a better quality of life? Do they become more productive at work? […] PROMS help us to collect that in a standardized way from patients.’ Another PROMs director pointed to the signal that the action of collecting PROMs sends: ‘Patients believe that PROM data …, [are], collected to help patients. And I think it’s a covenant’.

Finally, interviewees minimised any immediate financial incentive to collection of PROMs. ‘You keep saying incentive, there’s no financial incentive here at our institution’, one medical director stated. Another chief-level executive stated, ‘there’s no [PROMs-related] money coming in’, admitting that PROMs may work a’little counter to bringing in any kind of financial benefit to the institution’, suggesting that the costs of PROM initiatives are greater than any associated financial benefits.

The culture of the institution and innovative leaders

Hospital leaders felt that early PROMs implementation was in part facilitated by an institutional culture that encouraged innovation. Some pointed to the institutional focus on research, others to a commitment to the biopsychosocial model or patient-centred care. This culture was conducive to the work of PROMs ‘visionaries’. As one medical director put it, ‘it’s always great to get the winning team, we have the Super Bowl team here, their vision—to take the [PROMs] initiative and put it into a quality program.’ Another PROM manager spoke of ‘the perfect storm’ of stakeholder collaboration as a catalyst for widespread implementation and acknowledged that having executive office buy-in set the tone for institutions; ‘the most important thing was that it was the right people [internal institutional champions backing the program].’ Some pointed to their institution’s ‘entrepreneurial sense’, which encouraged individuals to pursue new ideas.

Notably, the use of PROMs in research was not an important driver for leaders’ motivation to implement them in clinical work, although some centres had been and still were extensively collecting PROMs for research purposes. Several hospital leaders emphasised that PROMs data were intended for clinical applications, not research. ‘The primary goal of [collecting PROMs] is to take care of patients’, stated one PROM director.

Finally, a PROMs project manager described a growing recognition that health systems that pioneer systems-wide PROMs programmes tend to be more focused on quality than those that are not. Therefore, the PROMs group was given a ‘huge amount of leeway … to pursue strategies that essentially demonstrate value-based care but may not show an immediate [return on investment].’

Value-related drivers

Enables delivering high value care to patients

Hospital leaders felt that incorporating PROMs into routine evaluation could improve patient care in different ways.

First, PROMs were believed to improve shared decision-making on an individual level by helping to set patient expectations around whether one might benefit from a treatment or not. For example, by understanding how other patients with similar PROMs scores have improved from a medical intervention, patients are more informed to ‘make a decision that works best for them’. Some also believed that PROMs could directly boost patient experience by improving patient–provider communication and enhancing the patient’s sense of control over his or her health.³²

Second, PROMs were perceived to enhance quality improvement and systems-level learning. On an aggregate level, PROMs data were thought to help better define what interventions are truly effective and identifying best practices as well as appropriateness of care. This notion was typically tied to surgical procedures. ‘The big vision is no one’s father, aunt, uncle, or mother should have knee surgery without knowing whether or not it’s likely to make a difference and we can use the data to help inform that decision’, one programme manager said. A chief medical officer explained: ‘There’s a recognition that in order for us to make better decisions on who receives elective surgery, we need better [patient reported] data.’

Demonstrate value to payors

Interviewees stated that PROMs have particular relevance in capitated payment systems, as they may help identify unnecessary and low-value interventions. ‘We can save the health system, in general, money by not [inappropriately] operating on too many people’, a PROMs project manager said. A chief medical informatics officer claimed that accountable care organisations would make it ‘much easier for us to drive adoption of our Patient-Reported Outcomes program [as opposed to a fee for service reimbursement system].’

Also, interviewees consistently claimed that being a market leader in PROMs offered an opportunity to proactively define and dictate value to payors—or ‘shape that conversation’ as one PROMs project manager put it. Another one explained, ‘we convinced [the insurer] … to the tune of [millions] that they should make patient-reported outcome measures the actual quality slate.’ One leader described how the institution used ‘PROMs to help communicate the extent to which we value and pay attention to our quality as … from the patient’s perspective.’ He moved on to describe how collecting PROMs helped ‘demonstrate the quality of care’ that the institution provides and that it was used to ‘attract larger volumes of commercial pay patients and improve our payor mix’.

Some pointed to the fact that centres of excellence or bundled payment programmes increasingly require submission of PROMs, and by collecting them they were also in position to more easily get exclusive contracting arrangements with payors.

Demonstrate superiority over peers

Interviewees from all institutions confirmed that their PROMs programmes enhanced their institutions’ marketing and helped demonstrate their superiority over peers: ‘…we want to communicate that we are the premium destination that delivers great value to patients’ one director said. A programme manager claimed that by being in the PROMs ‘sphere’, the institution was able ‘to demonstrate, frankly, what a better care system we are compared with the competitors.’ One chief clinical officer pointed out that PROMs align well with the healthcare business in general: ‘So in a service business, the way you win is to have the best product at an affordable price, right? And, PROMs let us know if we have the best product’. This marketing benefit was thought to have a promotional effect regardless of whether competitors reported comparable metrics.

PROMs are questionably useful as performance metrics

Interestingly, the intention to use PROMs as performance metrics for payer contracts was not a major driver among most hospital leaders. Interviewees found that current concepts or ways of measuring improvement, like minimal clinically important difference (MCID), were too complex and too dependent on surrounding factors to play a key role in contracts with payors. ‘There is just so much that has to go into risk adjustment and the statistics to make that type of analysis sound’, a chief medical officer explained, pointing to the fact that patient characteristics may significantly confound PROM scores. Therefore, using specific thresholds for value would become ‘unfair’ to some providers depending on their patient population.

At one institution, however, ‘a work in progress’ was to develop PROM thresholds and show that these were applicable in a clinical setting, before convincing other providers to use them. In the long run, there was a plan to introduce the thresholds to ‘policy-making bodies’ like Centers for Medicare and Medicaid Services and the National Quality Forum.

All interviewees agreed that payors were ‘not there yet’ in terms of finding a way to apply MCID or similar thresholds in contracts. One leader even stated that, ‘I don’t think that will [ever] happen’, while others believed this was ‘years away’.

Discussion

In this study, we found that the main driver to implement PROMs for leaders in seven US centres was that it was the ‘right thing to do’. This strong moral imperative was influenced by the fact that all interviewed leaders reported that their centres have a tradition of patient-centred care, value-based healthcare and innovation. The fact that there was no immediate return on investment had little impact on the strategies to systematically collect PROMs. However, leaders did regard PROMs as a way to demonstrate their institution’s superiority in the market to payors and the public in general. They also used their PROM strategy proactively in their negotiations with payors and saw it as a way to dictate or define how value is measured in a provider–payor contract (figure 1).

Finally, PROMs were regarded to be a way to enhance quality of care through improved patient experience and data analysis. The use of PROMs as performance metrics was of less importance in terms of driving implementation.

Literature

There are very limited data on the impetus for hospital leadership to establish PROM programmes. A recent survey performed by NEJM Catalyst did show that improving patient experience and quality metrics were the more important drivers for implementation.²⁴ The same survey found that only 8%–16% reported ‘financial incentives’ and ‘market demand’ as reasons to collect and use PROMs. Our study further explores these findings by elucidating the motives of US hospital managers and executives.

Study implications

Our findings have implications for both institutional leaders and policy-makers across the USA. Implementation models are unlikely to succeed unless people in an organisation embrace ‘the why’ in terms of the importance of PROMs in the clinical setting.

Although the marketing impetus is strong among the early-adopting institutions, there is a general and clear understanding among leaders that PROMs will help improve healthcare. More appropriate care and less waste, improved patient–physician communication, and enhanced quality based on data analysis were mentioned during interviews. All these aspects may be particularly relevant in a capitated healthcare system where providers are financially responsible for excessive low-value care. That said, it is always a possibility that PROMs may highlight underutilisation of appropriate interventions, leading to increased costs.

Leaders reported that there is no immediate return of investment for the use of PROMs. Though they claim this does not deter them from investing in PROMs, this may prove to be more of an obstacle for other healthcare systems. The highly prevalent fee-for-service model drives volume, but not cost effectiveness. One may not observe widespread use of PROMs unless there is a change in policies regarding reimbursements and rewards for cost-effectiveness.

Another key obstacle seems to be that hospital leaders are doubtful that PROMs can be readily used as a performance metric. Of note, clinicians have voiced similar concerns.⁸ This hinders the development of reimbursement structures based on PROM score improvements. Interviewees downplayed the role of MCID to measure treatment quality in a provider–payor contract, claiming consensus about thresholds for different PROMs were not likely to be achieved in the near future. MCIDs in PROMs have been defined and validated in several different fields.^33–37 Our interviewees’ hesitancy in using MCIDs for performance measurement may reflect their lack of awareness of the rigorous science in this field. More aggressive dissemination of the MCID literature or more proactive engagement of hospital leaders may be necessary to bridge this disconnect. Also, a current strategy to drive collection should focus on encouraging collection of PROMs rather than creating benchmarks.

Nevertheless, PROMs seem to align with clinicians’ intrinsic purpose and motivation for pursuing medicine. PROM collection also seems to create a competitive advantage for a provider when marketing its organisation both in relation to payors, but also in terms of demonstrating its value to patients and the public. Finally, the faith among leaders that collecting PROMs is ‘the right thing to do’ shows that there is a deep motivation in the system to implement outcomes into clinical practice. This aligns well with the reports from physician champions in the field.6 7 19 24

Limitations and strengths

Our study is subject to limitations. First, our subjects were selected on the basis of published reports and recommendations from experts in the field, the duration of PROMs collection, and the volume of PROMs collection. Second, we only investigated seven US institutions. Although the institutions provided care across 20 different states, we may not have captured a representative group. Third, there is a risk that leaders could be reluctant to reveal PROMs strategies in a competitive healthcare market. All interviewees, however, appeared comfortable and frank in the interview process. We, therefore, believe the risk of this bias is low. Finally, we only interviewed institutions that were successful early adopters of PROMs. Consequently, we did not engage organisations that have attempted and failed to implement durable PROMs programmes. Their perspective warrants further investigation.

The key strengths of this study are the broad sample of executives from across the US healthcare services and the many in-depth qualitative interviews. The data provide a unique insight into why leading US organisations invest in PROMs.

Conclusion

Our study is the first to investigate why healthcare leaders in leading PROMs-collecting centres believe it is important to invest in PROMs. We show that these leaders have a strong belief that collecting PROMs is the ‘right thing to do’. They all believed their institution delivered superior care and used PROMs to demonstrate their institution’s superiority and be proactive towards payors in contract negotiations. However, there were no direct financial incentives driving the collection of PROMs. Most leaders did not think that PROMs can be effectively used as a performance metric.

Our findings provide a blueprint of why healthcare leaders invest in PROMs. This will hopefully guide others who wish to implement PROMs in clinical practice either at an individual or systems level. The findings are also relevant to policy-makers who may seek to incorporate PROMs in national-level policies.

Data availability statement

Data are available on reasonable request. The data will be available on request for up to 1 year after article’s publication.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study was approved by the IRB at the Harvard School of Public Health. The ID number is: IRB20-0090. Participants gave informed consent to participate in the study before taking part.