New Evidence on What Works in Effective Public Reporting

Abstract

Objective

To describe the current state of the public reporting field and provide guidance to public report producers based on the evidence.

Principal Findings

Public reports should address the questions and priorities that consumers actually have; present information credibly and in a way that is understood by the intended audience; reach the intended audience; and enable consumers to act on the information.

Conclusions

Public reports have advanced greatly in recent years, but there remains much room for improvement. Report producers should continually evaluate their reports and apply the latest evidence to maximize their usefulness and impact.

Information and tools to inform consumers about the quality and cost of care from competing health care providers have never been more plentiful, or more needed. The Centers for Medicare and Medicaid Services (CMS) and many state governments and regional coalitions produce a growing list of public reports on the quality of care by hospitals, physicians, nursing homes, and other providers. Not‐for‐profits such as Leapfrog and Consumer Reports also produce national reports, and for‐profits such as Yelp and Angie's List are more recently getting into the act. While public reports for consumers can fulfill multiple purposes, such as supporting consumers' right to know and motivating providers to improve, the major impetus for these efforts is to help consumers make more informed provider choices and care decisions.

Given the huge variations in safety and quality among providers in virtually every health care sector, these public reports have tremendous potential to direct consumers to providers that offer higher quality and more affordable care, resulting in better outcomes and rewarding high‐performing providers. Choosing the right provider can literally be a matter of life and death.

The cost stakes for consumers are also large and growing. Employers and insurance marketplaces across the country are favoring high‐deductible, “consumer‐driven” health plans. These plans offer lower premiums but rely on consumers to make savvy decisions to make the most cost‐effective decisions and avoid paying deductibles that many of them cannot afford. In this high‐stakes environment, tools that allow people to compare doctors, hospitals, nursing homes, and other providers on quality and price are desperately needed.

Despite public reporting's promise, historical research has not displayed a strong track record, with few studies showing consumer migration to high‐quality providers following public reports. While some attribute this finding to lack of consumer interest, others hypothesize that the fault lies not with the consumer but with the reports themselves.

In 2012, the Agency for Healthcare Research and Quality (AHRQ) partnered with CMS to fund a family of 17 grants designed to build the science of public reporting. These 17 research teams set out to improve measurement and reporting in a wide variety of sectors, from maternity care and pediatrics to nursing homes and hospice. This journal issue brings together findings from six of these studies (see http://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/sciencepubreport/index.html for a full list of study descriptions and resulting publications from the 17 grants).

A fundamental premise of this initiative was that public reporting is a complex intervention, and evaluations that simply ask “does it work” are inadequate or misleading, and do not provide evidence on how to make the intervention work better. The more difficult but rewarding questions to be asked are “when, how, and why does it work—and for whom?” For public reports to achieve their intended goals, the design, content, and dissemination all need to work together, and failure of any one component can drastically diminish the impact. Public reports are complex interventions, where many things have to go well to achieve the desired outcomes. The articles in this issue present findings on how to achieve four interrelated, basic, but often absent “musts” for successful public reporting. Reports must (see Figure 1):

Figure 1.

Key Steps in Achieving Successful Public Reporting for Consumers

1. Address the questions and priorities that consumers (including subgroups of consumers) actually have.

2. Present information credibly and in a way that is understood by the intended audience.

3. Reach the intended audience.

4. Enable consumers to act on the information.

Address the Questions and Priorities That Consumers Actually Have

One might expect that the first step in creating reports for consumer use would always be to determine what consumers want to know. In fact, few past report creators have done this, instead relying on available data and measures selected by health care professionals, and attempting to make these relevant to consumers. Hefele et al. (2016) used focus groups and interviews to identify what information consumers wanted in selecting a nursing home and found that they wanted detailed information on the facility, policies, staff, and residents; a sense of the nursing home gestalt; and feedback and reviews from residents and their families—information rarely available on existing reports.

An equally important finding—one underscored in the articles by the Maurer et al. (2016) and Schapira et al. (2016) teams—is that consumer information needs differ by a number of important consumer characteristics, including education, age, insurance status, race, ethnicity, care history and needs, and language ability. The lesson here is that it is important for developers to seek out the needs of multiple categories of consumers, and then attempt to develop reports for specific audiences, or provide the ability to customize reports based on consumer preferences, interests, and abilities.

Present Information Credibly and in a Way That Is Understood by the Intended Audience

It is not enough to respond to what consumers want to know—reports must answer their questions accurately, credibly, and understandably. For a start, the information presented to consumers is only as valuable as the underlying data and methods. The articles in this issue show that the methodology for gathering data, calculating measures, and presenting the resulting information to consumers can be tricky and complex. The Silber et al. (2016) article identifies ways to improve common methods of rating hospital performance to address this and other methodological issues.

While ratings calculations need to become more sophisticated, the presentations themselves need to become simpler and more accessible—a challenging task, as consumers are rapidly changing in the way they access information, but many continue to face the barriers of low health care literacy and numeracy. Reports have evolved from sprawling paper statistical tables to interactive web‐based tools that allow easy searching, sorting, inclusion of contextual help, and accessible formats. However, consumer testing remains the best way to know if a given approach is working. In their article, Grob et al. (2016) describe a robust method for collecting patient narratives, which engage consumers and may help them interpret traditional survey‐based patient experience measures.

One compelling methodological and display need concerns the presentation of information on price. When AHRQ began this initiative, there was little provider price transparency, but now efforts abound at the state and national level, and employers and health plans are deploying their own price comparison tools. However, measures of price and quality often do not line up, and much work needs to be done on how best to link cost, price, and quality information to ensure that consumers are able to identify high‐value providers. In the absence of quality information, consumers often identify high‐priced providers as higher quality than more affordable providers.

Reach the Intended Audience

Even a perfectly designed report responding to salient consumer questions will have little impact if it does not reach the consumer. The qualitative work performed by these researchers indicates consumers have a range of familiarity with public reports that depends on both the specific report and consumer characteristics such as education. Commercial organizations such as U.S. World & News Report, Healthgrades, and Consumer Reports invest heavily in marketing and promotion compared with government and small local nonprofit reports, often resulting in greater market share. In this issue, Huesch et al. (2016) show how different social media tools can be used to expand the reach of report producers with only a modest marketing budget.

Enable Consumers to Act on the Information

The final and perhaps most critical step in this enterprise is for consumers to actually review and act on the information in public reports. There are many situations in which one cannot expect such an outcome. Not everyone has a choice of provider, and many medical visits (e.g., emergency care) do not allow time for information seeking. But many provider encounters—having a baby or entering a nursing home, for example—do allow time for the consumer to become informed. Werner, Konetzka, and Polsky (2016) found that CMS's introduction of a five‐star summary rating for nursing homes significantly reduced use of one‐star homes and increased use of five‐star homes, suggesting that there is promise for reports that present clear, actionable signals to consumers. To the extent that consumers are in fact reviewing and responding to public reports, it will be even more important to get the design and content right.

Looking Ahead: The Future of Public Reporting

The articles in this issue, and the other publications resulting from these 17 grants, provide critical information to fill the gaps in what we know about how to build evidence‐based, useful, and effective public reports for consumers. As the studies published here and elsewhere show, we have made some breakthroughs in understanding the four steps in report conception, development, deployment, and use, but each study has also raised many new questions.

Even as current research progresses, public reporting itself is evolving quickly. Researchers need to constantly be looking ahead, getting the evidence today to answer tomorrow's questions. Here are just a few examples of the need for new research:

• Personalized reporting: Public reporting is becoming a more nuanced affair, just as genomics and big data are personalizing medicine. As noted earlier, different subsets of consumers will differ in terms of what information they want to see in a public report, based on their socioeconomic status, stage in life, or clinical condition. Price transparency tools are also becoming tailored to consumers' insurance profiles, helping them understand how much they will pay out of pocket and the progress they have made toward meeting their deductible.

• Technology: Rapid technological change is altering the landscape of public reporting. Electronic health records and health information exchanges are opening vast new troves of data for secondary uses. Smartphones and tablets are changing the way people interact with the Internet and their environment, providing an opportunity for quality and cost information to be presented when and where consumers need it. With many paths to information and heavy competition for consumer attention, public report producers need to take advantage of new technology and clearly establish and market their value.

• Delivery system structure: As hospitals, physician offices, and other providers merge and take on different forms, the unit of decision making for the consumer is changing. If the unit of analysis in public reports differs from the unit of decision for consumers, reports will obviously be less effective. Identifying the right measures, data sources, and report designs for these new delivery systems will be a critical challenge for researchers and report developers.

Conclusion

Transparency through public reporting is here to stay. There is no perfect path forward, but the research featured in this issue provides hope that by thoughtfully incorporating evidence‐based best practices, public reports can be continually improved to more accurately capture the quality of care and effectively engage their target audiences.

Supporting information

Appendix SA1: Author Matrix.