Abstract
Background:
The Agency for Healthcare Research and Quality held a research meeting on using Consumer Assessment of Healthcare Providers and Systems (CAHPS®) data for quality improvement (QI) and evaluating such efforts.
Topics Covered.:
Meeting addressed: 1)What has been learned about organizational factors/environment needed to improve patient experience? 2)How have organizations used data to improve patient experience? 3)What can evaluations using CAHPS data teach us about implementing successful programs to improve patient experience?
Key Themes:
Providers and stakeholders need to be engaged early and often, standardize QI processes, complement CAHPS data with other data, and compile dashboards of CAHPS scores to identify and track improvement. Rigorous study designs are valuable, but much can be learned and accomplished through practical organization-level studies.
Keywords: CAHPS, patient-centeredness, patient experience, quality improvement, surveys
1. Background
Health care organizations increasingly focus on improving patient and family-centered care experiences as part of quality improvement (QI) efforts. The National Academy of Medicine (NAM) defines patient-centered care as ‘providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions.’(page 6, 40) [1] Patient-centeredness is one of NAM’s six aims of quality and has become a standard goal for general clinical practice [2,3]. Many organizations measure patient experience using Consumer Assessment of Healthcare Providers and Systems (CAHPS®) surveys. The adoption of CAHPS surveys and their use for QI, public reporting, value-based purchasing, and accreditation programs has led to a growing interest in rigorously tested, evidence-based strategies for improvement.
CAHPS surveys are reliable and valid measures of patient-centered care [4] that are widely recognized as the gold standard for patient experience assessment. They can be used to monitor improvements in care experiences and make comparisons across organizations [5–8]. CAHPS surveys have been used in pay-for-performance programs such as the Centers for Medicare and Medicaid Services (CMS) Hospital Value-Based Purchasing (VBP) Program [9], Merit-based Incentive Payment System (MIPS) under the Quality Payment Program (QPP) [10], and the Medicare Star Ratings[11].
However, implementing, sustaining, and evaluating efforts to improve patient experience is challenging. First, these efforts require leadership at multiple levels of the organization, time, resources, engagement across stakeholders including patients and families, commitment to identifying inefficiencies, and effective improvement strategies. In addition, sustaining improvements has proven difficult [12,13]. While organizational changes and a commitment to monitoring and tracking metrics can help maintain gains in patient experience scores [7], there is little evidence about how best to maintain system changes.
The ways in which improvement projects are typically implemented often complicate efforts to evaluate their effectiveness. Many QI strategies are based on evidence or effectiveness from a single organization or region. This is because often QI efforts involve small pilots to learn about barriers and facilitators, and then more widescale implementation to spread changes and adapt to variations in care delivery. Most assessments of QI efforts are case studies with limited generalizability. Tracking process and outcome measures for QI can be difficult. Changes resulting from interventions to improve patient experiences through QI can take time to become evident. Additionally, organizations often implement multiple QI initiatives simultaneously, thus complicating the evaluation of intervention effectiveness and identification of the source of change.
Given these challenges, Agency for Healthcare Research and Quality (AHRQ) dedicated its annual research meeting in 2020 to learning how different types of organizations implement and evaluate QI efforts focused on patient experience, with a focus on the use and utility of CAHPS data in these efforts.
1.1. Meeting objectives
This virtual research meeting on 7 October 2020 convened an invited group of CAHPS survey users, researchers, health care organization leaders, and policymakers from both government and the private sector to explore three major questions:
What has been learned from prior research about the organizational factors and environment needed to improve patient experience?
How have health care organizations used CAHPS survey results and other data to improve patient experience?
What can evaluations using CAHPS data teach us about implementing successful programs to improve patient experience?
1.2. Meeting structure
The meeting participants included 24 panelists and 70 invited attendees. The meeting included moderated panels with presenters from different health care settings (i.e. health plan, medical group, hospital, and other health care settings such as dialysis facilities and hospice). The first panel focused initially on research findings on improving patient experiences, with subsequent presentations discussing effective implementation of patient experience improvement strategies in the different settings. The second panel focused on methods of implementing and evaluating patient experience improvement using CAHPS data, and included presentations of examples of patient experience improvement initiatives using CAHPS surveys to measure effectiveness (see Appendix A for a list of the panelists and Appendix B for the meeting agenda).
Below, we summarize the panel presentations and main themes that emerged. We conclude with a summary of recommendations.
2. What do we know from research about why and how to improve patient experience?
The meeting began with an overview by Susan Edgman-Levitan summarizing what is known from prior research. She noted that the benefits of positive patient experience are driving many health care organizations to invest more in patient- and family-centered care. Better patient experience is associated with less health care utilization, adherence to recommended prevention and treatment, patient safety, and better outcomes [14–17]. Patients with positive care experiences are also more likely to return to the same hospital and ambulatory settings for health care needs, retain their health plan, and voice fewer complaints [18,19]. Evidence suggests that high quality nurse and physician care is consistently associated with retention, with communication and trust as the factors that most influence patients’ intentions to return [14] and overall ratings of a provider or facility [20–23].
Edgman-Levitan also discussed research that identified several organizational characteristics associated with good patient experiences: governance, leadership, and vision; partnerships with patients and families; human resources and employee engagement; systematic feedback, data management, and goal setting; use of best practices around communication strategies and technology support; and the built environment [24–26].
Governance, leadership, and vision.
System leaders should place patients at the center of their continuous learning and improvement efforts. Healthcare leaders need to engage the entire care team in problem solving and not silence individuals seeking improvement [27]. They should focus on removing barriers to improvement and on achieving excellence by creating a psychologically safe space where individuals feel comfortable identifying solutions that both reduce harm for patients and staff and improve care for patients and families. A key part of this QI work should be the frequent sharing of best practices among staff. This takes leadership at all levels and a consistently communicated vision, and standards for all staff [7].
Partnerships with patients and families.
Health systems should have formal mechanisms to include patients and families in improvement efforts, such as Patient and Family Advisory Councils. Other less formal methods for involving patients and families include participation in task forces, interviews, focus groups, and walkthroughs.
Systematic feedback, data management and goal setting.
Health care leaders should present patient feedback for review in leadership and QI team meetings [28,29]. Patient feedback includes survey data, patient comments, grievance reports, letters, and other qualitative data [8,30–33]. These data should be used to set performance targets and to monitor the effectiveness of QI activities. Health care leaders should conduct regular assessments using CAHPS surveys and Surveys on Patient Safety Culture (SOPS) [34] to identify improvement opportunities and create dashboards to monitor trends and identify high performers for shared learning [35].
Human resources (HR) and employee engagement.
Hiring the right people is key to creating a service-oriented culture. Human resources and organizational training department staff should link hiring, orientation, training, and performance evaluation to quality and, patient experience metrics as well as safety goals.
Communication strategies and technology support.
Health care leaders need to disseminate best practices around communication and technology support [25,36]. Leaders should implement training programs that focus on service excellence, service recovery, and empathy, as well as diversity and equity training for all staff. Organizational resources should include tools for providers to communicate with patients and support QI, including patient-centered portals, decision support tools, educational resources, and websites. For example, to facilitate communication between providers and patients/families, hospitals can implement ‘Quiet at Night’ electronic health record bundles or implement COVID-19 related iPad connections to families.
Build the patient care environment with an eye on the patient. Strategies such as wayfinding (guiding patients through the physical environment where care is delivered) [37–39] and walkthroughs [40,41] can identify areas requiring immediate changes for a range of patient safety and care experience issues, including fall prevention, medication safety, and noise reduction.
3. Panel on effective implementation of patient experience improvement strategies
Four presentations offered examples of organizations implementing patient experience improvement strategies using CAHPS data collected in different health care settings: health plan, medical group, hospital, and dialysis facilities.
Health Plan.
Lisa Franchetti of Neighborhood Health Plan of Rhode Island discussed the results of their CAHPS Health Plan Survey of Medicaid managed care plan enrollees and their use by state agencies and several membership committees. Franchetti discussed the usefulness and common challenges of CAHPS data for identifying opportunities for improving patient experience. Neighborhood Health Plan of Rhode Island staff collect patient experience survey data off-cycle to complement regular CAHPS surveys, analyze incoming member call logs, and monitor call center conversations, all of which have improved processes for conducting first call resolution and led to improvements in CAHPS scores.
Medical Group.
Samuel Skootsky and Deborah Wafer of UCLA Health discussed the multi-pronged Ambulatory Resource Team (ART) patient experience improvement program. The ART program uses the CAHPS Clinician & Group Survey (CG-CAHPS) 3.0 for all ambulatory patients. They discussed using CG-CAHPS data to coach physicians about patient communication and to train clinical directors, office managers, and other staff. They embed the CG-CAHPS metrics into targeted behaviors for training and management performance tracking as well as performance incentives for office managers, clinical directors and staff.
Hospital.
Stephanie Fishkin summarized how the Kaiser Permanente (KP) national health plan and hospital quality department improved medication communication using CAHPS Hospital Survey (HCAHPS) data. KP uses the HCAHPS results to identify key areas for improvement; they examine the specific items within the HCAHPS medication communication composite measure and use techniques to understand their performance, such as gap analysis. Working with interdisciplinary teams in a pilot study in two regions each with a high-performing hospital, KP created a playbook for improving communication around medication. In the pilot and full implementation of the HCAHPS Medication Communication Program, KP learned that stakeholder (local, cross-regional, cross-entity providers and staff, etc.) buy-in at the local, cross-regional, and cross-entity level was key to making needed changes and standardizing the strategies in new sites.
Dialysis Facilities.
Kerri Cavanaugh of Vanderbilt University discussed implementation of patient experience improvement strategies for end-stage renal disease care. Vanderbilt University Medical Center uses scores from the CAHPS In-Center Hemodialysis Survey to create unit-level reports of performance for its dialysis facilities. These reports help identify key domains for improvement, with specific attention to components that are associated with high or low scores. Vanderbilt implemented a variety of patient- and family-level strategies, such as sending follow-up letters to patients requesting feedback about the issues the survey identified to better understand what was affecting scores. Other strategies included identifying champions, establishing annual goals, and using staff meetings to discuss, develop, and refine changes. To guide their efforts and meet annual goals, Vanderbilt also collects complementary data such as patient grievances and customized patient experience surveys.
4. What do we know about methods for evaluating patient experience improvement strategies?
The panel began with an overview by Paul Cleary summarizing the challenges in evaluating patient experience improvement strategies. He noted that many QI efforts are ‘case studies,’ which makes it difficult to rule out the possibility that changes are due to other factors such as changes over time not related to the improvement efforts or underlying characteristics of the types of units participating in such efforts. Randomized or natural experiments, which can control for many confounding factors, are very difficult to conduct alongside the provision of care. Evaluating QI efforts can also be difficult because the number of units is usually small, interventions are not always implemented as planned (i.e. planned vs implemented) and the timing and level of the interventions may be inconsistent. Also, as noted in the current evidence [12] and mentioned in the previous panels, determining the optimal time to assess change and to measure effectiveness is difficult.
As organizations have become more sophisticated about QI, the focus of improvement efforts has expanded to include the use of organizational theories, system approaches, and continuous rapid changes. The models typically used for QI studies are processes such as rapid cycle improvement and plan-do-study-act (PDSA) that include both system and smaller unit perspectives. One novel perspective is implementation science, which can help elucidate QI interventions by examining the external environment, characteristics of the organization and the innovation, the implementation process, and how these influence the adoption, performance, and final outcome of a QI initiative.
A central tension in QI research is deciding whether to start with assessing the overall (or summative) impact of an intervention and then evaluate its component parts, or to evaluate components first and then develop more complex designs for overall impact.
Another challenge is determining whether there is an overarching ‘system’ in which the QI activity operates and then identifying the appropriate unit of analysis. For example, an improvement effort might focus on the use of interdisciplinary teams. It is important, however, to distinguish unit- or department-level work groups that discuss the implementation of strategies (the system) from the frontline interdisciplinary teams that interact with the patients and families (unit of analysis). In addition, it is important to recognize that teams can be complex and have a range of individual, group, and organizational characteristics. Teams have members with different skills, attitudes, and personalities, and also vary in size, structure, cohesiveness, and norms. Teams and their members are embedded in organizations with different reward structures, stresses, and supports. The complex makeup of teams presents difficulties when attempting to disentangle the mediating variables of the group process.
5. Panel on evaluating patient experience improvement strategies using CAHPS surveys
Five presentations provided examples of research or evaluaitons of patient experience improvement strategies using CAHPS data in different health care settings: medical group, health plan, Federally Qualified Health Center (FQHC), hospital and ambulatory care clinic, and hospice.
Medical Group.
Ingrid Nembhard of the University of Pennsylvania described a pilot test of the CG-CAHPS Narrative Item Set in medical groups. The research team’s partner, NewYork-Presbyterian Hospital, uses the CG-CAHPS Narrative Item Set on their ambulatory care survey. The Narrative Item Set consists of five open-ended questions that ask patients to describe in their own words their experiences with their provider and the office staff. The pilot study was designed to assess the feasibility, value, and use of the narrative items in ambulatory practices through a two-phase quasi-experimental study with nine practice sites in the NewYork-Presbyterian system. Nembhard highlighted the need for experimental study designs and developing interventions collaboratively with organizational members. The research team developed an online feedback tool to report the comments back to practices. Using a mixed-methods study approach, they found that the narrative data provides information that supplemented the closed-ended survey data. Practice site leaders and physicians reported more actionable feedback when using the narrative items in conjunction with the closed-ended survey items.
Health Plan.
Mark Friedberg of Blue Cross Blue Shield of Massachusetts discussed what it takes to improve CAHPS Health Plan Survey and CG-CAHPS Survey scores. He reviewed methods for improving CAHPS scores, including incentives for improvement, changing the provider network and direct-to-member approaches. He stressed that health plans need to distinguish significant changes from random changes in CAHPS scores when interpreting performance over time.
Federally Qualified Health Center (FQHC).
Denise Quigley of RAND and Efrain Talamantes of AltaMed Health discussed their evaluation of AltaMed’s shadow coaching and pay-for-performance efforts to improve patient-provider interactions using the CG-CAHPS Survey. AltaMed administers the CG-CAHPS 2.0 Visit Survey for adult and child patients. The research team evaluated the influence of the shadow coaching program on patient experience using spline models that indicate if an intervention is associated with a change in the trajectory of scores. The study also included an evaluation of the influence of re-coaching providers that were coached previously on patient experience using a wait-list control group design. This control group served as an untreated comparison group during the study but received the treatment later. The shadow coaching program used the CAHPS overall provider rating to identify medium-performing providers for peer coaching with a high-performing coach. In addition, every six months providers received a pay-for-performance (P4P) incentive payment based on their CAHPS overall provider rating. The evaluation team assessed P4P incentive payments before and after the peer coaching, provider perceptions assessed by survey and in interviews, and the content of the coaching reports given to the coached providers after the coaching sessions. The results showed that coaching improved providers’ overall rating and CAHPS communication scores, but these improvements faded over time [25]. The content analysis of the recommendations indicated that the coaching recommendations were split between encouragement of existing behavior and identifying new behaviors. Recommendations from the coaches focused on provider-patient communication and were considered tangible and actionable by the coached providers [42].
Hospital and Ambulatory Care Clinic.
Jim Schaefer and Jennifer Purdy of the Veterans Administration Health System discussed several examples of patient experience improvement implemented at Veterans Health Administration facilities. Since 2009, the Department of Veterans Affairs (VA) has administered the Survey of Healthcare Experience of Patients (SHEP), an ambulatory care survey based on CG-CAHPS, in VA clinics for outpatient care and HCAHPS for their inpatient care. In 2016, they added the CAHPS supplemental items for assessing Patient-Centered Medical Home (PCMH) experiences to their ambulatory care survey and expanded their survey to include specialty care. A patient experience toolkit for use across inpatient, outpatient, medical home and specialty care was developed based on associations with with overall ratings in a driver analysis and a journey mapping exercise plotting the flow of patients’ care experiences before, during, and after receiving health care or interacting at the VA. The four-year effort included: 1) building the pilot toolkit, 2) using it at sites to develop leadership and employee engagement, 3) assessing site needs and identifying key outcomes to target and improve, and 4) standardizing the program across sites and developing accountability for outcomes. This process improved the VA’s HCAHPS Star Rating scores for communication with nurses, care transitions, overall rating of the hospital, and the overall summary star rating.
Hospice.
Natalie McNeal of WellStar Community Hospice discussed the methods they use to implement and evaluate patient experience improvement using CAHPS surveys. WellStar administered the CAHPS Hospice Survey and used an A-3 Lean methodology with PDSA cycles, root cause analysis, and structured performance reports to make and track improvements in CAHPS scores. These methods are all problem-solving techniques originally developed in manufacturing industries. Wellstar’s CAHPS Hospice Survey scores were well below the benchmarks and system goals. While the work is still in progress, WellStar identified key barriers to making improvements, including the lag time between the last day of care and the mailing of the survey and the effect of the number of hospice days on scores. McNeal highlighted the need for experts to guide a QI team through the Lean process, assist in setting attainable goals, and support the practical steps of the QI process. She also recommended focusing on improvements in the implementation process rather than failures and examining possible causal factors in addition to root causes of inefficiencies in care processes.
6. Emergent themes from panel presentations
Five main themes emerged from the panel presentations:
Theme 1: CAHPS Data are Fundamental to Multiple Steps in Making Improvements; Other Data Typically Supplement CAHPS Data.
Presenters emphasized the use of traditional closed-ended questions for benchmarking and trend analysis to monitor QI efforts over time. Speakers noted organizational uses of CAHPS scores to identify areas that need improvement, identify high or low performers, key drivers of survey scores, and deficits in practice or care that patients experienced. Presenters discussed the creation of dashboards that track key performance metrics over time at the individual-, site-, or organizational-level; these are used to facilitate continuous QI initiatives. Dashboard elements typically included national benchmarks, overall performance and unit-level and provider-level performance. Such tools can also identify areas that need improvement or identify high or low performers by comparing individuals or sites. Nearly all participants reported supplementing CAHPS scores (i.e. the main outcome of interest) with other data that can inform QI activities and monitor changes in real time. The additional data, collected either as administrative or program-specific data, were used to highlight key trends or indicate whether changes are occurring, allowing for adjustments to improve patient care before receiving CAHPS results. Other forms of data, including call logs, administrative data, and open-ended comments, can be collected and reviewed much more quickly and inform QI activities rapidly while CAHPS survey data are collected and processed by vendors. Several presenters pointed out that the use of CAHPS scores in conjunction with other data increased trust in CAHPS survey results, especially if the complementary data were consistent with the final CAHPS results. It was mentioned that verbatim comments add specific information that may not be captured in CAHPS survey scores and can assist in identifying ways to improve patient experience. For example, at UCLA Health, the CAHPS survey scores and verbatim comments are shared in a monthly report, which has strengthened physician buy-in for the use of CAHPS measures and provided input on QI initiatives.
Theme 2: Provider Engagement is Important When Improving CAHPS Scores.
Most projects highlighted the need for provider feedback, buy-in and wellbeing. While providers want to improve patient care, they also want to have a role in patient experience improvement activities and to have a say in P4P incentives for improving quality. Engaging staff in the design and implementation of the QI effort increases their buy-in related to how changes are evaluated. Multiple participants discussed various means of engaging clinicians and staff to get buy-in and support for QI initiatives supported by CAHPS data. For example, integrating CAHPS results into management reviews was cited as a way to increase the use of patient experience data in clinical improvement activities. Others focused on involving clinicians and staff in the research design process and sought their input into what data should be used to measure success.
Theme 3: Quality Improvement Efforts Follow a Similar Process Across Health Care Settings; Standardizing QI Processes Adds Value.
The process of patient experience improvement is consistent across settings. The process generally consists of measuring patient experience with a CAHPS survey, identifying an area of interest or need for improvement, selecting the relevant measures or metrics to track from a CAHPS survey, incorporating other measures that complement the CAHPS measures, developing a QI initiative with the buy-in and engagement from clinicians and staff, implementing the various phases of QI (such as the steps in the PDSA cycle), tracking changes over time in the key metrics, adjusting the improvement plan as results emerge, and, in some cases, monitoring whether improvements are sustained over time. This process aligns with the evidence [43–45]. Multiple participants also discussed the value of standardizing QI processes, especially for multisite implementation of QI initiatives. These processes include creating a playbook or guide for sites to create a shared workflow in identifying problem areas, collecting and identifying data for evaluation, and implementing changes to practices and procedures.
Theme 4: Response Rates Can Undermine the Value of CAHPS Data in Improvement Efforts.
Low response rates are a potential barrier to using CAHPS data for QI initiatives [46,47], making it difficult to collect sufficient data to guide QI changes and to track improvements. Speakers noted problems associated with low response rate in CAHPS surveys, including the possibility that the data does not adequately represent their patient population. Evidence does support that over the years, the response rates for CAHPS surveys [48] and surveys in general [47] have declined; this results in a belief among some providers and staff that the results from CAHPS surveys do not represent their entire patient population. Low response rates also increase the time needed to get a sufficient number of completed surveys for monitoring and tracking changes in patient experience. This in turn results in aggregated data being less actionable for QI initiatives.
Another limitation often mentioned is the time needed to get the information collected from CAHPS surveys into the hands of those monitoring and implementing changes. This time lag is not specific to CAHPS surveys, but is connected to the data collection mode (paper, phone, e-mail, IVR) chosen by the organization and its leaders. Often supplemental surveys or other forms of data collected on a timelier basis are used in addition to CAHPS surveys for QI activities and help address specific aspects of change that are of interest for QI.
Theme 5: Organization-wide Assessments of CAHPS are Useful for Performance Improvement and Identifying Practical Strategies; Rigorous Evaluations and Study Designs are Less Common, But Valuable.
Panelists acknowledged that a well-designed and rigorous study design is valuable, but they can still learn and accomplish much with practical organization-level studies – both those studies using CAHPS measures or those on effective strategies for improvement. Quality improvement efforts are often geared toward understanding the local organization’s processes and mechanisms for providing quality patient experience. Evaluating and implementing strategies to understand generalizability is important but not necessary for using CAHPS data to improve patient experience.
Overall, we found that health care organizations across various health care settings take similar analysis and QI process steps when conducting QI using CAHPS survey data. There tends to be an emphasis on including provider feedback and buy-in from the initial conceptualization to the later phases of data review and tracking. Importantly, data from close-ended CAHPS survey questions are consistently used to guide QI activities, with additional data drawn from program and administrative data sources used to add context and timely insights. Because of the role that CAHPS data play in the monitoring and implementation of QI efforts, timely and frequent reporting of CAHPS data is crucial to their success. Using standardized CAHPS survey data collection and maximizing response rates are critical to representation of the underlying patient population and using CAHPS data for improvements.
7. Conclusion
Evidence from multiple settings indicates several foundational elements for health care organizations to be patient- and family-centered. Specifically, organizations should:
Encourage leaders and staff to place patients at the center of their work and develop a system of continuous learning and improvement.
Support and maintain provider engagement in QI activities and educate them about the science behind patient experience measurement.
Involve care teams in problem-solving partnerships with patients and families, sharing best practices and data often and consistently among staff.
Prioritize, synthesize, and review patient feedback in leadership and QI team meetings.
Create formal mechanisms to include patients and families in care improvement, including patient and family advisory councils for co-design and co-production of QI activities.
Regularly assess CAHPS data for areas to improve and create dashboards to identify trends and high performers for shared learning and systematic data-driven feedback.
Focus on standardizing processes for improvement across the organization to reduce the inherent variation in health care delivery processes across units of care (provider, unit/floor/team, site-level, group-level)
CAHPS surveys collect important information about patient and family experiences of care that can be compared to national data and compared internally across units, but it is important to focus on problem areas needing improvement. Numerous other sources of data, including patient complaints, service recovery, and patients/families’ comments, can also help uncover the nuances and processes of care. It is important to recognize the connections between patient experience and worker safety, patient safety, and provider well-being.
It is also necessary to address concerns about response rates and representativeness of CAHPS survey data. In addition, to working closely with their vendors to maximize response rates through innovative data collection strategies, health care organizations should collect enough survey data to obtain adequate reliability, conduct subgroup analyses to understand the patterns within the organization, and use performance dashboards to make scores more evident and transparent. Finally, a well-designed and rigorous study design is valuable, but health care leaders and providers may still learn and accomplish much with a less rigorous but very practical organization-level studies using CAHPS measures regarding effective strategies for improvement.
Lastly, it is important to distinguish the requirements related to surveys for accountability and public reporting efforts from the foundational need within health care organizations to collect data and administer surveys that support QI efforts. Policies that facilitate the sharing of QI strategies, challenges, and outcomes among and across health care systems are needed to help foster and accelerate improvements in health care and care experiences. The value of sharing and disseminating organization-wide assessments of patient experience, such as those measured by CAHPS surveys, for performance improvement and identifying practical strategies is crucial, but policies and research efforts that encourage rigorous designs to assess and evaluate practical strategies and QI efforts are also important.
Supplementary Material
Acknowledgments
The authors want to thank and acknowledge all the panel organizers, presenters and participants that attended the meeting.
Funding
This work was funded under cooperative agreements [U18 HS025920 and U18 HS016978] from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.
Footnotes
Declaration of interest
The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Reviewer disclosures
Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.
References
Papers of special note have been highlighted as either of interest (•) or of considerable interest (••) to readers.
- 1.National Academy of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington (D.C): National Academy Press; 2001. Papers of special note have been highlighted as either of interest (*) or of considerable interest (**) to readers [PubMed] [Google Scholar]
- 2.Barry MJ, Edgman-Levitan S. Shared decision making–pinnacle of patient-centered care. N Engl J Med. 2012;366:780–781. [DOI] [PubMed] [Google Scholar]
- 3.Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Med Care Res Rev. 2013;70:351–379. [DOI] [PubMed] [Google Scholar]
- 4. Khanna G, Ginsberg C. CAHPS®: 25 years of putting the patient first Rockville (MD): Agency for Healthcare Research and Quality; 2020. [Available from: https://www.ahrq.gov/news/blog/ahrqviews/cahps-25yrs.html] • This is an important reference as it provides the background and context for the rationale and purpose of CAHPS surveys.
- 5.Friedberg MW, SteelFisher GK, Karp M, et al. Physician groups’ use of data from patient experience surveys. J Gen Intern Med. 2011;26:498–504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Geissler KH, Friedberg MW, SteelFisher GK, et al. Motivators and barriers to using patient experience reports for performance improvement. Med Care Res Rev. 2013;70:621–635. [DOI] [PubMed] [Google Scholar]
- 7.Quigley DD, Mendel PJ, Predmore ZS, et al. Use of CAHPS® patient experience survey data as part of a patient-centered medical home quality improvement initiative. J Healthc Leadersh. 2015;7:41–54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Quigley DD, Qureshi N, AlMasarweh L, et al. Using CAHPS® patient experience data for patient-centered medical home transformation. Am J Manag Care. 2021;27:294–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Centers for Medicare & Medicaid Services. Hospital value-based purchasing Baltimore (MD): Centers for Medicare & Medicaid Services; 2020. Available from: https://www.cms.gov/outreach-and-education/medicare-learning-network-mln/mlnproducts/downloads/hospital_vbpurchasing_fact_sheet_icn907664.pdf
- 10.Centers for Medicare & Medicaid Services. CAHPS for MIPS survey Baltimore (MD): Centers for Medicare & Medicaid Services; 2021. Available from: https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/MIPS
- 11.Centers for Medicare & Medicaid Services. Medicare Advantage and Prescription Drug Plan CAHPS (MA and PDP CAHPS) Baltimore, MD: Centers for Medicare & Medicaid Services; 2020. [Available from: https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/MCAHPS [Google Scholar]
- 12.LaVela SL, Gallan AS. Evaluation and measurement of patient experience. Patient Exp J. 2014;1:28–36. [Google Scholar]
- 13.Davies E, Shaller D, Edgman-Levitan S, et al. Evaluating the use of a modified CAHPS survey to support improvements in patient-centred care: lessons from a quality improvement collaborative. Health Expect. 2008;11:160–176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Quigley DD, Reynolds K, Dellva S, et al. Examining the business case for patient experience: a systematic review. J Healthc Manag. 2021;66:200–224. • This review provides the evidence on what we know about how patient experience affects the bottom line and impact of patient experiences
- 15. Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev. 2014;71:522–554. • This review is important because it provides evidence for the value of patient experience survey data in measuring health care quality.
- 16.Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. British Medical Journal Open. 2013;3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lee TH, How US. Health care got safer by focusing on the patient experience. Brighton (MA): Harvard Business Review; 2017. [Google Scholar]
- 18.Rodriguez HP, Rodday AM, Marshall RE, et al. Relation of patients’ experiences with individual physicians to malpractice risk. Int J Qual Health Care. 2008;20:5–12. [DOI] [PubMed] [Google Scholar]
- 19.Stelfox HT, Gandhi TK, Orav EJ, et al. The relation of patient satisfaction with complaints against physicians and malpractice lawsuits. Am J Med. 2005;118:1126–1133. [DOI] [PubMed] [Google Scholar]
- 20.Hays RD, Larson C, Nelson EC, et al. Hospital quality trends. A short-form patient-based measure. Med Care. 1991;29:661–668. [PubMed] [Google Scholar]
- 21.Hays RD, Nelson EC, Rubin HR, et al. Further evaluations of the PJHQ scales. Med Care. 1990;28:S29–39. [DOI] [PubMed] [Google Scholar]
- 22.Keller S, O’Malley AJ, Hays RD, et al. Methods used to streamline the CAHPS hospital survey. Health Serv Res. 2005;40:2057–2077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Rothman AA, Park H, Hays RD, et al. Can additional patient experience items improve the reliability of and add new domains to the CAHPS hospital survey? Health Serv Res. 2008;43:2201–2222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Qureshi N, Quigley DD, Hays RD. Nationwide qualitative study of practice leader perspectives on what it takes to transform into a patient-centered medical home. J Gen Intern Med. 2020;35:3501–3509. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Quigley DD, Elliott MN, Slaughter ME, et al. Shadow coaching improves patient experience with care, but gains erode later. Med Care. 2021. Epub ahead of print;59:950–960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Shaller D Patient-centered care: what does it take? New York, NY: The Commonwealth Fund; 2007. Available from: https://www.commonwealthfund.org/publications/fund-reports/2007/oct/patient-centered-care-what-does-it-take [Google Scholar]
- 27.Davies EA, Meterko MM, Charns MP, et al. Factors affecting the use of patient survey data for quality improvement in the veterans health administration. BMC Health Serv Res. 2011;11:334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Quigley DD, Palimaru A, Lerner C, et al. A review of best practices for monitoring and improving inpatient pediatric patient experiences. Hosp Pediatr. 2020;10:277–285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Quigley DD, Predmore ZS, Chen A, et al. Implementation and sequencing of practice transformation in urban practices with underserved patients. Qual Manag Health Care. 2017;26:7–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Donetto S, Pierri P, Tsianakas V, et al. Experience based co-design and healthcare improvement: realizing participatory design in the public sector. Des J. 2015;18:227–248. [Google Scholar]
- 31.Grob R, Schlesinger M, Barre LR, et al. What words convey: the potential for patient narratives to inform quality improvement. Milbank Q. 2019;97:176–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Quigley DD, Slaughter ME, Gidengil C, et al. Usefulness of child HCAHPS survey data for improving inpatient pediatric care experiences. Hosp Pediatr. 2021;11:e199–e209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Stevenson DG. Nursing home consumer complaints and quality of care: a national view. Med Care Res Rev. 2006;63:347–368. [DOI] [PubMed] [Google Scholar]
- 34.Agency for Healthcare Research and Quality. Surveys on patient safety culture™ Rockville (MD): Agency for Healthcare Research and Quality; 2021. Available from: https://www.ahrq.gov/sops/index.html [Google Scholar]
- 35. Edgman-Levitan S, Shaller D, McInnes K, et al. The CAHPS improvement guide: practical strategies for improving the patient care experience. Baltimore (MD): Centers for Medicare & Medicaid Services; 2003. •• This reference is of considerable importance as it provides practical information and details on each of the steps in the QI process with how to use CAHPS data and provides strategies and examples
- 36.Leebov W, Afriat S, Presha J. Service savvy health care: one goal at a time. Lincoln (NE): Authors Choice Press; 2007. [Google Scholar]
- 37.Arthur P, Passini R. Wayfinding: people, signs, and architecture. New York (NY): McGraw-Hill, Inc.; 1992. p. 252. [Google Scholar]
- 38.Foltz M Designing navigable information spaces. Chapter 5: design principles for wayfinding. Cambridge, MA: Massachusetts Institute of Technology; 1998. Available from: http://www.ai.mit.edu/projects/infoarch/publications/mfoltz-thesis/node8.html [Google Scholar]
- 39.Lynch K The image of the city. Cambridge (MA): The Technology Press and the Harvard University Press; 1960. [Google Scholar]
- 40. Agency for Healthcare Research and Quality. Section 5: determining where to focus efforts to improve patient experience (page 2 of 2) Rockville (MD): Agency for Healthcare Research and Quality; 2020. [Available from: https://www.ahrq.gov/cahps/quality-improvement/improvement-guide/5-determining-focus/section5part2.html] ** This reference provides practical information on how to identify areas to improve in the CAHPS data, strategies and examples.
- 41.Gustafson D Walk-through tool boston, MA: Institute for Healthcare Improvement; 2021. Available from: http://www.ihi.org/resources/Pages/Tools/Walkthrough.aspx [Google Scholar]
- 42.Quigley DD, Qureshi N, Palimaru A, et al. Content and actionability of recommendations to providers after shadow coaching. Qual Manag Health Care. 2022;doi: 10.1097/QMH.0000000000000354. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.McDonald KM, Schultz EM, Chang C. Evaluating the state of quality-improvement science through evidence synthesis: insights from the closing the quality gap series. Perm J. 2013;17:52–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Portela MC, Pronovost PJ, Woodcock T, et al. How to study improvement interventions: a brief overview of possible study types. Postgrad Med J. 2015;91:343–354. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Weiss CH. Methods for studying programs and policies. Upper Saddle River (NJ): Prentice Hall; 1998. (Accessed: Mar 15, 2022). [Google Scholar]
- 46.American Hospital Association. Modernizing the HCAHPS survey. Chicago (IL): American Hospital Association; 2019. [Google Scholar]
- 47.Tesler R, Sorra J. CAHPS survey administration: what we know and potential research questions Rockville (MD): Agency for Healthcare Research and Quality; 2017. Available from: https://www.ahrq.gov/sites/default/files/wysiwyg/cahps/about-cahps/research/survey-administration-literature-review.pdf [Google Scholar]
- 48.Fowler FJ Jr., Cosenza C, Cripps LA, et al. The effect of administration mode on CAHPS survey response rates and results: a comparison of mail and web-based approaches. Health Serv Res. 2019;54:714–721. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
