Associations of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Survey Scores with Interventions and Site, Provider, and Patient Factors: A Systematic Review of the Evidence

Abstract

Patient experience is a key aspect of care quality. Since the 2007 release of the Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS) survey, no systematic review of factors associated with CG-CAHPS scores has been reported. We reviewed 52 peer-reviewed English language articles published in the United States using CG-CAHPS data. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines and used the Checklist for Analytical Cross-Sectional Studies. We identified several interventions (eg, adding a care coordinator focused on chronic care management) associated with improved overall provider rating and 2 interventions (eg, peer shadow coaching) that improved provider communication scores. Studies evaluating the implementation of patient-centered medical homes or patient-reported outcomes found mixed results. We identified site-level factors (eg, better team communication) and provider-level factors (eg, physician empathy) associated with better patient experience. In contrast, patient-level factors (eg, medication adherence) found mixed associations with patient experience. Policymakers, clinicians, and healthcare leaders can leverage this evidence for quality improvement efforts and interventions supporting patient-centered care.

Key Points

1. Our review identified several interventions (eg, adding a care coordinator focused on chronic care management) associated with improvement in overall ratings of the provider and 2 interventions (eg, peer shadow coaching) that improved provider communication.

2. Patient experience was inconsistently associated with patient-centered medical homes and the use of patient-reported outcomes in care.

3. Several site-level factors (eg, better team communication) and provider-level factors (eg, physician empathy) were associated with better patient experience. In contrast, patient-level factors (eg, medication adherence) had mixed associations with patient experience.

Introduction

Patient experience is an integral aspect of care quality and is positively associated with the performance of recommended clinical processes. ¹ The Consumer Assessment of Healthcare Providers and Systems Clinician and Group (CG-CAHPS^®) survey, which asks patients to report on their experiences with providers and staff in primary care and specialty care settings, was developed for public reporting, assessing the impact of care delivery interventions, facilitating patient selection of care provider, and providing actionable information to ambulatory care providers and stakeholders.^2,3 Multiple versions of the CG-CAHPS survey exist.^2,4–7 And they can be administered in several survey modes and languages.^8–10 Survey domains include provider communication, access to care, and care coordination. Optional, supplemental CG-CAHPS survey items are available for specific topics of interest: the patient-centered medical home (PCMH), ¹¹ health literacy, ¹² health information technology, ¹³ cultural competence, ¹⁴ and patient narratives. ¹⁵

A recent review documented the types of studies using CG-CAHPS survey data since its 2007 release, of which 37 were cross-sectional, 9 case-control comparisons, 3 cohort studies, and 3 randomized control trials. ¹⁶ The review also found that CG-CAHPS data was used primarily to evaluate interventions (24 studies) and cross-sectional associations (21 studies), with 4 studies examining disparities in patient experience. The authors concluded that the widespread use of CG-CAHPS data indicates its value in measuring and improving care quality. They suggested that policymakers, researchers, clinicians, and healthcare leaders leverage CG-CAHPS data in quality improvement (QI) efforts and interventions that support patient-centered care. However, no systematic review has examined the evidence on associations of CG-CAHPS patient experience measures with interventions and site-level, provider-level, and patient-level factors.

We examine studies using CG-CAHPS data in the United States since 2007 to identify interventions and associations with CG-CAHPS scores. This information can help policymakers, researchers, clinicians, and healthcare leaders use CG-CAHPS survey data for QI, and interventions to improve patient-centered care.

Methods

This paper extends the systematic review of how US research has used CG-CAHPS data. ¹⁶ It identifies interventions and factors associated with patient experiences measured by CG-CAHPS data. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines^17,18 and we have updated the reporting of our review based on PRISMA 2020^19,20 (see Supplemental Digital Content 1 for the PRISMA Checklist).

Search Strategy and Selection Criteria

We searched 6 databases (PubMed (using Ovid Medline^®), Web of Science, Scopus, Cochrane Database of Systematic Reviews (CDSR) (via Wiley), American Psychological Association PsycInfo, and Cumulative Index of Nursing and Allied Health Literature (CINAHL)) using a structured search strategy involving terms including patient experience, outcomes, surveys, and CG-CAHPS to identify peer-reviewed US English-language articles published from January 1, 2007, to July 31, 2023. Two reviewers independently screened each abstract using standardized eligibility criteria to identify articles for full-text review.

We excluded articles that did not use CG-CAHPS (n = 66), only concerned the measurement properties of CAHPS surveys (n = 24), used CG-CAHPS but not as an outcome (n = 21), were not conducted in the United States (n = 10), involved only pediatric care (n = 4), were a literature review (n = 5), were before 2007 (n = 2), or were not empirical studies (ie, commentaries) (n = 14).

Data Abstraction and Quality Assessment

One reviewer reviewed and abstracted each article; another reviewed the abstracted information to ensure accuracy. Abstracted information included the study's objective, use of CG-CAHPS, study design and type, statistical methods, and main findings. ¹⁶

Most of the articles were cross-sectional, so we used the Joanna Briggs Institute (JBI) Checklist for Analytical Cross-Sectional Studies to assess study quality and risk of bias. We also excluded 12 studies because they did not possess at least 6 of the 8 JBI-Critical Appraisal Tool Checklist elements. These 12 studies reported information about exposure and outcomes but did not describe information on sample selection or did not control for confounding factors.

We grouped articles by study type (eg, intervention, association), by CG-CAHPS measure(s), and by statistical significance found (P-values < .05). We also reviewed them by year published.

Results

This search identified 210 unique articles. Following screening and full-text review, we included 52 in our synthesis. The methods, population, measures, and main topic for each included study are described elsewhere. ¹⁶ Most studies (27 of 52) involved primary care settings (either general populations, people with diabetes, or Medicare beneficiaries). Since 2012, an average of 5 studies have been published annually, 2 to 9 each year.

Overall provider rating (n = 42 studies) and the provider communication composite (n = 41) were the most commonly used CG-CAHPS measures. Access measures were also used often (n = 22), as were the global recommendation (n = 17), office staff courteousness (n = 12), and PCMH items (n = 9).

Many studies used CG-CAHPS data to assess the impact of interventions (n = 24 studies) or identify associations between patient experience and factors of interest (n = 21 studies). Six studies compared subgroups, and 1 conducted external benchmarking. Table 1 presents studies using CG-CAHPS data grouped by setting.

Table 1.

Included Studies’ Main Use of CG-CAHPS Survey by CG-CAHPS Measure and Grouped by Setting.

Use of CG- CAHPS	Overall provider rating (N = 42)	Would recommend (N = 17)	Provider communication (N = 41)	Access (N = 22)	Care coordination (N = 7)	Follow-up on test results (N = 6)	Courteous, office staff (N = 12)	PCMH item set (N = 9)	Other CAHPS items (N = 3)
Impact of Intervention (N = 24)	Primary & Specialty Care: Fareed 2022***V Hays 2022***6 Primary Care: Behl-Chadha 2017**12 Dale 2016**12 De Leon 2012**V DiCapua 2017**12 Dorr 2016*12 Kern 2013**12 Nembhard 2012**V Quigley 2021**V Quigley 2023a**V Sarinopoulos 2017***6 Setodji 2017**V Swankoski 2017**12 Swankoski 2018**12 Ortho/Neuro: Bernstein 2019**V Bernstein 2023***V Oladeru 2017**V Zakare-Fagbamila 2019***6 Rheumatology: Honomichl 2020**12	Primary & Specialty Care: Fareed 2022***V Hays 2022***6 Primary Care: De Leon 2012**V DiCapua 2017**12 Nembhard 2012**V Setodji 2017**V Ortho/Neuro: Bernstein 2019**V Oladeru 2017**V Rheumatology: Honomichl 2020**12	Primary & Specialty Care: Boissy 2016**V Fareed 2022***V Hays 2022***6 Primary Care: Behl-Chadha 2017**12 Dale 2016**12 De Leon 2012**V Di Capua 2017**12 Dorr 2016*12 Kern 2013**12 Nembhard 2012**V Quigley 2021**V Quigley 2023a**v Sarinopoulos 2017***6 Setodji 2017**V Spiegel 2023**V Swankoski 2017**12 Swankoski 2018**12 Ortho/Neuro: Bernstein 2019**V Bernstein 2023***V Oladeru 2017**V Zakare-Fagbamila 2019***6	Primary & Specialty Care: Fareed 2022***V Hays 2022***6 Primary Care: Behl-Chadha 2017**12 Dale 2016**12 De Leon 2012**V Di Capua 2017**12 Dorr 2016*12 Kern 2013**12 Lin 2020**12 Nembhard 2012**V Nembhard 2020***6 Sarinopoulos 2017***6 Setodji 2017**V Swankoski 2017**12 Swankoski 2018**12	Primary & Specialty Care: Fareed 2022***V Hays 2022***6 Primary Care: Di Capua 2017**12 Dorr 2016*12 Nembhard 2020***6 Sarinopoulos 2017***6	Primary Care: Behl-Chadha 2017**12 De Leon 2012**V Dorr 2016*12 Kern 2013**12 Setodji 2017**V	Primary & Specialty Care: Fareed 2022***V Hays 2022***6 Primary Care: Behl-Chadha 2017**12 Di Capua 2017**12 Dorr 2016*12 Kern 2013**12 Nembhard 2012**V Setodji 2017**V	Primary Care: Behl-Chadha 2017**12 Dale 2016**12 Dorr 2016*12 Kern 2013**12 Nembhard 2020***6 Sarinopoulos 2017***6 Swankoski 2017**12 Swankoski 2018**12	Ortho/Neuro: Bernstein 2019**V : Talked with provider about problems or concerns
Associations to Patient Experience (N = 21)	Primary & Specialty Care : Chaitoff 2017**12V Bosko 2016**1V Quigley 2014**12 Primary Care : Chung 2020***6 Hasnain 2013**12 Willard-Grace 2021***6 Ophthalmology: Han 2021**V Ortho/Neuro: Hageman 2015**12 Johnson 2019**V Khan 2021**12 Lanz 2018**V Lapin 2019**V Matar 2021***6 Rabah 2021a**12 Rabah 2021b**12 Zakare-Fagbamila 2020***6	Primary Care : Hasnain 2013**12 Willard-Grace 2021***6 Ortho/Neuro: Hageman 2015**12 Johnson 2019**V Khan 2021**12 Lanz 2018**V Lapin 2019**V	Primary & Specialty Care : Chaitoff 2017**12V Quigley 2014**12 Primary Care : Bauer 2014***12 Chung 2020***6 Hasnain 2013**12 Lee 2020**12 Lin 2017**12 Ratanawongsa 2013***12 Willard-Grace 2021***6 Ortho/Neuro: Hageman 2015**12 Khan 2021**12 Lanz 2018**V Lapin 2019**V Zakare-Fagbamila 2020***6	Primary Care : Chung 2020***6 Hasnain 2013**12 Nembhard 2015**V Ortho/Neuro: Khan 2021**12 Zakare-Fagbamila 2020***6		Primary Care : Nembhard 2015**V	Primary Care : Hasnain 2013**12		Primary Care : Bauer 2014***12 : Shared decision making (SDM) score Hasnain 2013**12: SDM and provider interactions
Comparison of Subgroups (N = 6)	Primary Care : Kippenbrock 2019**12 Prasad 2021***6 Quigley 2023b**v Ortho/Neuro: Agarwal 2019**v Dermatology: Cheng 2022***12	Ortho/Neuro: Agarwal 2019**v	Primary Care : Carvajal 2014**12 Kippenbrock 2019**12 Prasad 2021***6 Quigley 2023b**v Ortho/Neuro: Agarwal 2019**v Dermatology: Cheng 2022***12	Primary Care : Carvajal 2014**12 Prasad 2021***6	Prasad 2021***6		Primary Care : Carvajal 2014**12 Prasad 2021***6	Primary Care : Prasad 2021***
External Benchmarking (N = 1)	Oncology : Kim 2021***6

Note: *CG-CAHPS Version 1.0, **CG-CAHPS Version 2.0, ***CG-CAHPS Version 3.0 and ^V indicates visit survey,⁶ indicates 6-month reference period and¹² indicates 12-month reference period.

The composite measures differ between CG-CAHPS version 2.0 and 3.0 in the following ways:

• Provider Communication—Version 3.0 contains a 4-item subset of the 6 items on Version 2.0.

• Getting Timely Appointments, Care, and Information—Version 3.0 contains a subset of the items on Version 2.0.

• Providers’ Use of Information to Coordinate Patient Care—new composite for Version 3.0 that includes an item in the Version 2.0 provider communication composite and includes the previous single item on Follow-up on Test Results.

• Helpful, Courteous, and Respectful Office Staff—identical in Versions 2.0 and 3.0.

• Follow-up on Test Results—identical in Versions 2.0 (as a single item measure) and 3.0 (included in care coordination composite).

• Overall Rating of the Provider—identical in Versions 2.0 and 3.0.

Associations of Interventions With Changes in CG-CAHPS Measures

Twenty-four studies used CG-CAHPS data to assess improvement in patient experience following an intervention. Table 2 summarizes these intervention studies and the extent to which the interventions were associated with improvements. We highlight several patterns, using bold text to highlight the evaluated interventions.

Table 2.

Impact of Interventions on CG-CAHPS Measures, by Type of Ambulatory Care and Intervention (n = 24 Studies).

Author, Year	Intervention	Overall provider rating	Would recommend	Provider communication	Access	Care coordination	Follow-up on test results	Courteous office staff	PCMH item set	Other CAHPS items
Primary and Specialty Care
Boissy 2016	Communication training			↑
Hays 2022	Telehealth during COVID-19	↑	↑	↑	↑	↑		↑
Fareed 2022	Patient portal activation	↔	↔	↑	↔	↑		↔
Primary Care (PC)
Di Capua 2017	Care coordination	↔	↔	↔	↔	↓		↑
De Leon 2012	EHR implementation	↔	↓	M (see NOTE)	M		↔			↔K
Behl-Chadha 2017	PCMH for homeless patients	↔	↓	↓	↓		↔	↓	M:↑(+,%), ↔&
Dorr 2016	Received PCMH coaching and used PCMH-QI strategies	↑		↔	↓	↔	↑	↔	↔ (+,%)
Kern 2013	PCMH implementation	↔		M : ↔ with ↑S	↑		↔	M :↔ with ↑H	↔ (+,%)
Sarinopoulos 2017	PCMH type (multipayer/PCMH/not PCMH)	↔		M:↑multipayer, ↔ PCMH	↑	↑			↑(+,%)
Spiegel 2023	CDS versus patient education (PE)			↑ PE
Nembhard 2012	Quality collaborative focused on quality-oriented climate	↑	↔	↔	↓			↑
PC in Federally Qualified Health Centers
Nembhard 2020	Care coordination				↔	↔			↔%
Quigley 2021	Shadow coaching	↑		↑
Quigley 2023a	Follow-up shadow coaching	↑		↑
Setodji 2017	PCMH with high use of chronic care management	↑	↔	↑	↓		↔	↓
PC for Medicare Beneficiaries
Dale 2016	Comprehensive Primary Care	↔		↔	↔				M:↑ (+,%),↔$
Lin 2020	Access interventions∼ in CPC				↔
Swankoski 2017	Comprehensive Primary Care	↔		↔	↑				M:↑(+,%),↔$
Swankoski 2018	Comprehensive Primary Care	↔		↔	↔				↔ (+,%)
Ortho/Neuro
Bernstein 2019	Use of PROM assessment	↑	↑	M :↔ with ↑S						↔*
Bernstein 2023	Use and discuss PROM	↔		M :↔ with ↑E
Oladeru 2017	Communication training	↔	↔	↔
Zakare-Fagbamila 2019	Use of real-time feedback	↔		↔
Rheumatology
Honomichl 2020	PROM implementation	↔	↔

Note: We annotate significance on CAHPS measures using ↑ to indicate statistically significant improvements (P < .05); ↓ to indicate statistically significant decline/worsening (P < .05); ↔ to indicate no statistical significance found; and M for results that were both positive and negative within the composite measure (ie, mixed). CDS indicates physician-directed clinical decision support. Some of the research reported out by item. For provider communication items, E indicates Explained things, S indicates Spent enough time. H indicates aspect of office staff composite about Office staff is helpful.  + indicates PCMH item about Self-management support. % indicates PCMH item about Provider paid attention to mental or emotional health. & indicates PCMH item about Provider discussed medication decisions. $ indicates PCMH item about Informed about care from other providers. * study used CAHPS item about Talked with provider about problems or concerns. For De Leon 2012, Provider Communication was mixed with ↓ in provider communication item about Listened carefully, ↔ for provider communication item about Explained things, ↓ in provider communication item about Spent enough time and ↓ in provider communication item Gave easy to understand information and also Access was mixed with. ↓ in access item about Got appointment for non-urgent care when needed, ↔ for access item about Got appointment for urgent care when needed; ↓ in access item about Got answers to medical questions same day; ↔ for access item about Got answers to medical questions after hours, and ↔ for access item about Seen within 15 min. ^∼ indicates interventions designed to enhance access of extended hours, flexible appointments, after-hours coverage, email, home visits, group visits, web visits, text messages, and telemedicine in CPC clinics (See Linn 2020).

Interventions that Improved Overall Provider Rating. Seven interventions were associated with statistically significant improvement in overall provider rating. One of these 7 studies was also related to improving all aspects of the measured patient experience. That intervention was tele-video visits during the COVID-19 pandemic for primary and specialty care, which had higher overall provider ratings than phone or office visits. ²¹ Two other interventions—coaching and follow-up coaching—improved overall provider rating and provider communication scores. Specifically, medium-performing Federally Qualified Health Center (FQHC) providers saw increases in overall provider rating and provider communication scores after receiving peer shadow coaching ²² ; coaching improved care experiences primarily for English-preferring patients. ²³ Also, follow-up coaching sessions improved patient experience even further. ²⁴ Such gains eroded over time, implying that coaching should recur every 6 to 12 months.^22,24

Four interventions (of the 7) had statistically significant improvements in adjusted mean overall provider rating but mixed results on other measures. Primary care sites that joined QI collaboratives (such as Leading a Culture of Quality Action Group ²⁵ ) focused on improving quality-oriented climate saw an increase in their overall provider rating and a decrease in access scores. ²⁶ Primary care sites that received monthly coaching from a third-party PCMH liaison to encourage and monitor progress on PCMH goals and used PCMH-specific QI strategies saw an improvement in top-box scores (a summary measure of positive responses) for overall provider rating and in follow-up on test results scores, but a decrease in access scores. ²⁷ PCMH sites with more activities related to chronic care management had more significant adjusted increases in mean overall provider rating but worsening scores for access and no change in patient recommendation of provider to friends and family. ²⁸ Outpatient orthopedic or neurology surgery patients who received a Patient-Reported Outcomes Measurement Information System (PROMIS^®) assessment during routine care gave higher overall provider ratings, were more likely to recommend the office, and indicated they spent more time with their providers; other aspects of provider communication were similar whether a patient received the assessment (or not). ²⁹ The PROMIS measures are patient self-reports of physical, mental, and social health.

Furthermore, there was no significant difference in overall provider rating for several interventions. These were a primary care homeless patient PCMH program ³⁰ ; comprehensive primary care model^31,32; various comparisons of PCMH status ³³ ; electronic health record implementation ³⁴ ; outpatient patient portal activation ³⁵ ; coordinated care with an additional coordinator to improve care for the most complex patients ³⁶ ; resident communication training for orthopedic/neurology outpatient care ³⁷ ; real-time provider feedback using the Clinic Satisfaction Tool (CST) ³⁸ ; and, for rheumatology outpatient care, implementation of patient-reported outcomes measures (PROMs). ³⁹

PCMH Interventions that Improved CG-CAHPS Measures. Nine studies^{27,28,30–34,40,41} evaluated the effect of PCMH on patient experience. Of these, 3 PCMH studies^33,34,40 showed improvements in access scores; 1 ³³ also showed improvements in care coordination scores, and the other 2^34,40 found no significant differences in other CG-CAHPS measures. Three other PCMH studies found significantly lower access scores: for homeless patients in a PCMH clinic program (compared to patients with non-PCMH primary care), ³⁰ or for PCMH sites who added a care coordinator and had high use of chronic care management.^27,28 Of the remaining 3 PCMH studies, 1 found no significant difference-in-differences for PCMH (relative to other practices) regarding access incentives ³¹ ; a second found no significant differences in access (or provider communication) scores for PCMH clinics (compared to matched non-PCMH clinics) ³² ; and the third found no significant differences in access scores in comprehensive primary care (CPC) clinics from several interventions, including extended hours, flexible appointments, after-hours coverage, email, home visits, group visits, web visits, text messages, and telemedicine. ⁴¹

Three PCMH-studies found improvements in self-management support and comprehensiveness (ie, attention to mental and emotional health) measures,^30,33,34 but 4 did not^27,31,32,40; 1 PCMH-study did not include this measure. None found differences in the CG-CAHPS PCMH item regarding whether providers were informed about care from other providers.

Care Coordination Interventions that Improved CG-CAHPS Measures. Care coordination interventions (2 studies) had mixed results. One study found significant worsening on the care coordination composite in sites that added care coordinators to improve care for the most complex patients. ³⁶ The other study, which added care coordination to nurses’ roles, showed no significant changes in care coordination scores or in the other outcomes measured (ie, access and provider attention to mental or emotional health). ⁴²

Provider Communication Interventions that Improved CG-CAHPS Measures. Regarding provider communication interventions, only experiential relationship-centered communication skills training for primary and specialty care providers ⁴³ and peer shadow coaching (and re-coaching) for primary care providers^22,24 resulted in significant improvements in provider communication scores. Resident communication training and use of the CST data to provide real-time feedback (within 24 hours of an encounter) to providers on provider–patient interactions within orthopedic/neurology clinics were not associated with changes in provider communication scores.^37,38

Patient Engagement Interventions that Improved CG-CAHPS Measures. Two strategies were studied to increase patient engagement in care. One study provided patients with an outpatient patient portal, providing 24/7 access to communicate with their providers and manage their healthcare; patient portal activation improved provider communication and care coordination scores but did not improve the overall provider rating, office recommendation, access, or office staff scores. ³⁵ The dimensions of patient experience that were not improved by activated patient portal status were those that the study authors indicated were unlikely to be related to patient portal use, such as better experiences with office staff (ie, office staff courteousness scores).

The other study of patient engagement compared 2 strategies commonly used to promote appropriate opioid use for chronic pain and found that patient-directed education was more likely than physician-directed clinical decision support (via electronic health record) to be associated with better patient–provider communication scores. ⁴⁴

PROMs Interventions that Improved CG-CAHPS Measures. Regarding PROMs, 2 studies found improvements with the use and discussion of PROM information by providers in the patient-provider encounter.^29,45 One study found that the use and discussion of PROM information by providers in the patient–provider encounter significantly increased the time providers spent talking about concerns during routine care for orthopedic/neurology patients. ²⁹ The second study found that the use and discussion of PROM assessments by providers in the patient–provider encounter of physical function, pain interference, and depression for new outpatient orthopedic foot and ankle surgery patients reported worse experiences with surgeons explaining things in a way that was easy to understand; however, the overall provider rating and the other aspects of provider communication measured were similar to patients whose provider did not use or discuss PROMs. ⁴⁵ A third study ³⁹ found no effect from the use of PROM assessments.

Cross-sectional Associations With CAHPS Measures

Twenty-one studies examined relationships of patient experience with other variables, including patient-, provider-, and site-level factors. Additionally, 1 study examined associations with survey mode ⁴⁶ and several others examined which CAHPS measures are most correlated with the overall provider rating.^46–49

Table 3 summarizes the 21 cross-sectional studies and the statistical associations with patient experience by CG-CAHPS measures used. We review the findings of the cross-sectional associations by site-level, provider-level, and patient-level factors, using bold text to identify the examined factor.

Table 3.

CG-CAHPS Measures Used in Association Studies, by Type of Ambulatory Care (n = 21 Studies).

Author, Year	Association with patient experience	Overall provider rating	Would recommend	Provider communication	Access	Care coordination	Follow-up on test results	Courteous office staff	PCMH item set	Other CAHPS items
Primary and Specialty Care
Bosko 2016	Clinical quality performance	M
Chaitoff 2017	Provider empathy	M		M
Quigley 2014	Provider specialty	↑		↑
Primary Care (PC)
Chung 2020	Provider burnout	↔		↓-inverse	↔
Hasnain 2013	Acculturation/Hispanic ethnicity	M	M	M	M			M		M: SDM, PI
Nembhard 2015	Organizational climate				M		M
Willard-Grace 2021	Provider burnout & engagement	M	M	M
PC in Federally Qualified Health Centers
Lee 2020	PCP-team communication and chronic disease management			↑
Lin 2017	Received colorectal screening			↔
Primary Care for Adult Diabetes Patients
Bauer 2014	Antidepressant adherence			M						M:SDM
Ratanawongsa 2013	Cardiometabolic medication refill adherence			↑
Ophthalmology
Han 2021	Patient demographics, visual acuity and appointment factors	M
Ortho/Neuro
Hageman 2015	Coworker feedback on awareness of impact on others	↑	↑	M
Johnson 2019	Patient demographics and survey mode	M	M
Khan 2021	Preoperative depression for patients after lumbar surgery	↓-inverse	↓-inverse	M	↔
Lanz 2018	Emotional stability	↑	↑	↔
Lapin 2019	Patient reported outcome measures (PROMs)	↑	↑	↑
Matar 2021	Pre-operative/surgical factors	↔
Rabah 2021a	Lumbar postoperative outcomes	M
Rabah 2021b	Provider communication	↑		↑
Zakare-Fagbamila 2020	Clinic performance metrics (eg, waiting-room times)	M		M	M

Note: In some studies, we can also annotate significance on CAHPS measures using ↑ to indicate positive statistically significant associations (P < .05); ↓ to indicate negative statistically significant associations (P < .05); ↔ to indicate no statistical significance found; and M for results that were both positive and negative within the composite measure (ie, mixed). SDM indicates the shared decision-making item about involvement in decisions. PI indicates items about provider interactions.

Site-level factors associated with CAHPS Measures. Four studies examined site-level factors. Leader-reported quality-oriented organizational climate (rather than reported by frontline staff) was negatively associated with getting an appointment for care when needed and having someone from the office follow up and give test results; staff-reported quality-oriented organizational climate (rather than reported by leaders) was positively associated with timely access scores and the measure of having someone follow up and give test results. ⁵⁰ Better primary care practice team communication and chronic disease management were associated with better patient–provider communication scores. ⁵¹ Co-worker feedback on physician's awareness of their impact on others (ie, co-workers belief that a physician understands how his/her behaviors impact others) were positively associated with overall provider rating and would recommend provider, as well as with provider communication measures of showed respect and explained things. ⁵² Less waiting room time and higher clinic-level patient volume (ie, clinic monthly total visits) were associated with better overall provider rating, and access and provider communication scores; less in-room wait time was associated only with better access scores (ie, seen within 15 min of appointment time). ⁵³

Provider-level factors associated with CG-CAHPS Measures. Seven studies examined provider-level factors. Two of these examined associations between CG-CAHPS measures and provider burnout. One of the burnout articles found provider burnout was negatively associated with provider communication scores. ⁵⁴ The other burnout article found that clinicians who had both high burnout and high engagement in the care and treatment of their patients had better provider communication scores, would recommend the office, and overall provider rating scores than clinicians with the 3 other combinations of high/low burnout and high/low engagement in the care and treatment of their patients. ⁵⁵ A third study (of the 7 on provider-level factors) on provider empathy found a positive association with overall provider ratings on the CG-CAHPS Visit survey 2.0 and with some provider communication items. ⁵⁶ A fourth study investigated associations of orthopedic surgeon's personality traits and found a positive association of surgeon's emotional stability with overall provider rating and would recommend provider, but no association of other surgeon personality traits with overall provider rating and would recommend provider. ⁴⁸ A fifth study found that better physician clinical quality performance on 3 Healthcare Effectiveness Data and Information Set (HEDIS) measures were positively associated with overall provider rating. ⁴⁹ HEDIS measures assess the percentage of time that the physicians perform a specific action for their patients; for example, the antibiotics measures assess a physician's appropriate antibiotic prescribing for 3 targeted clinical diagnoses: bronchitis, upper respiratory infection, and pharyngitis. The sixth study in a community health center found no association between provider communication scores and whether patients had received colorectal cancer screening. ⁵⁷

The seventh study found that for 23 out of 28 provider ambulatory care specialties showing respect was most strongly related to overall provider rating among all CAHPS communication items, and explained things was the least correlated of the communication items. ⁵⁸ However, the importance of specific aspects of communication varied significantly by specialty. For example, spending enough time had the highest correlation to the overall provider rating for interventional radiology, giving easy-to-understand information had the highest correlation to the overall provider rating for geriatric medicine and pulmonary disease, and showing respect had the highest correlation to the overall provider rating for plastic surgery.

Patient-level factors associated with CAHPS Measures. Ten studies examined patient-level factors across several care settings.

Three primary care studies were conducted in a community health center primary care setting, and 2 were specific to diabetic patients. For general primary care, the degree of acculturation for Hispanic patients had mixed associations with the patient's primary care experience. ⁵⁹ The authors found that unacculturated Hispanic patients reported higher scores for overall provider rating, would recommend provider, office staff courteousness, and shared decision-making, whereas bi-cultural Hispanic patients reported higher scores for office staff courteousness and shared decision-making. Neither Hispanic identity was associated with provider communication, access, and supplemental items about provider interactions.

Two studies examined the medication adherence of diabetic patients. One found that antidepressant adherence levels had mixed associations with provider communication and shared decision-making scores. ⁶⁰ Specifically, early non-persistence and primary non-adherence for diabetic patients were associated with lower involvement with shared decision-making but not with provider communication scores. The other study found that refill adherence to cardiometabolic medication was positively associated with improved provider communication scores for diabetic patients. ⁶¹

Four studies examined patients who had lumbar surgery. Pre-operative and surgical factors for lumbar surgery patients such as patient diagnosis, location of surgery (ie, Level I office surgery where anesthesia is local, topical, or none conducted in exam room; or Level III surgery which requires use of a general anesthesia), and discharge disposition were not associated with differences in overall provider ratings. ⁶² Pre-operative depression among lumbar surgery patients was not associated with CAHPS access scores. ⁶³ However, 1 study (and an associated commentary) ⁶⁴ found lower back pain after 1 year post-operation was associated with better overall provider rating, but not with patient-reported general physical or mental health. ⁶⁵ Lastly, 1 outpatient spine surgery study examined associations of nonmodifiable demographic factors and insurance with patient experience. ⁴⁶

For spine surgeons in the outpatient setting, 1 study also examined the relationship between overall rating and aspects of provider communication to identify the specific “drivers” of patient overall rating ⁴⁷ ; they found more positive overall provider rating was most associated with when the spine surgeon showed respect and least associated with having spent enough time and giving easy-to-understand information.

One study found that better patient self-reported PROM scores were associated with better overall provider rating, would recommend, and provider communication items in outpatient care in neurology clinics. ⁶⁶

For ophthalmology, appointment characteristics such as being seen in optometry clinic (vs in a Comprehensive, Cornea, and Glaucoma clinic), having orders (or not) of different types placed for each visit such as a procedure order, surgery order, refraction order, or eye testing order, and patient factors such as higher overall appointment attendance, being a new patient, or having high corrected visual acuity were positively associated with overall provider rating. ⁶⁷

Assessment of Subgroups Using CG-CAHPS Measures

Six studies used CG-CAHPS measures to assess differences by subgroups. Four of these studies found significant subgroup differences. At the site level, ambulatory clinics associated with teaching hospitals had better access scores than non-teaching hospitals but did not differ in provider communication and office staff courteousness scores. ⁶⁸ At the provider level, a national study of primary care providers found that Doctors of osteopathic medicine and nurse practitioners had better overall provider ratings and provider communication scores than medical doctors (MD) and physician assistants (PA).^69,70 Also, cranial surgeons (vs spinal surgeons) had better-unadjusted top-box scores for overall provider rating, would recommend provider, and provider communication scores, and reported significantly better scores on showed respect and listened carefully. ⁷¹ At the patient level, adult patients with acne (vs not) had significantly worse adjusted top-box scores for the provider listened carefully, explained things, and spent enough time.^69,70

There were no differences by provider–patient gender concordance in adjusted overall provider rating and adjusted scores measuring provider communication, access, care coordination, office staff courteousness, and PCMH-items about self-management support, got information about needed care on evenings/weekends/holidays, and informed about care from specialists. ⁷²

CG-CAHPS Measures and External Benchmarking

One study conducted external benchmarking. Kim et al 2021 ⁷³ identified differences in the overall provider rating of the same provider, whether measured by CAHPS versus external physician-review websites for outpatient visits with oncology physicians; the ratings were higher for CAHPS measures.

Supplemental Table 1 provides detailed study findings for each CG-CAHPS measure and rating (see Supplemental Digital Content 2).

Discussion

This review builds on a previous systematic review, which documented the types of studies that used CG-CAHPS survey data since its inception but did not review study findings. It systematically examines the evidence on which factors are significantly associated with CG-CAHPS measures and which interventions significantly changed CG-CAHPS measures as outcomes. We identified several interventions and site-level or provider-level factors that practice leaders can implement as strategies to improve patient experience measured by CG-CAHPS.

Interventions that significantly improved the CG-CAHPS overall provider rating included participation in QI collaboratives focused on improving quality-oriented climate, implementing PCMH-specific QI strategies, increased chronic care management activities, using peer shadow coaching for providers, offering tele-video visits as an encounter option (other than office or phone visits), and performing PROMIS^® assessments during routine care. Experiential relationship-centered communication skills training and peer shadow coaching significantly improved provider communication. PCMH interventions, use and discussion of patient-reported outcomes during an encounter, and care coordination interventions had mixed improvements on patient experience.

Site-level factors associated with better patient experience included better team communication. Better primary care practice team communication and increased chronic disease management activities were associated with better provider communication. Co-worker feedback on physicians’ awareness of their impact on others was associated with a higher overall provider rating, and the provider showed respect and explained things. Less in-room wait time was associated with better access scores. Less waiting room time and higher clinic-level patient volume (ie, clinic monthly total visits) were associated with better overall provider rating, access, and communication. Provider-level factors associated with better patient experience included more empathy and emotional stability. More physician empathy was associated with better provider communication, whereas the emotional stability of orthopedic surgeons was associated with higher provider ratings. Better physician performance on clinical quality measures was associated with better overall provider ratings. Provider burnout had mixed associations with patient experience and may be mediated by provider engagement in the care and treatment of their patients.

Our search focused on peer-reviewed literature with adequate study quality since we were interested in studies with broad generalizability. This approach excluded non-peer-reviewed work, such as foundation reports, single-case evaluation of QI, and other healthcare-driven patient-centered initiatives, since they are hard to publish in peer-reviewed journals but often contain helpful lessons. Future work that includes a grey literature review identifying and describing existing QI strategies for rural or small practices may help the field, as would work summarizing the evidence for interventions targeting specific CG-CAHPS domains or item sets.

This examination of research evidence using CG-CAHPS data has several implications. CG-CAHPS survey data have been consistently and widely used in research as valid measures of patient experiences of ambulatory care to assess overall care and specific aspects of care. CG-CAHPS measures are related to and reflect site changes and provider-level and patient-level factors. Importantly, CG-CAHPS measures are sensitive to changes in outcomes and associations across a wide range of ambulatory settings, including general primary and specialty care, care at community health centers (ie, FQHCs), primary care specific to diabetic patients, primary care for Medicare beneficiaries, outpatient care for orthopedic, spine and neurological surgical patients and multiple specialties (eg, Ophthalmology, Rheumatology, Dermatology). Despite this, CG-CAHPS surveys are not required by the Centers for Medicare and Medicaid Services (CMS) for performance-based payment adjustments to clinicians’ Medicare payments; however, the CMS Merit-based Incentive Payment System (MIPS) uses the CAHPS for MIPS survey, which includes the 10 core CG-CAHPS measures, as an optional quality measure. This evidence review demonstrates the broad use of CG-CAHPS survey data for measuring patient experience of ambulatory care.

Our review has limitations. First, we found inconsistent use of the same specific metrics of patient experience across the studies. There was heterogeneity across study settings (ie, about half of the studies took place in primary care settings [27 of 52]), selected outcome measures of patient experience (ie, overall provider rating was in 42 of 52 studies, whereas access scores were in 22 of 52 studies), and calculation of measure scores (ie, some studies calculated case mix adjusted mean scores while others used top-box scores), restricting our ability to perform a meta-analysis. We also by design limited our review to studies using patient's quantitative assessment of their experience using the CG-CAHPS survey data and did not include studies assessing the use of the open-ended narrative text responses from the CG-CAHPS survey, as we were assessing evidence from interventions and associations using CG-CAHPS. We have published on the potential use of CG-CAHPS open-ended responses for QI. ⁷⁴ However, qualitative studies based on narrative text responses are not appropriate for understanding significant improvements related to an intervention or for assessing associated factors or directionality of findings with aspects of patient experience. Our review, however, highlighted possible interventions and factors that merit more attention based on studies using CG-CAHPS scores.

Conclusion

Since its release in 2007, the CG-CAHPS survey has been used by healthcare providers to assess a wide range of pre-post interventional changes in global ratings. CG-CAHPS data is also widely used to understand important cross-sectional relationships with patient experience of care at the site, provider, and patient levels, in addition to assessing differences across patient and provider groups. Our review identified several promising interventions and site-level or provider-level factors that practice leaders can implement to improve patient experience. Policymakers, researchers, clinicians, and healthcare leaders can leverage this evidence for QI efforts and selecting interventions that support patient-centered care.