Performance on Patient Experience Measures of Former Chief Medical Residents as Physician Exemplars Chosen by the Profession

Key Points

Question

What can be learned about the medical profession’s ability to identify physician exemplars using the selection of chief medical residents and their subsequent performance as an example?

Findings

In this observational cross-sectional study using Medicare Fee-For-Service Consumer Assessment of Healthcare Providers and Systems data from 45 771 patients, patients of primary care physicians who were former chiefs reported significantly better care experiences than patients of nonchiefs within the same practice, especially for physician-specific interpersonal items reflecting skills that are typically valued in the chief selection process.

Meaning

The study results suggest that the medical profession possesses information about physician quality and that there may be potential benefits from harnessing such information to identify and repurpose exemplars for quality improvement.

Abstract

Importance

Physicians’ knowledge about each other’s quality is central to clinical decision-making, but such information is not well understood and is rarely harnessed to identify exemplars for disseminating best practices or quality improvement. One exception is chief medical resident selection, which is typically based on interpersonal, teaching, and clinical skills.

Objective

To compare care for patients of primary care physicians (PCPs) who were former chiefs with care for patients of nonchief PCPs.

Design, Setting, and Participants

Using 2010 to 2018 Medicare Fee-For-Service Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey data (response rate, 47.6%), Medicare claims for random 20% samples of fee-for-service beneficiaries, and medical board data from 4 large US states, we compared care for patients of former chief PCPs with care for patients of nonchief PCPs in the same practice using linear regression. Data were analyzed from August 2020 to January 2023.

Exposures

Receiving the plurality of primary care office visits from a former chief PCP.

Main Outcomes and Measures

Composite of 12 patient experience items as primary outcome and 4 spending and utilization measures as secondary outcomes.

Results

The CAHPS samples included 4493 patients with former chief PCPs and 41 278 patients with nonchief PCPs. The 2 groups were similar in age (mean [SD], 73.1 [10.3] years vs 73.2 [10.3] years), sex (56.8% vs 56.8% female), race and ethnicity (1.2% vs 1.0% American Indian or Alaska Native, 1.3% vs 1.9% Asian or Pacific Islander, 4.8% vs. 5.6% Hispanic, 7.3% vs 6.6% non-Hispanic Black, and 81.5% vs. 80.0% non-Hispanic White), and other characteristics. The Medicare claims for random 20% samples included 289 728 patients with former chief PCPs and 2 954 120 patients with nonchief PCPs. Patients of former chief PCPs rated their care experiences significantly better than patients of nonchief PCPs (adjusted difference in composite, 1.6 percentage points; 95% CI, 0.4-2.8; effect size of 0.30 standard deviations (SD) of the physician-level distribution of performance; P = .01), including markedly higher ratings of physician-specific communication and interpersonal skills typically emphasized in chief selection. Differences were large for patients of racial and ethnic minority groups (1.16 SD), dual-eligible patients (0.81 SD), and those with less education (0.44 SD) but did not vary significantly across groups. Differences in spending and utilization were minimal overall.

Conclusions and Relevance

In this study, patients of PCPs who were former chief medical residents reported better care experiences than patients of other PCPs in the same practice, especially for physician-specific items. The study results suggest that the profession possesses information about physician quality, motivating the development and study of strategies for harnessing such information to select and repurpose exemplars for quality improvement.

This cross-sectional study examines care for patients of primary care physicians who were former chiefs compared with care for patients of nonchief primary care physicians.

Introduction

Physicians’ knowledge about each other’s quality is central to clinical decision-making. As physicians practice, they accrue information about the quality of care provided by others. Physicians often find what they observe to be compelling, as evidenced by their formation of strong opinions about other physicians and exercise of such impressions when acting as agents for patients (eg, in selecting specific specialists for referrals).^1,2,3 In turn, patients value and rely on physicians’ recommendations.⁴ Yet, to our knowledge, the content of physicians’ knowledge about quality, including its association with standard measures, has not been well described.

Moreover, the collective insights of physicians are rarely harnessed in quality improvement efforts.⁵ In particular, organizations do not routinely identify exemplars based on peer input and repurpose them to teach, consult, or otherwise disseminate best practices.⁶ Doing so may produce distinct benefits compared with the use of quality measures alone to identify high performers. First, leveraging the information physicians accrue during practice to identify exemplars may help overcome the limitations of profiling physician performance with objective quality measures; these include measure validity issues, missing information on hard-to-measure quality aspects, inadequate risk adjustment, and random error.^{7,8,9,10,11,12,13,14}

Second, peer opinion may help identify physicians who are not only high performers but also skilled teachers and respected colleagues from whom others want to learn; the latter qualities may be difficult to ascertain from performance measures.^8,11,15,16 Third, exemplars must be visible to motivate peers to aspire to the same high standards.^17,18 Whereas physicians may be unaware or distrustful of peers’ high scores on performance measures, they are inherently aware of those whom they perceive to be exemplars. Thus, using the profession’s assessment of physician excellence to identify exemplars may help disseminate best practices within groups by facilitating learning and the application of insights from management science to leverage intrinsic motivation and peer effects in ways that improve quality rather than risk undermining motivation with purely measure-reliant strategies.^{9,10,18,19,20,21,22,23,24,25,26,27}

Prior work on such approaches is limited but shows promise. For example, one initiative that exchanged exemplars across hospitals was followed by changes in more than half of participating hospitals.²⁸ However, the success of repurposing exemplars identified by the profession requires the profession to identify physicians with knowledge or skills that are productive to share.

One example is the case of chief residents, who are typically selected based on supervisor and peer impressions of residents’ teaching, leadership, management, interpersonal, communication, and clinical skills.^29,30 In internal medicine, approximately 6% of eligible residents are chosen to serve as chief medical residents for a year.³¹ In addition to administrative and clinical responsibilities, the stated purpose of the position is to teach residents and serve as visible role models.³² The selection of chiefs presents an opportunity to learn what the profession detects as exemplary physician quality, at least regarding attributes valued in a chief.

In this study, we compared the care for patients with primary care physicians (PCPs) who are former chiefs with the care for patients with nonchief PCPs in the same practice. We focused primarily on patient experiences, as assessed by the Medicare Fee-For-Service (FFS) Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey, because patient experience ratings should reflect the advanced interpersonal and communication skills commonly prioritized in chief selection. We conducted subgroup analyses for historically disadvantaged groups, as they may particularly benefit from PCPs who meet high professional standards. In secondary analyses, we additionally compared the performance of former chiefs and other PCPs on spending and utilization measures.

Methods

Study Data and Sample

For the primary outcome, we analyzed 2010 to 2018 Medicare FFS CAHPS data with linked Medicare enrollment and claims data. The FFS CAHPS survey is administered annually to a nationally representative, cross-sectional sample of FFS beneficiaries. For spending and utilization outcomes, we analyzed Medicare enrollment and claims data during the same years for 20% annual samples of FFS beneficiaries. We limited the study population to 4 large states (Florida, Massachusetts, New York, and Texas) with available medical board data on postgraduate training. The study was approved by the institutional review board of the Harvard Faculty of Medicine with a waiver for informed consent and followed Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.

We assigned each beneficiary in each study year to the PCP accounting for the plurality of office visits with PCPs (eMethods in Supplement 1).^9,33 Because the CAHPS survey was administered early in the year and asked about experiences during the previous 6 months, we used the prior year’s visits for these assignments.³⁴ For spending and utilization analyses, we used the study year’s visits. We restricted to beneficiaries who were continuously enrolled in parts A and B of FFS Medicare without any Medicare Advantage coverage during the previous year (to assess diagnoses) and for the entire study year (or until death).

Chief Resident Identification

We used 2 data sources to identify former chief residents among PCPs in the study sample. First, using detailed postgraduate training data reported by physicians to state medical boards, we identified PCPs who opted to explicitly report a chief residency as additional information or whose training information included 1 additional year of residency at the same institution (eMethods in Supplement 1). Second, for PCPs in the study sample who were not identified as former chiefs using board data, we web scraped for online documentation of chief residency (specifically, a webpage including the PCP’s first name, last name, and chief resident). For PCPs meeting this criterion, we then confirmed former chief status via manual inspection of the webpages and only included confirmed chiefs. We also inspected webpages for physicians identified as former chiefs using board data to characterize the proportion of PCPs categorized as former chiefs in the study who self-identified as a former chief online or in board data.

Because some former chiefs may not report additional training to medical boards or online, our approach was designed to be more specific than sensitive. Thus, we expect that the comparison group of nonchiefs included some former chiefs, but these should constitute a very small proportion, as the share of PCPs who were chiefs is low.³¹ To limit the comparison group to nonchief PCPs in the same practices as chiefs, we used Medicare Data on Provider Practice and Specialty to identify nonchief PCPs who were billing primarily with the same tax identification numbers.

Outcomes

The prespecified primary outcome was a composite of 12 items in the FFS CAHPS survey (Table 1), including 6 that elicited patient ratings of their personal physician specifically (overall rating and 5 items assessing communication and interpersonal performance) and 6 items that assessed general care aspects that could be determined by physician-level or practice-level factors (access to care, care coordination, and overall care). To facilitate interpretation, we dichotomized responses into indicators of a top rating and calculated composites (eMethods in Supplement 1).³⁴ In sensitivity analyses, we examined indicators of low (instead of top) ratings and analyzed ratings as rescaled continuous variables (eFigures 1A, 1B, 3A, and 3B in Supplement 1). In addition to reporting results for the primary composite outcome, we report results for each item and 2 subcomposites (spanning items that were more vs less physician specific) to describe which items contributed more to the overall estimate. We expected larger differences between former chief and nonchief PCPs in physician-specific ratings than in general care ratings that reflect physician-level and practice-level factors.

Table 1. CAHPS Survey Measures of Patient Experiences by More vs Less Physician-Specific Items^a.

Category	Survey item	Survey question	Scale
More physician-specific	Overall rating
	Rating of primary physician	“What number would you use to rate your personal doctor?”	0 (Worst) to 10 (best)
	Interactions with primary physician
	Clear communication	“In the last 6 mo, how often did your personal doctor explain things in a way that was easy to understand?”	Never, sometimes, usually, always
	Careful listening	“In the last 6 mo, how often did your personal doctor listen carefully to you?”
	Respect	“In the last 6 mo, how often did your personal doctor show respect for what you had to say?”
	Concerns answeredb	“In the last 6 mo, how often did you leave your doctor’s office feeling that all of your concerns or questions were fully answered?”
	Sufficient time	“In the last 6 mo, how often did your personal doctor spend enough time with you?”
Less physician-specific	Overall rating
	Rating of health care	“What number would you use to rate all your health care in the last 6 mo?”	0 (Worst) to 10 (best)
	Timely access to care
	Timely access to urgent care	“In the last 6 mo, when you needed care right away, how often did you get care as soon as you thought you needed?”	Never, sometimes, usually, always
	Timely access to nonurgent care	“In the last 6 mo, not counting the times you needed care right away, how often did you get an appointment for your health care at a doctor’s office or clinic as soon as you thought you needed”
	Care coordination and management
	Primary physician informed about specialty carec	“In the last 6 mo, how often did your personal doctor seem informed and up-to-date about the care you got from specialists?”	Never, sometimes, usually, always
	Communication of test resultsd	“In the last 6 mo, when your personal doctor ordered a blood test, x-ray, or other test for you, how often did someone from your personal doctor’s office follow up to give you those results?”
	Timely communication of test resultsd	“In the last 6 mo, when your personal doctor ordered a blood test, x-ray, or other test for you, how often did you get those results as soon as you needed them?”

Abbreviation: CAHPS, Consumer Assessment of Healthcare Providers and Systems.

^a All questions asked in years 2010 to 2018, unless otherwise noted.

^b Survey question asked in years 2014 to 2018.

^c Survey question asked in years 2011 to 2018.

^d Survey question asked in years 2012 to 2018.

The secondary outcomes included annual total Medicare spending and spending for 13 low-value services (eMethods in Supplement 1),³⁵ hospitalizations for ambulatory care-sensitive conditions,³⁶ and emergency department (ED) visits. We focused primarily on patient experiences because they not only directly assess aspects of physician excellence that are typically valued in chiefs but also are normatively interpretable and include items assessing performance specifically at the physician level. In contrast, total spending or utilization differences can be more challenging to interpret, as higher may be better or worse for patients and totals may obscure offsetting differences in lower-value and higher-value care.³⁷ Moreover, differences in PCP practice patterns only partially determine the care received by patients from all clinicians,³⁸ and spending and utilization differences often reflect practice-level patterns.³⁹ Consequently, we expected substantial attenuation bias in analyses of these secondary outcomes.

Covariates

From the Master Beneficiary Summary File, we assessed the following patient characteristics to assess and control for observable differences between patients served by former chief and nonchief PCPs: age, sex, race and ethnicity (based on the Research Triangle Institute classification⁴⁰), dual eligibility for Medicaid, disability, end-stage kidney disease, and conditions from the Chronic Conditions Data Warehouse. From CAHPS, we assessed patients’ self-reported race and ethnicity, educational attainment, smoking status, need for help in completing the survey, and difficulty with activities of daily living. From the Medicare Data on Provider Practice and Specialty, we assessed physician age and sex.

Statistical Analysis

We used the following linear regression model to compare outcomes for patients of former chief PCPs with those of nonchief PCPs:

Y_ijkt = β₀ + β₁PCPFormerChief_j + α_k + δ_t + X_i + Z_j + ϵ_ijkt

in which y is the outcome for patient i in year t of PCP j in practice k, α_k are practice fixed effects (to isolate within-practice comparisons), δ_t are year fixed effects, X_i is a vector of patient covariates, and Z_j is a vector of physician covariates. The coefficient β₁ is the quantity of interest, ie, the adjusted difference in the mean outcome between patients with former chief PCPs and nonchief PCPs. Standard errors were clustered at the practice level. Analyses of CAHPS measures applied survey weights. To facilitate interpretation, we calculated effect sizes by dividing the estimate for β₁ by the standard deviation of mean outcomes across PCPs using a mixed model with physician random effects to estimate the PCP-level variance (eMethods in Supplement 1).³⁴

Analyses were conducted using Stata, version 16 (StataCorp), and R, version 3.6.0 (R Foundation). Statistical significance was set at P < .05.

Subgroup and Sensitivity Analyses

For the primary outcome, we estimated differences between patients of former chiefs and other PCPs within subgroups whose health care has historically been compromised by social disadvantage: patients who were dually eligible for Medicaid, patients of racial and ethnic minority groups (pooled due to limited power for examining each racial and ethnic minority group separately), and patients with no more than a high school education. We conducted an omnibus joint test of heterogeneity by these characteristics but were more interested in estimates within subgroups than differences between subgroups.

We also conducted subgroup analyses by dual eligibility and race and ethnicity for the secondary outcomes (education was only available for the CAHPS sample) but considered these exploratory given the interpretation challenges noted previously and multiple tests. For these secondary outcomes, we used a modified Hochberg procedure to correct for multiple testing in tests of overall differences and subgroup heterogeneity (eMethods in Supplement 1).^41,42

To characterize confounding from observed covariates, we compared unadjusted estimates with estimates that were adjusted for patient and physician covariates. Finally, we explored sensitivity to adjustment for physician panel size, as measured in the 20% samples.

Results

Study Population

Among all PCPs in the CAHPS samples, 1010 (8.5%) were former chiefs who were working in 658 practices. Among former chiefs, 696 (68.9%) were board-identified, and 314 were web-identified. In total, 649 PCPs (64.3%) categorized as former chiefs self-identified as such in board data (171 [26.3%]) or online (478 [73.7%]). The comparison group included 10 849 nonchief PCPs in the same practices.

The CAHPS samples included 4493 patients with former chief PCPs and 41 278 patients with nonchief PCPs. The average CAHPS survey response rate during the study period was 47.6% and differed minimally between patients with former chief PCPs and other PCPs (0.5%). The 20% samples included 289 728 patients with former chief PCPs and 2 954 120 patients with nonchief PCPs.

Patient and physician characteristics differed minimally between former chief and nonchief PCPs in the same practices (Table 2; eTables 1 and 2 in Supplement 1). In contrast, patient characteristics were strongly associated with physician ratings (eTable 4 in Supplement 1).

Table 2. Differences in Patient and Physician Characteristics Between Former Chief Resident PCPs vs Nonchief PCPs From 2010 to 2018^a.

Patient characteristics	Former chief resident PCPs (n = 4493)	Nonchief PCPs (n = 41 278)	Standardized mean difference
Mean (SD) age, y	73.1 (10.3)	73.2 (10.3)	−0.007
Female sex, %	56.8	56.8	0.000
Race and ethnicity from CAHPS, %
American Indian or Alaska Native	1.2	1.0	0.015
Asian or Pacific Islander	1.3	1.9	−0.044
Black, non-Hispanic	7.3	6.6	0.027
Hispanic	4.8	5.6	−0.035
Multiracial	0.3	0.3	−0.006
White, non-Hispanic	81.5	80.0	0.038
Other	1.0	1.1	−0.015
Missing	2.8	3.5	−0.036
Race and ethnicity from MBSF, %
Asian or Pacific Islander	0.9	1.6	−0.052
Black, non-Hispanic	7.9	7.3	0.021
Hispanic	4.7	5.3	−0.028
White, non-Hispanic	84.8	83.9	0.025
Other	0.6	0.9	−0.029
Missing	1.1	1.0	0.003
With a disability, %b	15.3	16.2	−0.024
End-stage kidney disease, %	0.6	0.8	−0.023
CCW conditions, mean (SD)c	5.82 (3.2)	5.84 (3.2)	−0.006
Dually eligible, %d	11.1	11.6	−0.016
Education, %
Less than high school diploma	8.7	9.2	−0.017
High-school diploma	24.1	26.4	−0.052
Some college or 2-y college degree	26.3	25.8	0.011
4-y College degree	14.9	13.2	0.050
More than 4-y college degree	21.7	20.6	0.027
Missing	4.4	4.7	−0.018
Smoker, %	7.3	8.2	−0.033
Proxy CAHPS survey respondent, %	9.6	10.4	−0.026
Had ADLs with difficulty, %e	31.8	32.8	−0.021
Physician characteristics
Female PCP, %	30.7	32.3	−0.034
Mean (SD) PCP age, y	49.7 (9.3)	51.6 (10.1)	−0.191
PCP panel size quartile, No.	3.2	3.2	0.013

Abbreviations: ADLs, activities of daily living; CAHPS, Consumer Assessment of Healthcare Providers and Systems; CCW, Chronic Condition Warehouse; MBSF, Master Beneficiary Summary File; MD-PPAS, Medicare Data on Physician Practice and Specialty; PCPs, primary care physicians.

^a Means were calculated among respondents to the CAHPS survey and adjusted for survey weights, practice, and year. Standardized mean differences were calculated by dividing the difference between chief and nonchief predicted adjusted means by the overall unadjusted standard deviation. Age, sex, race and ethnicity, disability status, dual eligibility status, end-stage kidney disease, and 25 CCW conditions were assessed from the Medicare MBSF. Race and ethnicity, educational attainment, smoking status, whether the respondent needed help completing the survey, and difficulty with ADLs were assessed from CAHPS. Race and ethnicity were separately assessed from the Research Triangle Institute race and ethnicity designation in the MBSF. Reporting of those identified as American Indian or Alaska Native by the MBSF is suppressed due to small cell sizes per the US Centers for Medicare & Medicaid Services terms for data use. The sex and age of PCPs were assessed from the MD-PPAS. Panel size quartiles for PCPs were assessed from the study sample constructed from 20% annual samples of Medicare fee-for-service beneficiaries’ claims.

^b Whether disability was the original reason for Medicare eligibility.

^c CCW conditions included acute myocardial infarction, anemia, asthma, atrial fibrillation, cancer (including breast, colon, endometrial, lung, and prostate), benign prostatic hyperplasia, chronic kidney disease, chronic obstructive pulmonary disease, Alzheimer or other dementias, depression, diabetes, heart failure, hip or pelvic fracture, hyperlipidemia, hyperthyroidism, hypothyroidism, ischemic heart disease, osteoporosis, rheumatoid arthritis or osteoarthritis, and stroke or transient ischemic attack.

^d For Medicaid or Medicare Savings Program.

^e ADLs included bathing, dressing, eating, getting in or out of chairs, walking, and using the toilet.

Patient Experiences

Patients of former chief PCPs rated their care experiences significantly higher than nonchief patients (adjusted difference in composite, 1.6 percentage points; 95% CI, 0.4-2.8; P = .01; Figure 1). The effect size (0.30 SD) is analogous to moving from the 50th to 62nd percentile of PCP performance. Differences were large for patients of racial and ethnic minority groups (1.16 SD), dual-eligible patients (0.81 SD), and those without any college education (0.44 SD) but did not differ significantly from differences within White, non–dual-eligible, or more educated groups (P = .19 for joint test of subgroup interactions; Figure 1).

Figure 1. Differences in Patient Experience Composite Ratings Between Patients With Former Chief Resident Primary Care Physicians (PCPs) and Patients With Nonchief PCPs by Patient Subgroups From 2010 to 2018.

Changes in patient experience composite ratings, presented by subgroup, for patients with PCPs who were former chief medical residents vs the control group of patients with nonchief PCPs in the same practice. The unadjusted overall estimate is adjusted for practice and year but not physician age and sex or patient covariates. The adjusted overall and patient subgroup estimates also adjust for physician age and sex and patient covariates, as described in the article. Adjusted mean outcomes for patients of nonchief PCPs are presented. A P value was estimated for the primary outcome of the adjusted composite in the full sample and was equal to .01. The (pp) abbreviation indicates percentage point.

As expected, the composite difference included differences in physician-specific items (adjusted difference in subcomposite, 1.8 percentage points; 95% CI, 0.4-3.3) that were greater than differences in care dimensions subject to physician-level or practice-level factors (1.3 percentage points; 95% CI, −0.1 to 2.6), although several of the latter items also contributed to the overall difference in the composite score (Figure 2).

Figure 2. Differences in Patient Experience Ratings Between Patients With Former Chief Resident Primary Care Physicians (PCPs) and Patients With Nonchief PCPs by More vs Less Physician-Specific Items From 2010 to 2018.

Adjusted differences in patient experience ratings by more vs less physician-specific ratings are presented for patients with PCPs who were former chief medical residents compared with the control group of patients with nonchief PCPs within the same practice, including adjustment for year, patient, and physician age and sex. Adjusted mean outcomes for patients with nonchief PCPs are presented. The (pp) abbreviation indicates percentage point.

Spending and Utilization

Total spending, low-value spending, ambulatory care-sensitive admissions, and ED visits did not significantly differ between patients of former chief vs nonchief PCPs (Table 3; eTable 3 in Supplement 1). Subgroup analyses suggested meaningfully lower admissions for ambulatory care–sensitive conditions among patients of racial and ethnic minority groups, but estimates did not vary significantly across subgroups (eFigures 4A, 4B, 4C, and 4D and eTable 3 in Supplement 1).

Table 3. Differences in Spending and Utilization Measures Between Patients With Former Chief Resident PCPs and Patients With Nonchief PCPs From 2010 to 2018^a.

Outcomes	Meanb	Difference (95% CI)	P valuec	Effect size (SD)d
Spending
Total spending, $	12 209	96.02 (−40.56 to 232.60)	.17	0.05
Low-value care spending, $e	50	−0.26 (−1.43 to 0.90)	.66	−0.02
Utilization
Ambulatory care-sensitive admissions, No.f	0.03	0.00003 (−0.0008 to 0.0009)	.95	0.004
Emergency department visits, No.	0.63	−0.004 (−0.011 to 0.004)	.32	−0.03

Abbreviation: PCP, primary care physician.

^a Outcomes were winsorized to the 99th percentile. Models adjusted for practice, year, physician age and sex, and patient covariates.

^b Means were adjusted for practice and year and calculated among patients with nonchief PCPs.

^c Significance thresholds were adjusted for the 4 multiple hypothesis tests, none of which were significant (eTable 3 in Supplement 1).

^d Effect sizes were calculated by dividing differences by the standard deviation of physician-level means.

^e Low-value care spending included 13 primary care–associated low-value care services, excluding preoperative services.³⁵

^f Admissions for ambulatory care-sensitive conditions included hospitalizations for diabetes, perforated appendix, chronic obstructive pulmonary disease, hypertension, congestive heart failure, dehydration, bacterial pneumonia, urinary infection, angina, and lower extremity amputation based on Agency of Healthcare Research and Quality definitions.³⁶

Sensitivity Analyses

Findings were qualitatively similar when survey items were analyzed as dichotomized low ratings or continuous rescaled ratings (eFigures 1 and 3 in Supplement 1). Covariate adjustment slightly increased differences (Figure 1; eFigure 2 in Supplement 1), and results were affected minimally by further adjustment for panel size.

Discussion

In this cross-sectional study of 4 states covering nearly 25% of the US population, patients of PCPs who were former chief residents reported significantly better care experiences than patients of other PCPs in the same practice. Contributing most to this overall finding were consistent differences in patient ratings of physicians’ communication and interpersonal skills, which are typically emphasized in chief resident selection processes.^29,30 We also found contributing differences in ratings of overall care and other aspects of patient-centered care, such as timely access to urgent care and test results, further suggesting that former chiefs may address patients’ needs more effectively.

Patient experience differences were particularly large for groups whose health is more adversely affected by social determinants and disability. Estimates for patients of racial and ethnic minority groups, patients who were dually eligible for Medicaid, and those without any college education were analogous to moving from median performance among physicians to the 88th, 79th, and 67th percentiles, respectively. These findings suggest either that these patients may especially value the enhanced aspects of quality that former chiefs offer to all patients or that former chiefs may be better at tailoring their interactions and effort to enhance care experiences for historically marginalized groups. For example, patients with limited health literacy may especially benefit from clear explanations. Alternatively, particularly high-performing former chiefs may prefer to work in practices that disproportionately care for underserved groups. Although estimates differed substantially and consistently for historically disadvantaged subgroups, we could not reject the null hypothesis of no variation between subgroups. Thus, additional research is necessary to understand whether some patients benefit more from exemplar physicians.

In contrast to care experiences, we found little evidence of differences in total spending, low-value spending, admissions for ambulatory care–sensitive conditions, or ED visits between patients of former chiefs and other PCPs, although the results did suggest lower acute care use for subgroups reporting the greatest patient experience gains. One explanation for the inconsistency is that spending and utilization measures are less sensitive to differences in PCP quality than physician-specific measures. Consistent with such attenuation bias, we found smaller within-practice differences between patients of former chiefs and nonchief PCPs for CAHPS items assessing mostly practice factors (eg, access to routine care). Prior research on low-value care describes practice-level variation and physician-level variation within practices, but also weak correlations between the provision of different types of low-value services; thus, the most evidence-based physicians in a practice may not be reliably identified by the few services included in our analysis.^35,39,43

Another possible explanation for these null findings is that the profession may not value evidence-based practice as much, or be able to discern it as well, as other quality dimensions. Alternatively, evidence-based practice may not be valued in the selection of chiefs specifically; if asked to select the most cost-effective physicians, physicians may be able to do so. For these reasons, this study’s findings should not be interpreted as convincing evidence that the qualities valued in chiefs do not affect utilization, or that the profession cannot identify and learn from evidence-based practitioners.

This study’s findings have potentially important implications for practice and policy. Measured performance of former chiefs did map onto physician quality attributes on which chiefs are typically selected, suggesting that the profession does possess meaningful information about physician quality. We studied the case of chief resident selection, but physicians may also act on this information when placing referrals, consulting peers informally, and choosing from whom to learn and whose recommendations to adopt. When accurately informed, these actions may help promote quality by disseminating best practices and establishing a natural system of rewards (eg, revenue for referrals, reputational benefits, and professional satisfaction from peer recognition) that sidesteps concerns about pay for performance.^{5,6,8,9,11,12,13,14,22,27,44,45,46,47,48}

This study’s findings additionally suggest potential quality gains from harnessing the profession’s “hive mind” more regularly in the practice of medicine, at least for the increasing proportion of physicians practicing in groups.^5,6,20,49 For example, the repurposing of exemplar physicians to disseminate best practices could be done beyond chief residency using fairer processes than those used to select chiefs⁵⁰ and encompassing a targeted range of desired skills (eg, masters of the evidence). Our subgroup analyses further suggest that having exemplars disseminate practices to others or matching patients with exemplars according to potential benefit may have especially sizeable benefits for historically disadvantaged groups.^51,52,53 This study’s findings support the potential of such strategies but not inferences about their effectiveness, which will require further research to elucidate. That being said, the widespread belief that chief residents are instrumental in physician training^28,30,54 suggests returns to redeploying physician excellence.

Insofar as chief residents embody aspirational standards espoused by the profession, this study’s findings also suggest that aspects of quality observable to and valued by patients (care experiences) are also observed and valued by physicians. This concordance is consistent with intact physician agency (that physicians will prioritize what patients value when they act on their behalf) and suggests that policies promoting competition^27,55,56 among organizations to attract patients may be associated with quality improvements that physicians, too, believe are important. Likewise, competition for physician labor could improve care aspects that patients also value.

Limitations

This study had several limitations. First, the findings pertain to the profession’s identification of exemplars, not the best use of exemplars, although understanding the former is important for gauging the promise of the latter. Second, our classification of a physician as a former chief was subject to some measurement error, as thoroughly validated data on chiefs are not available.³² Nevertheless, nearly two-thirds of former chiefs in the current study self-identified explicitly as a former chief to medical boards or online, and the proportion of PCPs identified as former chiefs in the study sample was similar to the proportion reported elsewhere for internal medicine,³¹ suggesting our approach was specific and sensitive. We expect measurement error would bias results toward the null. Moreover, former chiefs likely compose a small share of PCPs whom peers would identify as exemplars; thus, our estimates likely understate quality differences observable by physicians.

Second, we assumed that former chief PCPs and other PCPs in the same practice do not differ in their patient populations or other physician characteristics that might be associated with outcomes. Although former chiefs differed minimally from other PCPs across observed characteristics and adjustments slightly widened rather than attenuated differences in patient experiences, unobserved differences in confounders may remain.

Third, we cannot distinguish whether the study results are attributable to PCP qualities present at the time of the chief selection process or those subsequently developed during the chief year. However, if the study results partially reflect the latter effect, it is encouraging that skills associated with improved patient care experiences can be learned. More research is needed to understand how to best cultivate such skills in training and beyond.

Fourth, the personal physician referenced in the CAHPS survey by the beneficiary may not match the PCP with the most office visits, introducing additional measurement error biasing results for physician-specific ratings toward the null. However, we found qualitatively similar results for overall care ratings, which should be less subject to this concern. Finally, biases and opaqueness in the chief selection process have motivated calls for reforms with which we agree.⁵⁰ We examined former chiefs in this study but would expect that promoting equity in selecting chiefs, or exemplars more generally, would enhance the potential for reducing disparities by identifying and redirecting physician excellence.

Conclusions

In this cross-sectional study, patients of PCPs who were former chief medical residents reported better care experiences than patients served by other PCPs in the same practice. This suggests potential gains in the quality of care from better harnessing the information possessed by physicians.

Supplement 1.

eMethods.

eTable 1. Unadjusted Differences in Patient and Physician Characteristics between Former Chief Resident Primary Care Physicians (PCPs) vs. Non-Chief PCPs, 2010-2018

eTable 2A. Differences in Patient and Physician Characteristics between Medicare Fee-for-Service 20% Sample Patients with Former Chief Resident Primary Care Physicians (PCPs) vs. Patients with Non-Chief PCPs, 2010-2018

eTable 2B. Unadjusted Differences in Patient and Physician Characteristics between Medicare Fee-for-Service 20% Sample Patients with Former Chief Resident Primary Care Physicians (PCPs) vs. Patients with Non-Chief PCPs, 2010-2018

eFigure 1A. Differences in Patient Experience Composite Ratings (Dichotomized Low) between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by Patient Subgroups, 2010-2018

eFigure 1B. Differences in Patient Experience Composite Ratings (Continuous) between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by Patient Subgroups, 2010-2018

eFigure 2. Differences in Patient Experience Ratings between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by More vs. Less Physician-Specific Items, 2010-2018 – Unadjusted for Patient Characteristics

eFigure 3A. Differences in Patient Experience Ratings (Dichotomized Low) between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by More vs. Less Physician-Specific Items, 2010-2018

eFigure 3B. Differences in Patient Experience Ratings (Continuous) between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by More vs. Less Physician-Specific Items, 2010-2018

eFigure 4A. Patient Subgroup Analyses for Total Medical Spending Outcome

eFigure 4B. Patient Subgroup Analyses for Low-Value Care Spending Outcome

eFigure 4C. Patient Subgroup Analyses for Ambulatory Care-Sensitive Admissions Outcome

eFigure 4D. Patient Subgroup Analyses for Ambulatory Care-Sensitive Admissions Outcome

eTable 3. Multiple Hypothesis Corrections for Secondary Analyses of Spending and Utilization Outcomes

eResults.

eTable 4. Association of Patient Characteristics with Physician Ratings and with Former Chief Resident PCP

eFigure 5. Differences in Patient Experience Composite Ratings between Patients with Former Chief Resident Primary Care Physicians (PCPs) and Patients with Non-Chief PCPs, by Physician Age, 2010-2018

eReferences.

Supplement 2.

Data sharing statement