Skip to main content
NCBI home page
Health Services Research logoLink to Health Services Research
. 2009 Jun;44(3):880–901. doi: 10.1111/j.1475-6773.2009.00960.x

Organizational and Market Influences on Physician Performance on Patient Experience Measures

Hector P Rodriguez 1, Ted von Glahn 2, William H Rogers 3, Dana Gelb Safran 4
PMCID: PMC2699913  PMID: 19674429

Abstract

Objective

To examine the extent to which medical group and market factors are related to individual primary care physician (PCP) performance on patient experience measures.

Data Sources

This study employs Clinician and Group CAHPS survey data (n=105,663) from 2,099 adult PCPs belonging to 34 diverse medical groups across California. Medical group directors were interviewed to assess the magnitude and nature of financial incentives directed at individual physicians and the adoption of patient experience improvement strategies. Primary care services area (PCSA) data were used to characterize the market environment of physician practices.

Study Design

We used multilevel models to estimate the relationship between medical group and market factors and physician performance on each Clinician and Group CAHPS measure. Models statistically controlled for respondent characteristics and accounted for the clustering of respondents within physicians, physicians within medical groups, and medical groups within PCSAs using random effects.

Principal Findings

Compared with physicians belonging to independent practice associations, physicians belonging to integrated medical groups had better performance on the communication (p=.007) and care coordination (p=.03) measures. Physicians belonging to medical groups with greater numbers of PCPs had better performance on all measures. The use of patient experience improvement strategies was not associated with performance. Greater emphasis on productivity and efficiency criteria in individual physician financial incentive formulae was associated with worse access to care (p=.04). Physicians located in PCSAs with higher area-level deprivation had worse performance on the access to care (p=.04) and care coordination (p<.001) measures.

Conclusions

Physicians from integrated medical groups and groups with greater numbers of PCPs performed better on several patient experience measures, suggesting that organized care processes adopted by these groups may enhance patients' experiences. Physicians practicing in markets with high concentrations of vulnerable populations may be disadvantaged by constraints that affect performance. Future studies should clarify the extent to which performance deficits associated with area-level deprivation are modifiable.

Keywords: Performance measurement, ambulatory care quality, doctor–patient relationship

Patient experience surveys have become central to performance measurement activities nationwide, including pay-for-performance and public reporting initiatives (Cleary 1999; Damberg et al. 2005). Little is known about the extent to which characteristics and activities of medical groups and market factors are related to individual physician performance on patient care experience measures. For example, physicians belonging to medical groups that provide stronger financial incentives and organizational support for improving performance on patient experience measures may be more likely to perform well.

Previous studies indicate that patients of staff/group model health maintenance organizations (HMOs) consistently report worse experiences of care and lower visit continuity compared with patients of network HMOs (Safran, Tarlov, and Rogers 1994; Safran et al. 2000; Safran et al. 2002). Over the past decade, physicians belonging to staff/group model HMOs diversified their health plan contracts in order to gain access to patients who were opting for health plans that offered broader physician networks (Robinson and Casalino 1996; Robinson 2001). In response to competitive pressures, many integrated medical groups also reorganized primary care practices to meet consumer demands for patient-centeredness and choice (Grumbach and Bodenheimer 2004; Stein, Frankel, and Krupat 2005; Grembowski et al. 2008). No study, however, has assessed the association of medical group factors and individual physician performance on patient experience measures. Medical groups have a stronger influence on patients' experiences of care than health plans (Solomon et al. 2002; Safran et al. 2006a), so assessing medical group influences on physician performance on patient experience measures might offer important insight regarding the most effective methods for stimulating improvement.

Performance on patient experience survey measures might also be influenced by factors that cannot easily be modified by physicians, such as the availability of physicians in an area. An assumption when employing physician-level performance incentives is that physicians have some control over the care processes and outcomes being measured and rewarded. Case mix adjustment of performance measures is a common method for ensuring equitable comparisons (Zaslavsky et al. 2001) and most survey initiatives currently employ these statistical adjustments. There is increasing concern, however, that pay-for-performance programs might exacerbate racial and ethnic disparities in care quality (Casalino and Elster 2007; Felt-Lisk, Gimm, and Peterson 2007; Millett et al. 2007; Snyder and Neubauer 2007) because physicians working with disadvantaged populations might face difficulties improving performance. For example, patients who receive care in markets with primary care physician (PCP) shortages may experience difficulty scheduling appointments, longer office wait times, and clinical interactions that focus on resolving immediate medical problems rather than eliciting concerns and negotiating an agenda (Kroenke 1998; Marvel et al. 1999; Peltenburg et al. 2004; Rodriguez et al. 2008). As a result, lower financial resources for practices in resource constrained environments compared with other practices might cause or exacerbate performance differences and racial and ethnic disparities in care quality.

It is not yet known, however, the extent to which individual PCP performance on patient experience measures is related to market factors that are difficult to modify or the characteristics and activities of medical groups. This study assesses the extent to which organizational and market factors are related to individual physician performance on patient experience measures using survey data from the largest pay-for-performance program in the United States—the Integrated Health Association's statewide initiative in California (Damberg et al. 2005).

METHODS

Patient Sampling and Survey Administration

The study draws on commercially insured patients who had visits with 2,286 PCPs of adult patients from 43 medical groups in California during 2005 and 2006. Medical groups included a diverse range of physician organizations, including 27 independent practice associations (IPAs), 12 integrated medical groups, and four “hybrid” groups that are composed of a core integrated group and an associated IPA. During each of the 2 survey years (2006 and 2007), a random sample of approximately 100 patients per physician who had at least one visit with their PCP during the prior year were mailed a survey with items from the Clinician & Group CAHPS survey, a previously validated instrument that measures patients' experiences with a specific, named physician and that physician's practice (Agency for Healthcare Research Quality et al. 2006; Safran et al. 2006a). Mailings included an invitation letter, a printed survey, and a postage-paid return envelope. The survey invitation included a personal online code that gave respondents the option of completing the survey using the web. Previous work demonstrated the absence of web survey mode effects for questions in the Clinician & Group CAHPS survey (Rodriguez et al. 2006). The invitation listed a toll-free number for patients to obtain surveys in Spanish. A second invitation and questionnaire were sent to nonrespondents 2 weeks after the initial mailing. Each annual data collection effort spanned a period of approximately 8 weeks.

Patient Survey Content

For this study, we analyzed four composite measures: physician communication (six items), access to care (five items), care coordination (two items), and office staff interactions (two items) (Table 1). The physician communication, access to care, and office staff interactions measures represent core item and composite content of the Clinician & Group CAHPS Survey (Agency for Healthcare Research Quality et al. 2006), which was endorsed by the National Quality Forum for use in evaluating ambulatory care received from individual physicians and their practices. Survey questions use a six-point response scale ranging from “Never” to “Always” and reference care received from the patient's PCP and the PCP's practice over the prior 12 months. All composite measures analyzed achieve physician-level reliability of 0.70 or higher with samples of 40 established patients per physician (Safran et al. 2006a; Rodriguez et al. 2007).

Table 1.

Clinician & Group CAHPS Composite Measures, Summary Statistics

Patient-Level
 Physician-Level
Measure Mean SD Mean SD Minimum Maximum αMD at 40 patients/MD
Physician communication (α=0.94) 89.0 17.9 88.4 6.6 47.9 99.0 0.81
How often did this doctor explain things in a way that was easy to understand? 90.5 18.5 90.0 6.4 53.7 100.0 0.78
How often did this doctor listen carefully to you? 90.1 19.5 89.5 6.7 46.7 100.0 0.78
How often did this doctor give you easy-to-understand instructions about what to do to take care of the health problems or concerns that were bothering you? 89.6 19.7 88.9 6.5 53.8 100.0 0.75
How often did this doctor seem to know the important information about your medical history? 86.2 21.7 85.4 7.4 47.5 98.9 0.77
How often did this doctor spend enough time with you? 86.4 22.3 85.6 8.2 38.7 100.0 0.81
How often did this doctor show respect for what you had to say? 91.5 18.8 90.9 6.5 47.5 100.0 0.78
Care coordination (α=0.64) 78.5 28.4 77.3 10.5 35.0 99.5 0.82
How often did this doctor seem informed and up-to-date about the care you got from specialist doctors? 81.5 26.9 80.2 9.5 30.0 98.9 0.76
When this doctor sent you for a blood test, X-ray, or other test, how often did someone from the doctor's office follow-up to give you the test results? 76.6 33.3 75.3 13.3 30.0 100.0 0.85
Access to care (α=0.84) 75.6 21.8 75.0 10.2 34.0 97.5 0.90
When you called this doctor's office to get an appointment for care you needed right away, how often did you get an appointment as soon as you thought you needed it? 90.0 25.2 80.5 10.5 32.2 100.0 0.87
When you made an appointment for a check-up or routine care with this doctor, how often did you get an appointment as soon as you thought you needed it? 84.2 22.9 83.7 8.9 36.0 100.0 0.84
When you called this doctor's office with a medical question during regular office hours, how often did you get an answer to your question that same day? 75.4 28.6 74.5 11.9 24.0 99.3 0.86
When you called this doctor's office after regular office hours, how often did you get the medical help or advice you needed? 72.7 33.5 71.9 14.9 16.0 100.0 0.82
Wait time includes times spent in the waiting room and exam room. In the last 12 months, how often did your visits at this doctor's office start within 15 minutes of your appointment? 63.3 31.4 62.3 15.6 11.9 97.1 0.91
Office staff interactions (α=0.89) 83.9 21.5 83.2 8.1 43.1 98.7 0.82
How often were clerks and receptionists at this doctor's office as helpful as you thought they should be? 80.5 24.1 79.7 9.1 36.3 98.3 0.82
How often did clerks and receptionists at this doctor's office treat you with courtesy and respect? 87.3 20.9 86.7 7.3 49.5 100.0 0.79
Open in a new tab

All questions use a response scale ranging from “Always” to “Never” and reference care received during the prior 12 months.

α, Chronbach's α or internal consistency reliability; αMD, physician-level reliability or the concordance of patient responses within physician samples.

Patient Sample

Of 332,326 outgoing surveys over the 2 study years, 11,964 (3.6 percent) were undeliverable because of bad address information or patient death. Surveys were received from 120,952 respondents, yielding an unadjusted response rate of 36.4 percent and an adjusted response rate of 37.8 percent. The analytic sample included 112,650 respondents (average per physician=50.1) who confirmed having seen their PCP during the prior 12 months. Respondents who did not confirm the named physician as their PCP or indicated that they did not visit the physician during the prior 12 months (n=8,302) were excluded from the analysis.

Patient Survey Composite Scoring

As detailed elsewhere (Safran et al. 2006a), the survey composite scores could range from 0 to 100 points, with higher scores indicating more favorable performance. For example, a response of “Never” would be scored as “0” and a response of “Always” would be scored as “100”. Composite scores were computed for each respondent based on the unweighted average of responses to all items comprising the measure. Following the half-scale rule (Nunnelly and Bernstein 1994), respondents had to answer at least 50 percent of questions comprising the composite for a score to be computed. Survey question wording, response scales, and placement in the survey were identical during the 2 study years.

Medical Group Interview

Medical group director interviews were conducted via telephone between April and June 2007. The eligible medical groups consisted of groups that participated in the individual physician-level patient care experience survey in 2007 (n=43). The interview assessed the medical groups' current financial incentives, including whether PCPs were eligible for performance-based financial incentives and the magnitude of the incentives. In addition, directors were asked about the formula used to calculate physician incentives, including the percent of the incentive that was based on productivity (e.g., average patients seen per day), efficiency (e.g., limiting referrals, effective panel management), patient experience measures, and clinical quality measures. Directors were asked whether their organization was engaged in various patient experience improvement strategies, including sharing patients' experience survey results with physicians in individual feedback sessions, interpersonal skills training for clinicians, business practice redesign, and practice leader compensation for quality performance. Information about the total number of PCPs and medical group type, that is, IPA, integrated medical group, or hybrid model, was also collected.

Of the 43 eligible medical groups, interviews were conducted with 34 medical group directors or designees (79.1 percent response rate). These 34 medical groups account for 93.8 percent of patients (n=105,663) and 93.3 percent of physicians (n=2,099) in the initial sample. Of the nine medical groups that did not respond to the survey, seven were IPAs, one was an integrated medical group, and one was a hybrid group.

To avoid model convergence problems, we consolidated medical group variables when appropriate. First, we calculated a patient experience improvement strategy score for each medical group (α=0.67; range=0–4; median=2, standard deviation [SD]=1.4), which was a count of medical group engagement in sharing patients' experience survey results with physicians in individual feedback sessions, interpersonal skills training for clinicians, business process redesign, and practice leader compensation for quality performance. Second, we summed two medical director responses to questions that assessed the weight given to productivity and efficiency criteria in individual physician financial incentive formulae (α=0.32; range=0–95 percent; median=0 percent, SD=26.7 percent). The use of productivity and efficiency criterion to stimulate quality improvement has previously been associated with unintended consequences, including physician practice dissatisfaction (Grumbach et al. 1998).

Market-Level Data

Measures based on Primary Care Services Area (PCSA) data (Goodman et al. 2003) were used to characterize the market environment of PCP practices, including the PCP supply per 100,000 age and sex adjusted population (median=75.0, range: 20.1–175.1). The Division of Shortage Designation of the United States Department of Health and Human Services has defined Health Professional Shortage Areas as fewer than 60 PCPs per 100,000 unadjusted population (Center for Health Workforce Studies 2004). In addition, the area-level deprivation or the percentage of the adult population at or below 200 percent of the federal poverty level, and proportion of African American, Hispanic, elderly (age 65+), and migrant residents within each PCSA were assessed for each PCP practice. The participating physician practices are distributed across 140 diverse PCSAs that encompass markets with varying percentages of residents at or below 200 percent of the federal poverty level (median: 24.6 percent, range: 4.8 to 70.2 percent). The population within each PCSA ranged from 4,337 to 1,318,612 residents with a median of 100,369 residents.

Statistical Analyses

Associations between medical group and market factors and survey composite measures were examined using Pearson's correlations. We assessed the correlation of survey composite scores and medical group characteristics and activities using medical group-level data. Similarly, we assessed the correlation of survey composite scores and market characteristics using PCSA-level data. To assess the relative contribution of medical group and market factors to individual physician performance on patient experience measures, four-level multilevel regression models were examined for each composite measure. These multilevel regression models were specified using the XTMIXED module in Stata 10.0 to account for the clustering of patients within physicians, physicians within medical groups, and medical groups within PCSAs using physician, medical group, and PCSA random effects. In order to avoid convergence problems, we calculated collinearity diagnostics for the medical group variables and ultimately chose to include the following medical group predictors in all models: medical group type, number of PCPs per medical group, patient experience improvement activities count, and the use of productivity and efficiency incentives in individual physician financial incentive formulae. PCSA-level covariates were substantially correlated with one another, for example, the proportion of Hispanic residents and area-level deprivation. As a result, we included area-level deprivation (the percentage of residents at or below 200 percent of the federal poverty level) as the sole PCSA-level covariate. Models also controlled for patient characteristics commonly used to adjust patient experience measures (Zaslavsky, Zaborski, and Cleary 2000; Zaslavsky et al. 2001): patient age, gender, race/ethnicity, education, and self-rated physical health. All continuous measures, for example, age, self-rated health, were standardized to a mean of 0 and a variance of 1 for the comparable interpretation of regression coefficients.

RESULTS

Medical Group Financial Incentives and Quality Improvement Activities

There was substantial diversity in medical group financial incentive formulae for individual physician performance (Table 2). Clinical quality criteria were generally given the most weight in financial incentive formulae (mean=29.7.1 percent, SD=21.6), while, on average, performance on patient experience measures (mean=21.8 percent, SD=15.8) and productivity/efficiency (mean=21.8 percent, SD=26.7) criteria were equally emphasized. Though all of the medical groups sponsored physician-level patient care experience surveying, seven groups (24.1 percent) increased the weight given to performance on patient care experience measures since the introduction of individual physician performance-based financial incentives in 2004, a single group decreased the weight, and the remaining groups maintained the weight overtime. A high percentage disseminated each physician's patient care experience results among their peers, either in a blinded (50.0 percent) or unblinded (14.7 percent) fashion, while several (32.4 percent) did not share individual physician results with colleagues. Most medical groups were engaged in at least one quality improvement strategy targeted at improving patient care experience performance (82.3 percent). The most common patient experience performance improvement strategy was interpersonal skills training for clinicians (67.7 percent), and the least common activity was practice leader compensation for quality performance (41.2 percent).

Table 2.

Medical Group Characteristics: Financial Incentives and Patient Experience Improvement Activities

Medical Group Characteristic N (%) Mean (SD)
Medical group type
Independent practice association 20 (58.8)
Integrated medical group 11 (32.4)
Hybrid model 3 (8.8)
Primary care physicians per medical group
Overall 55.5 (75.5)
Independent practice associations 46.9 (84.7)
Integrated medical groups 68.3 (62.4)
Hybrid models 72.5 (40.0)
Financial incentive strength
PCPs not eligible 5 (14.7)
PCPs eligible, 10% or less of base compensation 20 (58.8)
PCPs eligible, >10% of base compensation 9 (26.5)
Financial incentive payment formula (weight given)
Productivity/efficiency 21.8% (26.7)
Patient experience 21.8% (15.8)
Clinical quality 29.7% (21.6%)
Other 26.7% (27.9)
Patient care experience financial incentive weight change over time (n=29)
Increase (%) 7 (24.1)
Decrease (%) 1 (3.5)
No change (%) 21 (72.4)
Patients' experience is a top priority (% endorsing statement) 24 (67.7)
Patient experiences results dissemination
No sharing with peers (%) 11 (32.4)
Share with peers—blinded (%) 17 (50.0)
Share with peers—unblinded (%) 5 (14.7)
Patient experience performance improvement strategies
Interpersonal skills training 23 (67.7)
Business practice redesign 19 (55.9)
Individual physician feedback on patient experience performance 19 (55.9)
Practice leader compensation for quality performance 14 (41.2)
Total patient experience improvement strategies used
None 6 (17.7)
1 4 (11.8)
2 8 (23.5)
3 9 (26.5)
4 7 (20.6)
Open in a new tab

PCPs, primary care physicians.

Bivariate Associations

Table 3 summarizes the bivariate associations of medical group and market characteristics and each survey composite measure. Most correlations were modest (range: −0.17≤r≤0.25) and some variables had consistent associations with the composite measures. For example, greater area-level deprivation (percentages of residents in PCSAs at or below 200 percent of the federal poverty level) was associated with lower performance on the access to care (r=−0.13) and care coordination (r=−0.13) measures. Most medical group activities were not associated with performance, but the number of patient experience improvement strategies used by medical groups was positively associated with performance on the access to care (r=0.14) and office staff interaction (r=0.13) composites. Medical groups with greater numbers of physicians had significantly better performance on the access to care (r=0.25), care coordination (r=0.13), and office staff interactions (r=0.19) measures. There were also some counterintuitive bivariate associations. For example, sharing the patient experience results of peers with individual physicians in an unblinded fashion was consistently associated with worse performance (range:−0.17≤r≤−0.07).

Table 3.

Bivariate Associations of Medical Group and Market Factors and Clinician & Group CAHPS Composite Scores

Physician Communication
 Access to Care
 Care Coordination
 Office Staff Interactions
r p-value r p-value r p-value r p-value
Medical group characteristics
Medical group type
Independent practice association −0.09 −0.01 −0.11 −0.07
Integrated medical group 0.06 0.02 0.11 0.06
Hybrid model 0.06 −0.01 0.01 0.02
Primary care physicians per medical group 0.08 0.25 *** 0.13 * 0.19 ***
Financial incentive strength
PCPs not eligible for incentives −0.06 −0.04 −0.12 * −0.08
PCPs eligible for incentives, 10% or less of base compensation 0.12 * −0.04 0.10 0.07
PCPs eligible for incentives, > 10% of base compensation −0.09 0.08 −0.03 0.12 *
Financial incentive payment formula
Productivity/efficiency (%) −0.07 0.07 0.02 0.11
Patient experience (%) −0.01 −0.09 0.00 −0.07
Clinical quality (%) 0.11 0.02 0.03 −0.02
Other (%) 0.04 0.01 0.12 * 0.05
Patient experience financial incentive weight change
Increase −0.09 −0.08 −0.04 −0.15 *
Decrease −0.08 −0.14 * −0.06 0.01
No change 0.11 0.14 * 0.06 0.13 *
Patient experiences results dissemination
No sharing with peers 0.13 * −0.02 0.09 −0.06
Share with peers—blinded −0.04 0.03 0.02 0.11
Share with peers—unblinded −0.07 −0.17 ** −0.17 ** −0.15 **
Patient experience improvement strategies score 0.01 0.14 * 0.05 0.13 *
Market characteristics
PCP supply per age and sex adjusted population −0.04 0.02 0.05 −0.10
Black population (%) 0.06 −0.06 −0.07 −0.01
Latino population (%) 0.01 −0.04 −0.04 0.02
Age 65+(%) 0.00 −0.04 0.03 −0.04
Area-level deprivation (percent at or below 200% of the federal poverty level) −0.01 −0.13 * −0.13 * −0.02
Migrant population (%) 0.06 0.07 0.08 0.06
Open in a new tab

Note Correlations were calculated at the medical group level for medical group variables and at the PCSA level for market variables.

*

p<.05,

**

p<.01,

***

p<.001.

PCP, primary care physician; PCSA, primary care services area.

Multilevel Predictors of PCP Performance

Table 4 presents the multilevel regression model results for the four composite measures. The number of medical group patient care experience improvement strategies used by medical groups was not associated with individual physician performance on any of the survey composite measures. Physicians belonging to integrated medical groups had better performance on the communication (p=.007) and care coordination (p=.02) composites compared with physicians belonging to IPAs. Physicians belonging to medical groups with greater numbers of PCPs had significantly better performance on all composite measures (range: 0.81–2.39 points per SD increase in the number of PCPs). Physicians belonging to medical groups that placed more emphasis on productivity and efficiency in individual physician financial incentive formulae had worse performance on the access to care (p=.03) composite measure. Physicians practicing in PCSAs with greater area-level deprivation had worse performance on the access to care (−0.77 points per SD increase in area-level deprivation, p=.04) and care coordination (−1.37 points per SD increase in area-level deprivation, p<.001) composite measures.

Table 4.

Multilevel Predictors of Individual Physician Performance on Patient Experience Measures: Final Models

Physician Communication
 Access to Care
 Care Coordination
 Office Staff Interactions
β p-value β p-value β p-value β p-value
Patient characteristics
Age 2.16 <.001 3.15 <.001 3.85 <.001 3.66 <.001
Gender (male) 0.49 <.001 0.64 <.001 1.30 <.001 0.58 <.001
Race/ethnicity
White — — — — — — — —
Black 0.33 <.001 0.32 <.001 0.49 <.001 0.64 <.001
Latino 0.12 .05 −0.31 <.001 −0.09 <.001 0.32 <.001
Asian −1.01 <.001 −1.47 <.001 −1.36 <.001 −1.53 <.001
Other race −0.27 <.001 −0.43 <.001 −0.33 <.001 −0.26 <.001
Education
College graduate — — — — — — — —
Less than high school 0.39 <.001 0.56 <.001 1.10 <.001 0.91 <.001
High school graduate 0.80 <.001 1.34 <.001 1.52 <.001 1.59 <.001
Some college 0.69 <.001 1.16 <.001 1.23 <.001 1.23 <.001
Physical health, self-rated 3.01 <.001 2.88 <.001 3.32 <.001 2.41 <.001
Medical group characteristics
Medical group type
Independent practice association — — — — — — — —
Integrated medical group 1.55 .007 0.25 .73 2.07 .03 1.04 .11
Hybrid model 1.56 .07 −0.12 .91 0.85 .55 0.37 .71
Primary care physicians per medical group 0.81 .01 2.39 <.001 1.12 .04 1.61 <.001
Patient experience improvement strategies count −0.20 .51 −0.49 .22 −0.65 .21 −0.47 .19
Financial incentive formula—productivity/efficiency (%) −0.54 .07 −0.79 .04 −0.50 .32 −0.19 .59
Market characteristics
Area-level deprivation (percent of population at or below 200% of FPL [%]) −0.33 .19 −0.77 .04 −1.37 .001 −0.45 .13
Constant 87.0 <.001 80.0 <0.001 75.7 <.001 82.1 <.001
Open in a new tab

Statistically significant medical group and market variables are bold.

DISCUSSION

This study assessing the influence of medical group and market factors on individual physician performance on patient experience measures has several important findings relevant to the design and implementation of public reporting and pay-for-performance programs. First, our medical group survey results indicate that most of the California medical groups using Clinician & Group CAHPS measures to monitor individual physician performance on patient experience measures are engaged in activities aimed at improving performance on patient experience measures, including interpersonal skills training for clinicians. However, the use of more performance improvement strategies by medical groups was not associated with better performance on any of the survey composite measures. One possible explanation for the lack of association is that some medical groups with lower performing physicians are more likely to adopt strategies to facilitate performance improvement, while groups with high performing physicians do not invest in many of the strategies because of low perceived need. Another explanation is that the “reach” of medical groups' improvement activities to individual physicians explains performance variability. For example, physicians belonging to medical groups that use interpersonal skills training for a large proportion of their practicing physicians might achieve better performance on the communication composite than physicians belonging to groups that only involve a small minority of physicians in training. In addition to longitudinal data on medical group activities, data on the extent of engagement in performance improvement activities, physician attitudes toward financial incentives (Meterko et al. 2006), and organizational culture (Helfrich et al. 2007) could clarify the relationship between engagement in performance improvement activities and physician performance on patient experience measures. Assessments of organizational culture, however, would necessarily rely on multiple informants at various levels of the organizations (Marsden et al. 2006).

Second, greater emphasis on productivity and efficiency criteria in individual physician financial incentive formulae was associated with lower performance on the access to care composite measure. Previous studies suggest that an emphasis on efficiency and productivity criteria might result in unintended consequences, including physician practice dissatisfaction (Grumbach et al. 1998). Our results suggest that these incentives might also affect patients' experiences of care. For example, patients of physicians with strong incentives to see more patients per day may be more likely to experience longer office wait times compared with patients of physicians that do not have these incentives. Given that greater area-level deprivation was associated with worse access to care, productivity and efficiency incentives might be used by groups to encourage physicians to meet excess patient demand in these areas. On the other hand, the use of productivity criteria may be indicative of an medical group culture that emphasizes hierarchical controls to induce performance improvement (Yano et al. 2007; Zazzali et al. 2007). Productivity incentives might not effectively cultivate the working relationships of physicians, advanced practice clinicians, and office staff (Safran, Miller, and Beckman 2006b), and a weak relationship emphasis may spillover to patient care.

Third, physicians belonging to integrated medical groups had better performance on the physician communication and care coordination measures and groups with greater numbers of PCPs had better performance on all measures. This may be because large and integrated medical groups achieve some economies of scale by pooling resources across practices and may devote more attention to developing organized care processes (Casalino et al. 2003) and financial resources for adopting robust clinical information systems. Previous studies have shown that integrated groups are more likely to invest in information technology and other systems to enhance care coordination (Safran, Tarlov, and Rogers 1994; Burton, Anderson, and Kues 2004; Simon, Rundall, and Shortell 2005). Our results related to medical group type differ from previous studies of health plans, which find that patients of staff/group model HMOs have comparable (Gillies et al. 2006) or inferior care experiences (Safran et al. 1998; Safran et al. 2002) compared with other health plans. The results may differ because the past several years have seen market-driven changes in staff/group model HMOs, wherein some of them shed their health plan function to become integrated medical groups who contract with multiple health plans. Moreover, as the market has evolved to focus increased attention on performance measurement and reporting—including measures of both clinical performance and patient experiences—these types of organized, well-resourced groups (Rittenhouse et al. 2004) may have been better positioned than their smaller IPA counterparts to galvanize improvement in the areas subject to public reporting and accountability. Recent initiatives undertaken by integrated groups to improve patient-centeredness might also account for the advantage we observe (Grumbach and Bodenheimer 2004; Stein, Frankel, and Krupat 2005; Grembowski et al. 2008). Our results are consistent with recent evidence that indicates integrated medical groups provide higher technical quality of care compared with IPAs (Casalino 2006; Mehrotra, Epstein, and Rosenthal 2006).

Finally, our results indicate that physicians practicing in markets with greater economic deprivation have worse performance on the access to care and care coordination measures. Our findings mirror recent evidence that high practice concentration of underserved patients can result in lower clinical quality for all patients (Pham et al. 2005; Wilson et al. 2005). Physician supply constraints coupled with the concentrated demands from Medicaid recipients and uninsured patients (Gusmano, Fairbrother, and Park 2002) might also result in worse care experiences for the commercially insured patients in a physician's practice. Information on the percentages of Medicaid and uninsured patients in individual physician practices could help clarify whether “spillover effects” are at play (Pauly and Pagan 2007). Nevertheless, our results raise concern about the equitable comparison of individual physician performance on patient experience measures. It will be important for future studies to clarify the extent to which performance deficits associated with area-level deprivation are modifiable through focused improvement activities or through improved resource allocation to physicians practicing in disadvantaged markets.

Our study results should be considered in light of important limitations. First, the patient survey response rate was modest and it is possible that differential patient nonresponse by individual physician might affect the observed relationships between medical group and market factors and performance. Previous analyses using the same survey measures, however, indicate that the nature of nonresponse does not differ significantly across physicians, and that differences in the extent of nonresponse across physicians are too small to meaningfully affect overall results (Safran et al. 2006a). As a result, nonresponse seems unlikely to threaten the integrity of the physician-level analyses presented. Second, the medical group interviews were conducted with one informant per organization and at one point in time, so the reliability and validity of responses could not be ascertained. However, most questions assessed fairly concrete aspects of medical group financial incentives and participation in various quality improvement activities, so the potential for response bias is limited. Finally, although our patient sample is large, the results might not generalize to other states with different demographic distributions or market conditions or to medical groups not engaged in physician-level performance measurement. However, the medical groups studied are a mix of integrated medical groups and IPAs and are likely representative of groups that are actively engaged in physician-level performance improvement. Given that the participating medical groups are all involved in physician-level performance measurement, they are likely better resourced and more innovative (Rogers 1995; Rye and Kimberly 2007) than medical groups not engaged in the same performance measurement and improvement activities.

In conclusion, the study results suggest that medical group and market factors have important influences on individual physician performance on patient experience measures. Consistent with studies that assess the effects of physician organization and practice size on quality of care (Wilson et al. 2005; Mehrotra, Epstein, and Rosenthal 2006; Landon and Normand 2008; Landon et al. 2008), physicians belonging to integrated medical groups and groups with greater numbers of PCPs had superior performance on several measures. Large and integrated medical groups are more likely to adopt electronic health records (Simon, Rundall, and Shortell 2005; Simon et al. 2007) and adopt more care management processes for common chronic conditions (Casalino et al. 2003). Our study results highlight the value of care integration and coordination from patients' perspectives. The results also indicate that greater emphasis on productivity and efficiency criteria in individual physician financial incentive formulae is associated with lower performance on some measures. Health care organizations should carefully evaluate how the nature of financial incentives and organizational culture influence physician engagement in improving quality performance. Moreover, because area-level deprivation was associated with worse performance on some measures, physician organizations should be attentive to the potential unintended consequences of pay-for-performance programs (Casalino and Elster 2007; Felt-Lisk, Gimm, and Peterson 2007; Millett et al. 2007), including the refusal of physicians to accept patients with Medicaid coverage and/or from certain racial/ethnic groups. It will be important for future studies to assess the extent to which performance deficits associated with area-level deprivation (Doran et al. 2008) are modifiable through focused activities aimed at improving patients' care experiences.

Acknowledgments

Joint Acknowledgment/Disclosure Statement: This project was supported by a research grant from the Commonwealth Fund (#20070067). We would like to thank Serwar Ahmed, Rachel Brodie, and Amy Rassbach for contributions to the collection of medical director interview data. We are grateful to Jeff Burkeen and his staff at the Center for the Study of Services (CSS) for their expertise in the sampling and data collection activities that supported this work. We would like to thank David Goodman and colleagues at the Center for the Evaluative Clinical Sciences at Dartmouth for assembling and validating the market data that were central to this work.

Disclosures: An early version of the paper was presented at the Academy Health Research Meeting on June 9, 2008.

Disclaimers: None.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

hesr0044-0880-SD1.doc (258.5KB, doc)

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supportingmaterials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.

REFERENCES

  1. Agency for Healthcare Research Quality, American Institutes for Research, Harvard Medical School, and Rand Corporation. The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Clinician and Group Survey: Submission to National Quality Forum [accessed on March 1, 2009]. Available at http://www.aqaalliance.org/October24Meeting/PerformanceMeasurement/CAHPSCGtext.doc.
  2. Burton LC, Anderson GF, Kues IW. Using Electronic Health Records to Help Coordinate Care. Milbank Quarterly. 2004;82(3):457, 81. doi: 10.1111/j.0887-378X.2004.00318.x. table. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Casalino LP. Which Type of Medical Group Provides Higher-Quality Care? Annals of Internal Medicine. 2006;145(11):860–1. doi: 10.7326/0003-4819-145-11-200612050-00012. [DOI] [PubMed] [Google Scholar]
  4. Casalino LP, Elster A. Will Pay-for-Performance and Quality Reporting Affect Health Care Disparities? Health Affairs. 2007;26(3):w405–14. doi: 10.1377/hlthaff.26.3.w405. [DOI] [PubMed] [Google Scholar]
  5. Casalino L, Gillies RR, Shortell SM, Schmittdiel JA, Bodenheimer T, Robinson JC, Rundall T, Oswald N, Schauffler H, Wang MC. External Incentives, Information Technology, and Organized Processes to Improve Health Care Quality for Patients with Chronic Diseases. Journal of the American Medical Association. 2003;289(4):434–41. doi: 10.1001/jama.289.4.434. [DOI] [PubMed] [Google Scholar]
  6. Center for Health Workforce Studies. California Physician Workforce: Supply and Demand through 2015. University of California Office of Health Affairs.
  7. Cleary PD. The Increasing Importance of Patient Surveys. Now That Sound Methods Exist, Patient Surveys Can Facilitate Improvement. British Medical Journal. 1999;319(7212):720–1. doi: 10.1136/bmj.319.7212.720. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Damberg CL, Raube K, Williams T, Shortell SM. Paying for Performance: Implementing a Statewide Project in California. Quality Management in Health Care. 2005;14(2):66–79. doi: 10.1097/00019514-200504000-00002. [DOI] [PubMed] [Google Scholar]
  9. Doran T, Fullwood C, Kontopantelis E, Reeves D. Effect of Financial Incentives on Inequalities in the Delivery of Primary Clinical Care in England: Analysis of Clinical Activity Indicators for the Quality and Outcomes Framework. Lancet. 2008;372(9640):728–36. doi: 10.1016/S0140-6736(08)61123-X. [DOI] [PubMed] [Google Scholar]
  10. Felt-Lisk S, Gimm G, Peterson S. Making Pay-for-Performance Work in Medicaid. Health Affairs. 2007;26(4):w516–27. doi: 10.1377/hlthaff.26.4.w516. [DOI] [PubMed] [Google Scholar]
  11. Gillies RR, Chenok KE, Shortell SM, Pawlson G, Wimbush JJ. The Impact of Health Plan Delivery System Organization on Clinical Quality and Patient Satisfaction. Health Services Research. 2006;41(4, part 1):1181–99. doi: 10.1111/j.1475-6773.2006.00529.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Goodman DC, Mick SS, Bott D, Stukel T, Chang CH, Marth N, Poage J, Carretta HJ. Primary Care Service Areas: A New Tool for the Evaluation of Primary Care Services. Health Services Research. 2003;38(1, part 1):287–309. doi: 10.1111/1475-6773.00116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Grembowski DE, Anderson ML, Conrad DA, Fishman PA, Larson EB, Martin DP, Ralston JD, Carrell D, Hecht J. Evaluation of the Group Health Cooperative Access Initiative: Study Design Challenges in Estimating the Impact of a Large-Scale Organizational Transformation. Quality Management in Health Care. 2008;17(4):292–303. doi: 10.1097/01.QMH.0000338550.67393.a9. [DOI] [PubMed] [Google Scholar]
  14. Grumbach K, Bodenheimer T. Can Health Care Teams Improve Primary Care Practice? Journal of the American Medical Association. 2004;291(10):1246–51. doi: 10.1001/jama.291.10.1246. [DOI] [PubMed] [Google Scholar]
  15. Grumbach K, Osmond D, Vranizan K, Jaffe D, Bindman AB. Primary Care Physicians' Experience of Financial Incentives in Managed-Care Systems. New England Journal of Medicine. 1998;339(21):1516–21. doi: 10.1056/NEJM199811193392106. [DOI] [PubMed] [Google Scholar]
  16. Gusmano MK, Fairbrother G, Park H. Exploring the Limits of the Safety Net: Community Health Centers and Care for the Uninsured. Health Affairs. 2002;21(6):188–94. doi: 10.1377/hlthaff.21.6.188. [DOI] [PubMed] [Google Scholar]
  17. Helfrich CD, Li YF, Mohr DC, Meterko M, Sales AE. Assessing an Organizational Culture Instrument Based on the Competing Values Framework: Exploratory and Confirmatory Factor Analyses. Implementation Science. 2007;2:13. doi: 10.1186/1748-5908-2-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kroenke K. Patient Expectations for Care: How Hidden Is the Agenda? Mayo Clinic Proceedings. 1998;73(2):191–3. doi: 10.1016/S0025-6196(11)63654-9. [DOI] [PubMed] [Google Scholar]
  19. Landon BE, Normand SL. Performance Measurement in the Small Office Practice: Challenges and Potential Solutions. Annals of Internal Medicine. 2008;148(5):353–7. doi: 10.7326/0003-4819-148-5-200803040-00006. [DOI] [PubMed] [Google Scholar]
  20. Landon BE, Normand SL, Meara E, Qi Z, Simon SR, Frank R, McNeil BJ. The Relationship between Medical Practice Characteristics and Quality of Care for Cardiovascular Disease. Medical Care Research and Review. 2008;65(2):167–86. doi: 10.1177/1077558707310208. [DOI] [PubMed] [Google Scholar]
  21. Marsden PV, Landon BE, Wilson IB, McInnes K, Hirschhorn LR, Ding L, Cleary PD. The Reliability of Survey Assessments of Characteristics of Medical Clinics. Health Services Research. 2006;41(1):265–83. doi: 10.1111/j.1475-6773.2005.00480.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Marvel MK, Epstein RM, Flowers K, Beckman HB. Soliciting the Patient's Agenda: Have We Improved? Journal of the American Medical Association. 1999;281(3):283–7. doi: 10.1001/jama.281.3.283. [DOI] [PubMed] [Google Scholar]
  23. Mehrotra A, Epstein AM, Rosenthal MB. Do Integrated Medical Groups Provide Higher-Quality Medical Care Than Individual Practice Associations? Annals of Internal Medicine. 2006;145(11):826–33. doi: 10.7326/0003-4819-145-11-200612050-00007. [DOI] [PubMed] [Google Scholar]
  24. Meterko M, Young GJ, White B, Bokhour BG, Burgess JF, Jr., Berlowitz D, Guldin MR, Nealon Seibert M. Provider Attitudes toward Pay-for-Performance Programs: Development and Validation of a Measurement Instrument. Health Services Research. 2006;41(5):1959–78. doi: 10.1111/j.1475-6773.2006.00582.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Millett C, Gray J, Saxena S, Netuveli G, Khunti K, Majeed A. Ethnic Disparities in Diabetes Management and Pay-for-Performance in the UK: The Wandsworth Prospective Diabetes Study. PLoS Medicine. 2007;4(6):e191. doi: 10.1371/journal.pmed.0040191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nunnelly J, Bernstein I. Psychometric Theory. New York: McGraw-Hill; 1994. [Google Scholar]
  27. Pauly MV, Pagan JA. Spillovers and Vulnerability: The Case of Community Uninsurance. Health Affairs. 2007;26(5):1304–14. doi: 10.1377/hlthaff.26.5.1304. [DOI] [PubMed] [Google Scholar]
  28. Peltenburg M, Fischer JE, Bahrs O, van Dulmen S, van den Brink-Muinen A. The Unexpected in Primary Care: A Multicenter Study on the Emergence of Unvoiced Patient Agenda. Annals of Family Medicine. 2004;2(6):534–40. doi: 10.1370/afm.241. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Pham HH, Schrag D, Hargraves JL, Bach PB. Delivery of Preventive Services to Older Adults by Primary Care Physicians. Journal of the American Medical Association. 2005;294(4):473–81. doi: 10.1001/jama.294.4.473. [DOI] [PubMed] [Google Scholar]
  30. Rittenhouse DR, Grumbach K, O'Neil EH, Dower C, Bindman A. Physician Organization and Care Management in California: From Cottage to Kaiser. Health Affairs. 2004;23(6):51–62. doi: 10.1377/hlthaff.23.6.51. [DOI] [PubMed] [Google Scholar]
  31. Robinson JC. Physician Organization in California: Crisis and Opportunity. Health Affairs. 2001;20(4):81–96. doi: 10.1377/hlthaff.20.4.81. [DOI] [PubMed] [Google Scholar]
  32. Robinson JC, Casalino LP. Vertical Integration and Organizational Networks in Health Care. Health Affairs. 1996;15(1):7–22. doi: 10.1377/hlthaff.15.1.7. [DOI] [PubMed] [Google Scholar]
  33. Rodriguez HP, Anastario MP, Frankel RM, Odigie EG, Rogers WH, von Glahn T, Safran DG. Can Teaching Agenda-Setting Skills to Physicians Improve Clinical Interaction Quality? A Controlled Intervention. BMC Medical Education. 2008;8:3. doi: 10.1186/1472-6920-8-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Rodriguez HP, von Glahn T, Chang H, Rogers WH, Safran DG. Patient Samples for Measuring Primary Care Physician Performance: Who Should Be Included? Medical Care. 2007;45(10):989–96. doi: 10.1097/MLR.0b013e318074ce63. [DOI] [PubMed] [Google Scholar]
  35. Rodriguez HP, von Glahn T, Rogers WH, Chang H, Fanjiang G, Safran DG. Evaluating Patients' Experiences with Individual Physicians: A Randomized Trial of Mail, Internet, and Interactive Voice Response Telephone Administration of Surveys. Medical Care. 2006;44(2):167–74. doi: 10.1097/01.mlr.0000196961.00933.8e. [DOI] [PubMed] [Google Scholar]
  36. Rogers EM. Diffusion of Innovations. New York: Free Press; 1995. [Google Scholar]
  37. Rye CB, Kimberly JR. The Adoption of Innovations by Provider Organizations in Health Care. Medical Care Research and Review. 2007;64(3):235–78. doi: 10.1177/1077558707299865. [DOI] [PubMed] [Google Scholar]
  38. Safran DG, Karp M, Coltin K, Chang H, Li A, Ogren J, Rogers WH. Measuring Patients' Experiences with Individual Primary Care Physicians. Results of a Statewide Demonstration Project. Journal of General Internal Medicine. 2006a;21(1):13–21. doi: 10.1111/j.1525-1497.2005.00311.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Safran DG, Miller W, Beckman H. Organizational Dimensions of Relationship-Centered Care. Theory, Evidence, and Practice. Journal of General Internal Medicine. 2006b;21(suppl 1):S9–15. doi: 10.1111/j.1525-1497.2006.00303.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Safran DG, Rogers WH, Tarlov AR, Inui T, Taira DA, Montgomery JE, Ware JE, Slavin CP. Organizational and Financial Characteristics of Health Plans: Are They Related to Primary Care Performance? Archives of Internal Medicine. 2000;160(1):69–76. doi: 10.1001/archinte.160.1.69. [DOI] [PubMed] [Google Scholar]
  41. Safran DG, Tarlov AR, Rogers WH. Primary Care Performances in Fee-for-Service and Prepaid Health Care Systems. Results from the Medical Outcomes Study. Journal of the American Medical Association. 1994;271(20):1579–86. [PubMed] [Google Scholar]
  42. Safran DG, Wilson IB, Rogers WH, Montgomery JE, Chang H. Primary Care Quality in the Medicare Program: Comparing the Performance of Medicare Health Maintenance Organizations and Traditional Fee-for-Service Medicare. Archives of Internal Medicine. 2002;162(7):757–65. doi: 10.1001/archinte.162.7.757. [DOI] [PubMed] [Google Scholar]
  43. Simon JS, Rundall TG, Shortell SM. Drivers of Electronic Medical Record Adoption among Medical Groups. The Joint Commission Journal on Quality and Patient Safety. 2005;31(11):631–9. doi: 10.1016/s1553-7250(05)31081-6. [DOI] [PubMed] [Google Scholar]
  44. Simon SR, Kaushal R, Cleary PD, Jenter CA, Volk LA, Poon EG, Orav EJ, Lo HG, Williams DH, Bates DW. Correlates of Electronic Health Record Adoption in Office Practices: A Statewide Survey. Journal of the American Medical Information Association. 2007;14(1):110–7. doi: 10.1197/jamia.M2187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Snyder L, Neubauer RL. Pay-for-Performance Principles that Promote Patient-Centered Care: An Ethics Manifesto. Annals of Internal Medicine. 2007;147(11):792–4. doi: 10.7326/0003-4819-147-11-200712040-00011. [DOI] [PubMed] [Google Scholar]
  46. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in Patient-Reported Quality among Health Care Organizations. Health Care Financing Review. 2002;23(4):85–100. [PMC free article] [PubMed] [Google Scholar]
  47. Stein T, Frankel RM, Krupat E. Enhancing Clinician Communication Skills in a Large Healthcare Organization: A Longitudinal Case Study. Patient Education and Counseling. 2005;58(1):4–12. doi: 10.1016/j.pec.2005.01.014. [DOI] [PubMed] [Google Scholar]
  48. Wilson IB, Landon BE, Ding L, Zaslavsky AM, Shapiro MF, Bozzette SA, Cleary PD. A National Study of the Relationship of Care Site HIV Specialization to Early Adoption of Highly Active Antiretroviral Therapy. Medical Care. 2005;43(1):12–20. [PubMed] [Google Scholar]
  49. Yano EM, Soban LM, Parkerton PH, Etzioni DA. Primary Care Practice Organization Influences Colorectal Cancer Screening Performance. Health Services Research. 2007;42(3, part 1):1130–49. doi: 10.1111/j.1475-6773.2006.00643.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zaslavsky AM, Zaborski L, Cleary PD. Does the Effect of Respondent Characteristics on Consumer Assessments Vary across Health Plans? Medical Care Research and Review. 2000;57(3):379–94. doi: 10.1177/107755870005700307. [DOI] [PubMed] [Google Scholar]
  51. Zaslavsky AM, Zaborski LB, Ding L, Shaul JA, Cioffi MJ, Cleary PD. Adjusting Performance Measures to Ensure Equitable Plan Comparisons. Health Care Financing Review. 2001;22(3):109–26. [PMC free article] [PubMed] [Google Scholar]
  52. Zazzali JL, Alexander JA, Shortell SM, Burns LR. Organizational Culture and Physician Satisfaction with Dimensions of Group Practice. Health Services Research. 2007;42(3, part 1):1150–76. doi: 10.1111/j.1475-6773.2006.00648.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

hesr0044-0880-SD1.doc (258.5KB, doc)

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust

RESOURCES