Are Patients’ Ratings of Their Physicians Related to Health Outcomes?

Abstract

PURPOSE Observational studies using patient reports suggest associations between physician interpersonal styles and patient outcomes. Possible confounding of these associations has not been carefully examined.

METHODS Approximately 4,700 patients of 96 physicians completed a survey instrument that included reported health status changes during the previous year, perceptions of their physician (satisfaction, trust, knowledge of patient, and autonomy support), and sociodemographic and clinical covariates. We examined the adjusted relationship between patient perceptions of their physicians and reported health status changes. Using multilevel analyses, we then explored differences among physicians in patient perceptions of their physicians and whether these differences were explained by the relationship between patient perceptions and reported health status changes.

RESULTS There were significant adjusted relationships between patient perceptions of their physician and reported health status changes: better perceptions were associated with a smaller risk of health status decline (adjusted odds ratio = 1.14; 95% confidence interval [CI], 1.05–1.24; P <.01). Multilevel analysis showed significant differences between physicians in patient perceptions of their physicians (ρ = 0.10; 95% CI, 0.07–0.13; P <.01), but these physician differences were unrelated to reported health status decline (ρ = 0; P >.99).

CONCLUSIONS Using methods similar to those of previous studies, we found a relationship between patient perceptions of their physicians and reported health status declines. Multilevel analysis, however, suggested that this relationship is not a physician effect; it may reflect unmeasured patient confounding. Multilevel analyses may help to examine the relationships between physician styles and outcomes.

INTRODUCTION

Observational studies examining relationships between patients’ perceptions of their physicians and patient-reported adherence, health status changes, and symptom resolution suggest that physicians’ interpersonal styles may influence patient outcomes.^1–9 Among these physician styles, greater patient-centeredness (manifested as engaging in a more participatory style, obtaining agreement on treatment, or supporting patient autonomy) has been associated with greater improvements in back pain, headache resolution, diabetes control, health status, compliance, and satisfaction. Unmeasured patient factors that might alter the reporting of physician style could confound interpretation of these studies, however. For example, certain patient personality characteristics have been associated with better perceived outcomes.^10–13 Because these characteristics might also be associated with more positive assessments of physicians, there may be a spurious association between physician styles and outcomes.

An alternative statistical approach to examining the relationship between physician styles and patient outcomes involves using multilevel modeling to test whether patient perceptions mediate the apparent physician effect on patient outcome. Multilevel modeling allows direct assessment of any physician contribution to patient outcome by analyzing how the physician variance component contributes to the total variance observed. One can then examine the extent to which physician effect (or variance component) is mediated by the patients’ ratings of the physician. A significant contribution of the physician variance component to the total variance associated with patient outcome, which is also mediated by the patients’ ratings of the physician, is a necessary condition for evidence of a relationship between physician style and patient outcome.

In this study, we studied the relationship between patients’ ratings of their physicians and reported changes in health status. Although controversial,^14–16 retrospectively reported health status change is considered a sensitive and valid measure of health outcomes.^17–19 We examined whether physician interpersonal style was associated with health status change, hypothesizing that a better perceived style would be associated with a smaller risk of health status decline. Mediation was assessed by first analyzing the physician’s contribution to the total variance while excluding patients’ perceptions. The analysis was then repeated including patients’ perceptions. The amount of observed reduction in the physician variance component is a measure of the extent of mediation resulting from patients’ perceptions.

METHODS

Physician Sample

The data were derived from a larger study conducted in the Rochester, NY, area in 2001 to 2002. More than 95% of local physicians participate in a managed care organization (MCO) that insures more than 50% of the population. To ensure adequate data for each physician, primary care physicians, internists, and family physicians were considered eligible if they had more than 100 patients in the MCO.²⁰ We recruited 100 of 506 eligible primary care physicians into the study. Details about the patients, their primary care physicians, and recruitment are presented elsewhere.²¹

Patient Survey Data

Approximately 50 consecutive patients aged 18 to 65 years were approached by a research assistant in the waiting room of each study physician to complete a 10-minute survey instrument before their office visit. The survey yielded the dependent variable (health status change), the key independent variables (patient perceptions of care), and covariates. Patients compared their current health status with their health status 1 year earlier using a 5-point scale: much worse, worse, about the same, better, or much better.²²

The survey instrument comprised the 5-item Health Care Climate Questionnaire,²³ which measures autonomy supportiveness and patient involvement in decision making; 2 subscales from the Primary Care Assessment Survey (the 4-item knowledge subscale, which measures physician knowledge of the patient, and the 8-item trust subscale²⁴); and a single question about satisfaction, a 6-option Likert scale (“All things considered, how satisfied are you with your regular doctor?”) derived from the Patient Satisfaction Questionnaire.²⁵

Covariates in the survey instrument included demographics, the duration of the physician-patient relationship, functional status, morbidity, and somatization. Demographic information included age, sex, educational level, race, and ethnicity. Functional status was measured with the Medical Outcomes Study Short Form-12 physical and mental component summary scores.²⁶ We assessed patient morbidities by asking whether they were receiving medication for any of 16 chronic illnesses (arthritis, asthma, cancer [other than skin], cardiovascular disease, cerebrovascular disease, chronic obstructive pulmonary disease, cirrhosis, colitis, diabetes, heart failure, human immunodeficiency virus infection or acquired immunodeficiency syndrome, hypertension, kidney disease, peptic ulcer, or thyroid problems) or 2 mental illnesses (depression or anxiety) or had visual, hearing, or functional impairments. Somatization was measured by using the Symptom Checklist 90 somatization subscale.²⁷

Analyses

High correlations (all exceeding 0.63) among the 4 patient perception scales (Health Care Climate Questionnaire, Primary Care Assessment Survey knowledge subscale, Primary Care Assessment Survey trust sub-scale, and satisfaction) and concern about type I errors arising from the use of 4 patient perception measures led us to use factor analysis to reduce the number of variables.²⁸ Principal component factor analysis, with physician effects partialed out by using dummy variables for each physician applied to the 4 scales, produced a single factor that accounted for 75% of the variance in the individual scales; no other factors emerged. The individual scales all loaded highly and similarly (0.83–0.86) on the factor. The Cronbach α for a single scale, with the 4 scales as items on that scale, was 0.88. Taken together, these findings suggest a single underlying factor, which we termed the satisfaction/trust/autonomy/knowledge (STAK) score, that accounts for much of the variation in the observed responses. Subsequent reported analyses used this factor, although analyses using the individual scales produced similar results. STAK scores were standardized to a mean of 0 and a standard deviation of 1.

Multilevel analyses allowed nesting of patient observations within physician (included as a random effect).²⁹ The primary analyses were random effects logistic regression models. The dichotomous dependent variable was whether the patient reported a decline in health status. The key independent variable was the STAK score. Secondary analyses used random effects linear models that treated the dependent variable—reported health status change—as a continuous variable with 5 levels. Although the dependent variable is not properly a continuous variable, this analytic approach has more statistical power and reduces the likelihood of missing a small relationship.

The analyses adjusted for the following covariates: patient age, sex, race, ethnicity, and education level; Medical Outcomes Study Short Form-12 physical and mental component summary scores; the Symptom Checklist 90 somatization score; a dummy variable for each condition or disability; the duration of the patient-physician relationship; and physician specialty.

We examined whether STAK had a relationship to health status change after covariate adjustment. We examined the contribution of the physician variance component to the total variance (ρ, or the intraclass correlation coefficient) with the variable STAK omitted as evidence of a physician effect on health status change. We examined the percentage of reduction in the physician variance component with STAK included, as evidence of its mediation of the physician effect.²⁹

We conducted a random effects linear regression analysis with STAK as the dependent variable and the patient survey covariates as independent variables. This analysis examined whether STAK itself showed evidence of a significant physician variance component.

RESULTS

Analysis of MCO claims data showed that the sociodemographic, utilization, and clinical characteristics of MCO patients of enrolled physicians and physicians not enrolled were similar (Table 1 ▶). Survey patients (Table 2 ▶) were predominantly female and white, had some college education, and had at least a 5-year relationship with their primary care physician. A decline in health status during the previous year was reported by 1,051 (22.2%) patients.

Table 1.

Characteristics of Patients and Physicians in Study and Comparison Practices

Characteristics	Enrolled Physicians	Not Enrolled Physicians
Patients
Number	121,806	483,094
Age, y, mean (SD)	41.0 (11.0)	41.1 (11.2)
Sex, % female	53.9	52.7
Income, US$, median (SD)*	37,830 (10,683)	36,874 (10,160)
High school graduate, %, mean (SD)*	64.8 (7.8)	63.8 (7.9)
Any visit to a physician, %	83.1	82.5
Percentage of patients referred	25.7	25.6
Years enrolled in the MCO, mean (SD)	3.07 (1.12)	3.07 (1.12)
Annual adherence indicators, %
Women with Papanicolaou tests	52.5	51.8
Women >40 y with mammograms	46.2	46.2
Diabetic with eye examinations	47.0	46.4
Diabetic with glycohemoglobin tests	67.0	66.1
Diabetic with cholesterol tests	54.7	54.5
Per-patient costs in 1996 dollars; mean, median (SD)
Diagnostic testing	189,19.6 (521)	196, 23.6 (513)
Inpatient	254, 0 (4,702)	264, 0 (4,038)
Total costs	938, 228 (5,493)	950, 228 (4,914)
Physicians
Number	100	594
Specialty, % family practice	47	24
Patients in panel, No. (SD)	1,218 (758)	813 (776)

MCO = managed care organization.

* Socioeconomic variables derived from patient Zip code linked to 1990 US census data. Information is based on 1996 to 1999 claims data.

Table 2.

Characteristics of Patients Responding to Survey Instrument

Characteristic	Patients Responding No. (%)	Mean (SD)
Total	4,746 (100)
Sex
Missing	41 (0.9)
Female	2,955 (62.3)
Male	1,750 (36.9)
Patient race/ethnicity
Missing	34 (0.7)
African American	499 (10.5)
Hispanic	109 (2.3)
Other	110 (2.3)
White	3,994 (84.2)
Patient education
Missing	21 (0.4)
<12 y	337 (7.1)
12th grade	1,370 (28.9)
1–3 y of college	1,490 (31.4)
4 y of college	828 (17.4)
Graduate school	700 (14.7)
Length of patient-doctor relationship
Missing	12 (0.3)
<1 y	360 (7.6)
1–3 y	1,035 (21.8)
3–5 y	814 (17.2)
>5 y	2,525 (53.2)
Age, y	4,680 (98.6)	44.9 (12.1)
Number of conditions	4,746 (100)	2.3 (1.5)
MCS-12 score	4,458 (93.9)	48.8 (10.5)
PCS-12 score	4,458 (93.9)	46.0 (11.1)
Somatization scale	4,724 (99.5)	7.6 (6.5)
HCCQ scpre	4,733 (99.7)	18.0 (3.1)
PCAS—knowledge	4,734 (99.7)	15.1 (4.3)
PCAS—trust	4,744 (100)	34.1 (4.0)
Patient satisfaction scale	4,729 (99.6)	5.2 (0.8)
Health status change	4,720 (99.5)	0.0 (0.92)

Note: Survey data were available for patients of only 96 physicians.

MCS-12 = Medical Outcomes Study Short Form-12 mental component summary score; PCS-12 = Medical Outcomes Study Short Form-12 physical component summary score; HCCQ = Health Care Climate Questionnaire; PCAS = Primary Care Assessment Survey.

There was a significant adjusted association between the composite measure of patients’ ratings of their physicians (STAK) and the risk of health status decline (adjusted odds ratio = 1.14; 95% confidence interval [CI], 1.05–1.24; P <.01). A 1-SD increase in the mean STAK score was associated with a 14% decreased risk of health status decline.

Multilevel analysis showed no evidence of a physician effect on reported health status decline. ρ, the proportion of total variance attributable to a physician effect, was 0.00 (95% CI, 0.00-0.00; P >.99). Because of concern about overadjustment, we repeated the analyses and excluded the Medical Outcomes Study Short Form-12 variables, with similar results.

We also repeated the health status change analysis, treating that variable as an interval-level 5-point scale in a random effects linear model. When we examined whether the absence of a significant effect in the dichotomous decline/no decline variable reflected insufficient statistical power, ρ was also 0.00 (95% CI, 0.00–0.05; P = .3). One standard deviation of the physician component or effect (4 SDs encompass approximately 95% of the total predicted physicians’ effect) was associated with a 0.03 step (95% CI, -0.03 to 0.10; P = .3) on the 5-step health status change scale. Note that 1 SD of the physician effect derives from the square root of the physician variance component.

We examined the physician variance component with STAK as a dependent variable. ρ was 0.10 (95% CI, 0.07–0.13; P <.01). One standard deviation of the physician effect was associated with a 0.30-SD change in STAK (95% CI, 0.25–0.35; P <.01).

DISCUSSION

Using methods similar to those of previous studies, we replicated previous research showing that patient ratings of their physicians are associated with patient-reported health outcomes.^1–9 With multilevel analysis, however, we found little evidence of a physician contribution to the risk of reported health status change. The absence of a significant physician effect (variance component) suggests there is little substantive association between perceived physician interpersonal style and patient outcome; the apparent relationship may reflect unmeasured patient confounding.

These findings have implications for understanding the relationships between physician behavior and patient outcomes. Previous research has linked patient-reported assessments of their care to patient-reported outcomes, including health status change, adherence, and satisfaction.^1–9 These studies have been used to conclude that a “better” physician style is related to improved outcomes. When patients provide both the physician assessment and outcome measures, however, caution is necessary, because the findings may be ascribed simply to shared method variance: patients who provide higher ratings of their physicians may also report better health. Our results suggest that although patient perceptions of physicians are related to reported health status changes, these effects may not be related to physician interpersonal styles.

Although we did not measure patient psychological factors, such factors may affect both patients’ ratings of their physicians and their reported health status changes. Patients who express negative affect also tend to report worse health and health care.^10–13 Self-efficacy may also influence adherence, health status, and physician ratings.^30,31 The potential confounding might also extend to biomedical outcomes, because these psychological factors have also been associated with glycohemoglobin,³² immune function,³³ and mortality.³⁴ Future studies in this field should adjust for these and other psychological characteristics.

We observed a significant physician effect (variance component) associated with STAK. This finding replicates earlier research suggesting that patient satisfaction is significantly clustered by physician.^20,35 This finding may reflect a physician effect on patient satisfaction, and this effect is possibly related to physician behaviors unconnected to reported health status changes. It is also plausible, however, that patients with differing levels of negative affect (or other characteristics) may be attracted to different physicians. Evidence also suggests that patients who are more optimistic and have high levels of self-efficacy are also more satisfied with their doctor-patient relationships³²; conversely, patients with more negative stereotypes of physicians also report lower satisfaction and adherence.³⁶ Careful research is necessary to tease out these alternatives, especially given the increasing use of patient satisfaction surveys to evaluate physicians.

Alternative approaches to assessing physician style may more robustly address some of the problems discussed here. Randomized trials to improve physician communication are less susceptible to confounding and reporting biases but are difficult to implement. Some, but not all, of these studies have shown only changes in communication and modest effects on patient satisfaction.^5,37,38 Studies using objective ratings of clinical encounters circumvent the biases described here, but they may be subject to other limitations; studies typically have linked physician behaviors with outcomes only for patients observed in the same encounters,³⁹ thus making it difficult to discern whether the observations are due to the effects of patients on physicians or vice versa. Observations on physician encounters with standardized patients may circumvent this problem.⁴⁰ All these approaches may be necessary to tease out the elements of physician communication style that contribute to optimal patient outcomes.

This study has a number of limitations. The cross-sectional design used retrospective assessments of health state changes (or transition states). As noted in the introduction, a modest literature, although controversial,^14–16 suggests that reported transition states are sensitive and valid measures of health outcomes.^17–19 Prospective assessments of changes in health status with time (before-and-after scores) are not without their own measurement problems.¹⁶ Thus, it is not clear that serial measurements of health status are the best and most bias-free way to address this question. It seems unlikely that a bias in our study would explain the essential findings: an association was observed with simple regression analyses similar to those used in previous studies, but that association disappeared on multilevel analysis.

There are several other notable study limitations. Although the claims data of study and comparison physicians were similar, physicians who agreed to participate in this study probably represent a selected, narrow spectrum of physician styles. The absence of observed physician effects remains consistent with a small effect that would require a larger sample size per physician to detect.^20,35 The narrow confidence intervals around the negative effect observed in this study, however, suggest that the power was adequate. We found, furthermore, a relationship between patients’ ratings of their physicians and the outcome to be comparable to effects reported in previous studies when we used methods similar to those in other studies.

Our study is the first to examine directly evidence for a physician contribution to the total variance observed. Our approach assumes a physician interpersonal style that is sufficiently consistent across patients to be measurable. This assumption also lies behind many published studies in the field, physician performance assessment, and much teaching about interpersonal communication. It is possible that interpersonal communication is entirely specific to particular patient-physician interactions, but, on average, physicians—at least those in this study—do not differ.

We conclude by quoting Francis Bacon: “For what a man had rather were true he more readily believes.” These results represent a cautionary tale: they suggest that apparent physician effects on care can be difficult to interpret. The uncritical assumption that all elements of a patient-centered physician interaction style (or, perhaps, styles that score well on measures of patient-centeredness) are always good for patients may impede the discovery and development of optimal ways of being with patients.⁴¹ Some studies suggesting the benefits of patient-centered care may reflect, in part, unmeasured patient confounding. Alternative study methods and analytic approaches, including multilevel analyses, will be necessary to tease out the components of physician behaviors that contribute to better patient outcomes.