Skip to main content
The Journal of Bone and Joint Surgery. American Volume logoLink to The Journal of Bone and Joint Surgery. American Volume
. 2015 Jul 1;97(13):1041–1048. doi: 10.2106/JBJS.N.00950

Association Between Orthopaedic Outpatient Satisfaction and Non-Modifiable Patient Factors

Amir M Abtahi 1, Angela P Presson 2, Chong Zhang 2, Charles L Saltzman 1, Andrew R Tyser 1
PMCID: PMC4574907  PMID: 26135070

Abstract

Background:

Increasingly, patient satisfaction surveys are being utilized to evaluate hospital and physician performance. Despite this, little is known about factors associated with patient satisfaction. The objective of this study was to determine whether selected non-modifiable patient characteristics are associated with outpatient satisfaction scores.

Methods:

We reviewed patient satisfaction scores from 12,177 outpatient clinical encounters at an academic orthopaedic outpatient clinic between December 2010 and October 2013. Any adult patient who completed at least one patient satisfaction survey at any point during the study period was included in this study. Factors including age, sex, employment status, type of health insurance, zip code, and orthopaedic subspecialty were recorded. Patients were divided into more satisfied and less satisfied groups, and generalized estimating equation logistic regression analysis was performed to identify factors predictive of lower patient satisfaction.

Results:

Age was found to be strongly associated with patient satisfaction, with younger patients reporting less satisfaction; the adjusted odds ratio for the patient age of eighteen to twenty-nine years compared with the patient age of eighty years or older was 2.78 (95% confidence interval, 1.74 to 3.82) (p < 0.001). This relationship was maintained in a predictive model across all age groups, both sexes, all travel distances, and all orthopaedic subspecialties. Travel distance was also associated with patient satisfaction, with patients who live closer reporting less satisfaction compared with patients who live farther away; the adjusted odds ratio for a distance of less than fifty miles compared with a distance of fifty miles or more was 1.18 (95% confidence interval, 1.03 to 1.33) (p = 0.016).

Conclusions:

The finding that non-modifiable patient factors such as age and geographic location affect patient satisfaction challenges the utility of comparing patient satisfaction between populations that differ significantly with regard to such characteristics.

Clinical Relevance:

A more complete knowledge of the factors that influence patient satisfaction may enable better insight into the interpretation of current patient satisfaction metrics and may allow physicians and hospitals to improve their delivery of care.


Patient satisfaction is becoming an increasingly important aspect of the delivery of medical care in the United States. Under the Hospital Value-Based Purchasing Program, established by the Patient Protection and Affordable Care Act of 2010 (P.L. 111-148), Medicare makes incentive payments to hospitals based on specific quality domains that include the patient experience of care (Hospital Consumer Assessment of Healthcare Providers and Systems [HCAHPS])1,2. Although the reporting of outpatient patient satisfaction scores is not currently mandated, outpatient satisfaction metrics are increasingly used by health-care organizations to evaluate physician performance and have been used to determine provider compensation3-5.

Despite the increased emphasis on these metrics, knowledge regarding the factors that influence patient satisfaction is incomplete. Hekkert et al. studied the influence of hospital, department, and patient characteristics on patient satisfaction and found that patient characteristics explained more of the variation in patient satisfaction compared with other factors6. Investigations in predominantly non-orthopaedic populations have demonstrated that older age6-17, male sex8,18, white race10,11, lower educational level6,9,12, health insurance coverage15, and employment19 are associated with higher satisfaction. A more complete understanding of the factors that influence patient satisfaction may enable better insight into the interpretation of current patient satisfaction metrics and may allow physicians and hospitals to improve their delivery of care. Our aim in conducting this study was to determine whether various non-modifiable patient characteristics have an influence on orthopaedic outpatient satisfaction scores.

Materials and Methods

This study was reviewed and was approved by our institutional review board. We retrospectively reviewed patient satisfaction scores from 12,177 clinical encounters (7258 patients) at a single academic orthopaedic outpatient clinic between December 2010 and October 2013. All patients with a valid e-mail address on file were sent a link to the Press Ganey Medical Practice Survey following their clinic visit20. Any patient who was eighteen years of age or older and completed at least one patient satisfaction survey during the study period was included in this study. Survey data were linked to the patient medical record, and patient demographic variables including age, sex, employment status, type of insurance provider, and zip code were recorded20. The zip code was used to approximate each patient’s travel distance to the clinic (i.e., the shortest distance between the patient’s zip code and the zip code of the clinic). Questionnaire responses were linked to the orthopaedic subspecialty provider associated with each encounter.

The Press Ganey Medical Practice Survey consists of twenty-four questions divided into six subdomains: access, moving through your visit, nurse or assistant, care provider, personal issues, and overall assessment. Each question measures responses on a Likert scale ranging from 1 (indicating very poor) to 5 (indicating very good). Responses are converted to a 0 to 100-point scale, and the mean overall score is calculated from the mean scores for the six individual subdomains21.

Patients were categorized as less satisfied if their mean overall score was in the ≤33rd percentile. Patients with scores in the >33rd percentile were categorized as more satisfied. This threshold was chosen a priori and corresponds to an overall mean score of 86.3 points (Fig. 1). To characterize this threshold, we tabulated the percentage of questions answered with a score of <3 points (low satisfaction) and found that only 5.63% of surveys in the more satisfied group had at least one question with a score of <3 points, compared with 49.6% in the less satisfied group. We also evaluated the 25th and 50th percentile thresholds and found that our results were highly conserved across different thresholds (data not shown).

Fig. 1.

Fig. 1

Frequency histogram of patient satisfaction scores.

We evaluated each potential predictor of less satisfaction with use of generalized estimating equation logistic regression with an exchangeable correlation matrix to account for multiple responses from the same person. Results included odds ratios, 95% confidence intervals (95% CIs), and p values from univariate and multivariate models. The multivariate models controlled for age, sex, and their interactions with each predictor. The adjusted odds ratios corresponded to the main or marginal effect of each predictor, which was calculated by converting averaged adjusted predictions to odds ratios (see Appendix). Age was typically modeled as a natural cubic spline with one knot at the median (55.6 years), to allow for a nonlinear relationship with satisfaction.

Model averaging was used to construct a predictive model for patient satisfaction22. To simplify the procedure, all predictors and two-way interactions among significant predictors of patient dissatisfaction in univariate models were explored; significant predictors in univariate models included age, sex, subspecialty, and distance from the clinic. For further clarification, see the Appendix. Results were averaged across the top models to create a final predictive model. Prediction accuracy was estimated with use of tenfold cross-validation of the area under the receiver operating characteristic curve (AUC). Using this model, the predicted probability of less satisfaction and the interquartile range were calculated for different values of age, sex, subspecialty, and travel distance23. Significance was assessed at p < 0.05 and all tests were two-tailed.

Source of Funding

This investigation was supported by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant 5UL1TR001067-2. Funds were used to pay for statistical support at the University of Utah Study Design and Biostatistics Center.

Results

During the study period, 12,177 patient satisfaction surveys from 7258 patients were returned for outpatient clinical encounters (Table I). Among the patients included in the study, 2474 (34.1%) completed more than one survey.

TABLE I.

Demographic Information Summarized Across Patient Visits

Variable Summary*
Age
 Eighteen to twenty-nine years 1165 (10%)
 Thirty to thirty-nine years 1489 (12%)
 Forty to forty-nine years 1716 (14%)
 Fifty to fifty-nine years 3265 (27%)
 Sixty to sixty-nine years 3034 (25%)
 Seventy to seventy-nine years 1247 (10%)
 Eighty years or more 261 (2%)
Sex
 Male 5028 (41%)
 Female 7149 (59%)
Distance
 Fifty miles or more 1944 (16%)
 Less than fifty miles 10,229 (84%)
Employment§
 Employed 4725 (39%)
 Unemployed 1428 (12%)
 Retired 2842 (24%)
 Other 3089 (26%)
Insurance#
 Private 8524 (72%)
 Medicare 2463 (21%)
 Other 893 (8%)
Subspecialty
 Adult reconstruction 1115 (9%)
 Foot and ankle 1991 (16%)
 Hand 1555 (13%)
 Nonoperative** 3699 (30%)
 Spine 877 (7%)
 Sports 2345 (19%)
 Trauma 595 (5%)
*

The values are given as the number of visits, with the percentage in parentheses.

The percentages were based on 12,177 visits.

There were missing data for four visits, so the percentages were based on 12,173 visits.

§

There were missing data for ninety-three visits, so the percentages were based on 12,084 visits.

#

There were missing data for 297 visits, so the percentages were based on 11,880 visits.

**

This group includes psychiatry and primary care sports medicine.

Age was found to be strongly associated with patient satisfaction, with younger patients reporting less satisfaction; the adjusted odds ratio for the patient age of eighteen to twenty-nine years compared with the patient age of eighty years or older was 2.78 (95% CI, 1.74 to 3.82) (p < 0.001) (Table II). Travel distance was also associated with patient satisfaction, with patients residing less than fifty miles from the clinic reporting less satisfaction compared with patients living farther away; the adjusted odds ratio for a distance of less than fifty miles compared with a distance of fifty miles or more was 1.18 (96% CI, 1.03 to 1.33) (p = 0.016) (Table II). No significant differences in patient satisfaction were identified among orthopaedic subspecialties (Table II).

TABLE II.

Odds Ratios from Generalized Estimating Equation Logistic Regression Models

Variable Unadjusted Odds Ratio* Adjusted Odds Ratio* Adjusted P Value
Age
 Eighty years or more (reference)
 Eighteen to twenty-nine years 2.83 (1.93 to 4.15) 2.78 (1.74 to 3.82) <0.001
 Thirty to thirty-nine years 2.94 (2.02 to 4.29) 2.91 (1.84 to 3.99) <0.001
 Forty to forty-nine years 2.30 (1.58 to 3.35) 2.29 (1.44 to 3.13) 0.003
 Fifty to fifty-nine years 1.96 (1.35 to 2.83) 1.94 (1.24 to 2.64) 0.009
 Sixty to sixty-nine years 1.42 (0.98 to 2.06) 1.41 (0.89 to 1.92) 0.120
 Seventy to seventy-nine years 1.00 (0.67 to 1.49) 0.99 (0.61 to 1.38) 0.977
Sex
 Male (reference)
 Female 1.11 (1.01 to 1.22) 1.09 (0.99 to 1.18) 0.084
Insurance
 Private (reference)
 Medicare 0.58 (0.51 to 0.65) 0.94 (0.70 to 1.17) 0.594
 Other 1.09 (0.93 to 1.28) 1.05 (0.86 to 1.25) 0.602
Employment
 Employed (reference)
 Unemployed 0.92 (0.79 to 1.06) 0.98 (0.81 to 1.16) 0.853
 Retired 0.57 (0.50 to 0.65) 1.44 (0.50 to 2.39) 0.357
 Other 1.00 (0.89 to 1.11) 1.00 (0.87 to 1.13) 0.966
Subspecialty
 Adult reconstruction (reference)
 Foot and ankle 1.07 (0.90 to 1.27) 0.87 (0.71 to 1.03) 0.115
 Hand 1.28 (1.08 to 1.52) 1.01 (0.82 to 1.21) 0.883
 Nonoperative 1.06 (0.91 to 1.23) 0.86 (0.72 to 1.01) 0.071
 Spine 1.15 (0.94 to 1.41) 1.03 (0.79 to 1.27) 0.789
 Sports 1.05 (0.90 to 1.24) 0.85 (0.70 to 1.01) 0.068
 Trauma 1.32 (1.05 to 1.65) 1.11 (0.84 to 1.38) 0.419
Distance
 Fifty miles or more (reference)
 Less than fifty miles 1.26 (1.11 to 1.43) 1.18 (1.03 to 1.33) 0.016
*

The values are given as the odds ratio, with the 95% CI in parentheses, predicting less satisfaction in univariate (unadjusted) and multivariate models adjusting for age and sex and interactions between them.

These values were significant at p < 0.05 in the unadjusted analysis.

The predictive model identified age, distance, sex, subspecialty, distance × subspecialty, sex × subspecialty, age × sex, and distance × sex as the top predictors. The interaction between age and sex achieved significance (p = 0.022), with a model odds ratio of 0.48 (95% CI, 0.26 to 0.90), and younger male patients (age of <55.6 years) had more stable satisfaction than younger female patients, whose probability of less satisfaction decreased sharply with age (Fig. 2-A). The predictive model also demonstrated that the effect of age on patient satisfaction was maintained across all age groups, both sexes, all travel distances, and all orthopaedic subspecialties (Figs. 2-A and 2-B). The AUC of the predictive model, a measure of its accuracy, was 0.61 (95% CI, 0.60 to 0.62), and the tenfold cross-validated AUC was 0.60.

Figs. 2-A and 2-B Graphs showing the predicted probability of less satisfaction according to different variables. The shaded area represents the interquartile range.

Fig. 2-A.

Fig. 2-A

Predicted probability of less satisfaction according to age, stratified by sex (columns) and distance (less than fifty miles and fifty miles or more).

Fig. 2-B.

Fig. 2-B

Predicted probability of less satisfaction according to age, stratified by distance (columns) and subspecialty. Joints = adult reconstruction.

Discussion

The results of this study demonstrate that age predicts patient satisfaction as measured by the Press Ganey Medical Practice Survey in an orthopaedic outpatient setting. As a group, older patients reported greater satisfaction. These results are consistent with those of other studies demonstrating greater satisfaction in older patients6-17. In contrast, Mira et al. found that older inpatients tended to be more satisfied, while younger outpatients were more satisfied24. The interaction between age and sex also was significant, with younger male patients tending to have greater and more stable satisfaction than younger female patients (Fig. 2-A). Travel distance was also found to be a significant predictor of patient satisfaction; however, the effects of travel distance on patient satisfaction are likely due to differences in the demographic composition of these groups in our patient population. In particular, we found that patients who traveled farther tended to be older than patients who traveled shorter distances (data not shown).

Interestingly, no significant differences in patient satisfaction were identified among orthopaedic subspecialties when controlling for other factors (Table II and Fig. 2-B). These results suggest that comparisons among orthopaedic subspecialty providers at a single institution are valid as long as the patient populations do not differ significantly with regard to one or more characteristics that have been shown to influence patient satisfaction, most notably, age.

A notable limitation of this study was the fact that it was limited to a single orthopaedic department at a single academic institution. Therefore, the results may not be generalizable to other patient populations. For example, our state’s Medicaid enrollment rate (13%) is significantly lower than the national mean (21%), and the Medicaid enrollment in our patient cohort (1.1%) is significantly lower than our state’s Medicaid enrollment rate25. There are several possible explanations for these discrepancies. The most obvious potential reason is that a responder bias exists for patient satisfaction surveys. To our knowledge, no studies to date have identified significant responder bias for patient satisfaction surveys.

Although our model is useful for describing satisfaction in our orthopaedic outpatient population, its low accuracy (60%) is insufficient for predicting satisfaction at the individual patient level. Patient satisfaction is multifactorial and it is likely that a number of other variables outside the scope of the current study also explain, to various degrees, variation in patient satisfaction.

In summary, our findings demonstrate that non-modifiable patient variables significantly influence patient satisfaction as measured by the Press Ganey Medical Practice Survey. However, patient satisfaction is multifactorial and cannot be fully understood in terms of a single variable or group of variables. Interactions between variables may explain further differences in satisfaction scores as demonstrated by our predictive model. These results indicate that factors outside the control of the provider significantly influence patient satisfaction, a finding that challenges the validity of comparing patient satisfaction scores between populations that differ significantly with regard to one or more of these characteristics6,26.

Appendix

Methods Corresponding to Table II

In Table II, we evaluated each predictor of less satisfaction with use of generalized estimating equation logistic regression with an exchangeable correlation matrix to account for multiple responses from the same person. Results included odds ratios, 95% CIs, and p values from univariate and multivariate models. The multivariate models controlled for age, sex, and their interactions with each predictor. The adjusted analyses for age and sex used age as a categorical variable. For the remaining variables, a natural cubic spline with two degrees of freedom (one knot at the median of 55.6 years) was used to model age, indicated as “ns(Age,2).” The cubic spline transformation allowed us to evaluate nonlinear relationships with less satisfaction and avoid the empty cell issue that occurred when the categorical version of age was used to form interactions (no subjects younger than thirty years of age had Medicare insurance and no subjects eighty years of age or older were employed).

Each model in Table II consisted of interactions between a predictor and age and sex. For example, for distance, the adjusted odds ratio was extracted for the main or marginal effect of distance from the model: less satisfaction ∼ ns(Age,2) + sex + distance + ns(Age,2) × distance + sex × distance. Marginal effects for each variable in the presence of interactions were extracted with use of STATA version 12.0 (StataCorp, College Station, Texas), where adjusted predictions were estimated for each subject and were averaged with use of the “margins” command. Odds ratios were formed from the mean adjusted predictions and the delta method was used to estimate the standard errors and construct the 95% CIs with use of the “nlcom” command. The p values were calculated by applying the “test” command to the odds ratio against the null hypothesis of odds ratio = 1 (as the “nlcom” default is to test the estimated value [here, odds ratios] against a null hypothesis of 0).

Methods Corresponding to the Predictive Model-Building Process (Figs. 2-A and 2-B and Table III)

TABLE III.

Odds Ratios and P Values from Our Predictive Model*

Variable Odds Ratio P Value
Distance of less than fifty miles 1.13 (0.79 ∼ 1.59) 0.505
ns(Age,2)
 Age of <55.6 years 0.27 (0.15 ∼ 0.50) <0.001
 Age of ≥55.6 years 0.15 (0.08 ∼ 0.28) <0.001
Female sex 1.64 (0.96 ∼ 2.79) 0.068
Subspecialty
 Foot and ankle 0.89 (0.57 ∼ 1.39) 0.609
 Hand 1.28 (0.81 ∼ 2.01) 0.296
 Nonoperative 1.28 (0.79 ∼ 2.07) 0.311
 Spine 1.17 (0.72 ∼ 1.88) 0.523
 Sports 0.67 (0.44 ∼ 1.01) 0.053
 Trauma 0.93 (0.53 ∼ 1.63) 0.804
Distance of <50 miles × subspecialty foot and ankle 1.19 (0.77 ∼ 1.86) 0.434
Distance of <50 miles × subspecialty hand 0.87 (0.55 ∼ 1.39) 0.562
Distance of <50 miles × subspecialty nonoperative 0.78 (0.49 ∼ 1.25) 0.299
Distance of <50 miles × subspecialty spine 1.20 (0.74 ∼ 1.94) 0.465
Distance of <50 miles × subspecialty sports 1.30 (0.86 ∼ 1.97) 0.218
Distance of <50 miles × subspecialty trauma 1.43 (0.80 ∼ 2.58) 0.232
ns(Age,2) of <55.6 years × female sex 0.48 (0.26 ∼ 0.90) 0.022
ns(Age,2) of ≥55.6 years × female sex 1.26 (0.59 ∼ 2.71) 0.555
Female sex × subspecialty foot and ankle 0.76 (0.53 ∼ 1.09) 0.138
Female sex × subspecialty hand 0.87 (0.61 ∼ 1.25) 0.453
Female sex × subspecialty nonoperative 0.75 (0.54 ∼ 1.03) 0.075
Female sex × subspecialty spine 0.71 (0.46 ∼ 1.09) 0.118
Female sex × subspecialty sports 1.06 (0.75 ∼ 1.49) 0.743
Female sex × subspecialty trauma 0.87 (0.54 ∼ 1.40) 0.570
Distance of <50 miles × female sex 0.97 (0.74 ∼ 1.26) 0.812
Insurance
 Medicare 0.92 (0.78 ∼ 1.08) 0.297
 Other 1.09 (0.93 ∼ 1.29) 0.296
Employment
 Other 0.98 (0.87 ∼ 1.10) 0.766
 Retired 0.96 (0.81 ∼ 1.15) 0.682
 Unemployed 0.92 (0.79 ∼ 1.06) 0.247
Distance of <50 miles × ns(Age,2) of <55.6 years 1.40 (0.60 ∼ 3.25) 0.440
Distance of <50 miles × ns(Age,2) of ≥55.6 years 0.82 (0.27 ∼ 2.48) 0.720
ns(Age,2) of <55.6 years × subspecialty foot and ankle 1.32 (0.31 ∼ 5.68) 0.710
ns(Age,2) of ≥55.6 years × subspecialty foot and ankle 0.94 (0.25 ∼ 3.54) 0.925
ns(Age,2) of <55.6 years × subspecialty hand 1.67 (0.38 ∼ 7.34) 0.497
ns(Age,2) of ≥55.6 years × subspecialty hand 0.62 (0.16 ∼ 2.30) 0.472
ns(Age,2) of <55.6 years × subspecialty nonoperative 1.34 (0.34 ∼ 5.27) 0.671
ns(Age,2) of ≥55.6 years × subspecialty nonoperative 1.51 (0.54 ∼ 4.22) 0.428
ns(Age,2) of <55.6 years × subspecialty spine 3.78 (0.48 ∼ 29.64) 0.205
ns(Age,2) of ≥55.6 years × subspecialty spine 1.56 (0.43 ∼ 5.64) 0.497
ns(Age,2) of <55.6 years × subspecialty sports 3.60 (0.91 ∼ 14.32) 0.069
ns(Age,2) of ≥55.6 years × subspecialty sports 2.46 (0.68 ∼ 8.84) 0.168
ns(Age,2) of <55.6 years × subspecialty trauma 2.05 (0.35 ∼ 12.17) 0.430
ns(Age,2) of ≥55.6 years × subspecialty trauma 1.44 (0.26 ∼ 7.95) 0.674
*

The notation ns(Age,2) refers to the natural spline of age with 2 degrees of freedom (1 knot at the median age of 55.6 years).

The values are given as the odds ratio, with the 95% CI in parentheses.

We used model averaging implemented in the MuMIn (1) package in R27,28 to construct a predictive model for less satisfaction. To simplify the model space explored by the averaging procedure, we first constructed a generalized estimating equation logistic regression model with use of all six possible predictors (age, sex, type of insurance, subspecialty, geographic location, employment status), and only considered two-way interactions among predictors that had a significant type-3 p value of <0.05 in this model. Type-3 p values were calculated on the basis of a Wald chi-square test (approximation to the likelihood-ratio test) comparing the full model (six predictors) with the model with each term omitted. All predictors had significant type-3 p values except insurance and employment status.

Thus, the model averaging procedure explored all possible subsets of models from this model space: (ns[Age,2] + subspecialty + sex + distance)2 + insurance + employment status. A total of 452 models were explored, and top models were identified by selecting the set with highest quasi-likelihood under the independence model criterion (QIC) so that the cumulative weight achieved 95%. Our final predictive model was obtained by averaging across these top models, a process that is automated by the MuMIn package (Table III). The standard errors and p values reflect model selection uncertainty by incorporating QIC weights. QIC weights were as follows: distance, ns(Age,2), sex, and subspecialty had the highest weight (QIC = 1), followed by distance × subspecialty (0.98), sex × subspecialty (QIC = 0.84), ns(Age,2) × sex (QIC = 0.71), distance × sex (QIC = 0.30), employment status (QIC = 0.09), distance × ns(Age,2) (QIC = 0.04), and ns(Age,2) × subspecialty (QIC < 0.01).

Footnotes

Investigation performed at the Department of Orthopaedics, University of Utah, Salt Lake City, Utah

Disclosure: One or more of the authors received payments or services, either directly or indirectly (i.e., via his or her institution), from a third party in support of any aspect of this work. In addition, one or more of the authors, or his or her institution, has had a financial relationship, in the thirty-six months prior to submission of this work, with an entity in the biomedical arena that could be perceived to influence or have the potential to influence what is written in this work. No author has had any other relationships, or has engaged in any other activities, that could be perceived to influence or have the potential to influence what is written in this work. The complete Disclosures of Potential Conflicts of Interest submitted by authors are always provided with the online version of the article.

References


Articles from The Journal of Bone and Joint Surgery. American volume are provided here courtesy of Journal of Bone and Joint Surgery, Inc.

RESOURCES