Abstract
Background
Recent work has shown that clinically complex patients are more likely to receive recommended care, but it is unknown whether higher achievement on individual performance goals results in improved care for complex patients or detracts from other important but unmeasured aspects of care, resulting in unmet needs and lower satisfaction with care.
Objective
To examine the relationship between measured performance and satisfaction with care among clinically complex patients
Design and Participants
An observational analysis of a national sample of 35,927 veterans included in the External Peer Review Program in fiscal years 2003 and 2004.
Measurements
First, compliance with individual performance measures (breast cancer screening with mammography, colorectal cancer screening, influenza vaccination, pneumococcal vaccination, lipid monitoring, use of ACE inhibitor in heart failure, and diabetic eye examination), as well as overall receipt of recommended care, was estimated as a function of each patient’s clinical complexity. Second, global satisfaction with care was estimated as a function of clinical complexity and compliance with performance measures.
Main Results
Higher clinical complexity was predictive of slightly higher overall performance (OR 1.13, 95% CI 1.09 to 1.18) and higher performance on most individual performance measures, an effect that was mediated by increased visit frequency. High measured performance was associated with higher satisfaction with care among patients with high clinical complexity. In fact, as complexity increased, the effect of achieving high performance on the odds of being satisfied with care also increased
Conclusions
Not only was measured performance higher in clinically complex patients, but satisfaction with care was also higher among clinically complex patients with high measured performance, suggesting that compliance with performance measures in clinically complex patients does not crowd out unmeasured care.
KEY WORDS: measured performance, patient satisfaction, clinically complex patients
INTRODUCTION
Performance measurement is potentially a powerful tool to improve health care quality,1 and has been widely adopted to improve quality of care. One potential limitation of performance measurement is the impossibility of measuring all important aspects of care. By necessity, measures of clinical care are limited to what is measurable, and what is measurable is not always what is most important. As a result, clinicians have raised the concern that performance measurement focuses physicians’ attention too narrowly on compliance with performance measures and detracts from the broader needs of the individual patient.2 If an individual patient has needs that lie outside of the measured conditions, performance measurement may “crowd-out” unmeasured care by creating incentives for physicians to attend to measured medical problems over unmeasured ones.3,4 If patients’ needs are unmet, compliance with performance measures may reduce satisfaction with care.
Clinically complex patients may be particularly vulnerable to this type of crowd-out because complex patients often have needs that are unmeasured, and those needs may compete for time with performance measures.5,6 As a result, many have questioned whether applying performance measurement to clinically complex patients is feasible and whether it results in higher overall quality.
Recent work has shown that clinically complex patients are more likely to receive recommended care,7,8 but it is unknown whether this focus on achieving performance goals results in improved satisfaction with care for clinically complex patients or detracts from other important but unmeasured aspects of care and thus decreases satisfaction with care. Therefore, our goal is to test whether high performance inadvertently reduces some satisfaction with care in the setting of clinically complex patients, who may be particularly vulnerable to this unintended consequence. We first test the impact of clinical complexity on measured performance and then test the impact of higher measured performance on satisfaction with care among clinically complex patients.
METHODS
Study Population
The Veterans Health Administration (VHA) has been measuring provider performance for over a decade,9,10 routinely collecting nationwide data on performance measures through its External Peer Review Program (EPRP). These data are collected by trained abstracters and interrater reliability is high.11,12 To be eligible for inclusion in the EPRP, a patient must be continuously enrolled in the VA for two years and have a qualifying visit anywhere in VHA in the previous 12 months. Random samples are selected from this cohort on a monthly basis from each facility. Veterans are excluded from the final sample if they have a documented diagnosis of cancer of the liver, esophagus, or pancreas; they are enrolled in a VHA or community-based hospice program; or they have a life expectancy of less than six months.
In fiscal years 2003 and 2004 veterans included in EPRP were also administered the Survey of Healthcare Experiences of Patients (SHEP), an ongoing survey of veterans’ experiences and satisfaction with care at the VA.13 SHEP also gathers detailed self-reported information about race and socioeconomic status of respondents. The response rate for the outpatient SHEP survey is 70.3%.13 Our study sample included all veterans in the outpatient EPPR and SHEP survey in fiscal years 2003 and 2004 ( = 35,927).
Performance Measures
In these analyses, we use process measures available in the EPRP that are similar to those used in other common outpatient performance measurement systems:14,15 breast cancer screening with mammography, colorectal cancer screening, influenza vaccination, pneumococcal vaccination, lipid monitoring, use of an ACE inhibitor in heart failure, and diabetic eye examination. Eligibility for each performance measure has been previously described.11 We also assess each patient’s overall receipt of recommended care, computed as the total number of performance measures passed, divided by the total number of performance measures triggered, expressed as a percentage. Because patients trigger different measures, and some measures have lower pass rates than others,16 each observed score is compared against an expected score,7,17 calculated as the mean of the population pass rate for each measure. The difference between the observed and expected score is the overall performance score. Thus, a patient with a score of greater than zero received recommended care more frequently than expected whereas a patient with a score of less than zero received recommended care less frequently than expected.
Clinical Complexity
To capture each patient’s clinical complexity, we used the diagnostic cost groups–hierarchical condition categories (DCG) model. The DCG uses age, sex, and diagnoses to summarize a patient’s medical conditions using all available International Classification Diseases, 9th revision, Clinical Modification (ICD-9-CM) codes. DCG then imposes a hierarchy that identifies only the most costly manifestation of each distinct disease, predicts expenditures in the present year (i.e. the year in which performance is measured)18 and then summarizes this information as the ratio of predicted to mean resource utilization.19 We used ICD-9-CM codes from the VA patient treatment files and the outpatient care file over a one-year period to summarize health care utilization over the one-year prior to inclusion in the EPRP using software version 2.1.1 from DxCG, Inc.. Following prior VA studies, we use Medicare weights to estimate risk scores in our DCG-HCC model.20,21
Satisfaction with Care
To measure unmet needs and satisfaction with care, we used the global satisfaction measure, “All things considered, how satisfied are you with your health care in the VA?” Satisfaction was rated on a 5-point Likert scale, ranging from completely satisfied (value of 1) to completely dissatisfied (value of 5). We dichotomized these responses to indicate whether the respondent was satisfied with their health care (vs. neither satisfied or dissatisfied with care). Alternative categorizations of satisfaction based on this variable were tested and produced similar results; therefore we present results based on the dichotomized variable here. We chose to measure satisfaction because performance measures may compete with care that is unmeasured but important to patients, thus discounting patient preferences and reducing satisfaction with care.
Patient Characteristics
We included several key patient characteristics from SHEP that have been hypothesized to confound the relationship between performance and clinical complexity.22 These characteristics were chosen a priori and include age, age-squared, race, sex, and income.
Analyses
We used hierarchical logistic regression to estimate the relationship between overall performance and individual patients’ clinical complexity. Because the dependent variable, overall measured performance, was not normally distributed and transformation of the variable did not result in a normal distribution, the dependent variable was dichotomized to equal 1 if performance was higher than expected and 0 if performance was lower than expected. The main independent variable was a continuous measure of clinical complexity, as described above. For simplicity of interpretation, we present the predicted odds of receiving recommended care for a person with clinical complexity in the 10th percentile compared to a person with clinical complexity in the 90th percentile (between the range of 0.09 to 1.7), rather than for a one-unit change in clinical complexity. We also included the covariates of age, age-squared, race, sex, and income group as well as a Veteran Integrated Service Networks (VISN)-specific random effect. The random effect accounts for clustering within VISN and estimates the variance in performance between VISNs.
The relationship between individual measures of performance and patient clinical complexity was also tested using hierarchical logistic regression. In this case, the dependent variable was compliance with each measure of performance, where 1 indicated a patient received recommended care and 0 indicated a patient was eligible for but did not receive recommended care. A separate regression equation was estimated for each measure.
We considered office visit frequency as a potential mediator in the relationship between performance and clinical complexity by re-estimating the relationship between measured performance and clinical complexity controlling for visit frequency, as frequent office visits give more opportunity to comply with performance measures. Additionally, patients that see multiple providers (resulting in frequent office visits) may increase the likelihood that someone will remember to complete the measured task. Outpatient visit frequency was measured in the most recent year of performance measurement using outpatient claims data where visits with physicians or physician-extenders were considered outpatient visits. Preliminary analyses suggested that the relationship between performance and visit frequency was non-linear, so we controlled for visit frequency using the log of the number of visits. A Hosmer–Lemeshow test indicates no evidence of lack of model fit (Pearson chi-squared 33829.86; p-value 0.35).
Finally, we tested the hypothesis that compliance with performance measures leads to unmet needs and lower satisfaction among clinically complex patients using hierarchical logistic regression to test the impact of overall performance on satisfaction with care controlling for complexity and visit frequency. We also included an interaction term between performance and clinical complexity to consider whether the influence of performance on satisfaction is moderated by complexity. If, for example, performance is positively associated with satisfaction, a negative interaction between performance and complexity would suggest that this is less true (or perhaps not true) when patients are highly complex, suggesting that higher performance predicts lower satisfaction among highly complex patients. We also investigated whether visit frequency mediated the relationship between performance and satisfaction, found it had no effect, so do not include visit frequency in these models.
Sensitivity Analyses
Although EPRP excludes veterans with a short life-expectancy, life-expectancy is not always well documented in clinical charts and could influence the relationship between measured performance and clinical complexity. To test for the sensitivity of our findings to this possibility, we estimated Charlson comorbidity scores, which are predictive of mortality in administrative data.23,24 We then re-estimated our analyses using only patients with a Charlson score of less than 4 ( = 32,656), thereby excluding those with a high predicted probability of death.
Since a significant proportion of veterans receive some of their care outside of the VA,25 we tested whether this potential source of missing data had an impact on our findings by re-estimating our analyses on patients who responded to the question, “During the past 12 months, have you been seen by...” with the answer, “VA providers only” ( = 16,732).
RESULTS
A total of 35,927 people were included in the study. Characteristics of the study cohort are summarized in Table 1. Veterans with higher clinical complexity had higher unadjusted performance overall and for most of the individual measures (Table 2). Higher clinical complexity was associated with higher measured performance for all deciles of clinical complexity. Higher clinical complexity was also associated with higher visit frequency (see Fig. 1).
Table 1.
Characteristics of Study Cohort, Stratified by Low Clinical Complexity (Below the Median Value of Clinical Complexity) vs. High Clinical Complexity (Above the Median)
| Low clinical complexity ( = 17,963) | High clinical complexity ( = 17,964) | -value* | |
|---|---|---|---|
| Clinical complexity score, mean (sd) | 0.21 (0.11) | 1.24 (1.00) | <.001 |
| Overall satisfaction, mean (sd) | 2.08 (1.3) | 2.06 (1.25) | 0.09 |
| Age, mean (sd) | 66.0 (13.6) | 67.6 (13.2) | <.001 |
| White | 88.6% | 87.2% | <.001 |
| Male | 79.7% | 83.7% | <.001 |
| Income, $ | <.001 | ||
| <15,000 | 35.8% | 44.0% | |
| 15,000 to 30,000 | 41.4% | 39.3% | |
| 30,000 to 60,000 | 18.0% | 14.1% | |
| >60,000 | 4.7% | 2.6% | |
| Number of outpatient visits in prior year, mean (sd) | 4.5 (4.6) | 11.3 (11.3) | <.001 |
| Charlson comorbidity index, mean (sd) | 0.96 (0.88) | 2.2 (1.5) | <.001 |
*P-values based on Chi-square tests for categorical variables and t-tests for continuous variables
Table 2.
Number of Study Participants Eligible for any Process Measure (for Overall Performance) and Each Individual Measure, and Proportion Who Received Recommended Care or Achieved Outcomes Benchmarks Among Participants with Low Clinical Complexity
(Below the Median Value of Clinical Complexity) vs. High Complexity (Above the Median)
| Low clinical complexity | High clinical complexity | -value* | ||
|---|---|---|---|---|
| Process measures | ||||
| Overall process performance higher than expected | 35,927 | 53.1% | 56.1% | <.001 |
| Mammography | 2,917 | 86.6% | 84.1% | 0.06 |
| Colon cancer screening | 30,865 | 72.4% | 72.7% | 0.6 |
| Influenza vaccination | 20,137 | 71.5% | 77.5% | <.001 |
| Pneumococcal vaccination | 29,619 | 85.7% | 88.6% | <.001 |
| Lipids checked | 33,573 | 93.5% | 95.5% | <.001 |
| ACE-I in heart failure | 3,103 | 86.5% | 88.2% | 0.2 |
| Diabetic eye examination | 10,627 | 76.2% | 80.4% | <.001 |
*P-values based on chi-square test
Figure 1.
Mean overall process performance (on left) and mean number of office visits (on right) across deciles of clinical complexity.
Measured Performance is Higher among Clinically Complex Patients
In multivariate analyses, higher clinical complexity was associated with higher odds of receiving recommended processes of care more often than expected (Table 3). The odds of receiving recommended care was 1.13 (95% confidence intervals (CI) 1.09 to 1.18; p-value < .001) times higher for patients with clinical complexity in the 90th percentile (vs. 10th percentile). In addition, patient income was inversely associated with process performance.
Table 3.
Odds of Receiving Recommended Processes of Care More Often than Expected from Multivariable Model
| Odds ratio | 95% CI | -value* | |
|---|---|---|---|
| Clinical complexity | 1.13 | 1.09 to 1.18 | < .001 |
| Age | 1.10 | 1.08 to 1.12 | < .001 |
| Age-square | 1.00 | 1.00 to 1.00 | < .001 |
| Non-white | 0.96 | 0.90 to 1.03 | 0.28 |
| Male | 0.84 | 0.80 to 0.89 | < .001 |
| Income less than $15,000 | 0.79 | 0.71 to 0.89 | < .001 |
| Income $15,000 to $30,000 | 0.93 | 0.83 to 1.04 | 0.19 |
| Income $30,000 to $60,000 | 0.99 | 0.87 to 1.11 | 0.81 |
| Income greater than $60,000 | Omitted |
*P-value based on hierarchical logistic regression results
The relationship between higher clinical complexity and higher measured performance held true for 5 of the 7 individual performance measures in multivariate analyses (colon cancer screening, influenza vaccination, pneumococcal vaccination, lipids checked, and diabetic eye exam) (Table 4).
Table 4.
Effect of Clinical Complexity on Odds of Receiving Recommended Care in Expected from Multivariable Model
| Odds ratio | 95% CI | -value** | |
|---|---|---|---|
| Overall process performance higher than expected | 1.13 | 1.09 to 1.18 | <.001 |
| Mammography | 0.90 | 0.75 to 1.09 | 0.27 |
| Colon cancer screening | 1.07 | 1.02 to 1.12 | 0.006 |
| Influenza vaccination | 1.39 | 1.30 to 1.49 | <.001 |
| Pneumococcal vaccination | 1.30 | 1.21 to 1.40 | <.001 |
| Lipids checked | 1.63 | 1.44 to 1.84 | <.001 |
| ACE-I in heart failure | 0.88 | 0.75 to 1.02 | 0.08 |
| Diabetic eye examination | 1.13 | 1.04 to 1.23 | 0.002 |
*The relationship between clinical complexity and each performance measure is estimated separately using multivariate regression controlling for age, age-squared, race, sex, and income category
**P-value based on hierarchical logistic regression results
This relationship between measured performance and clinical complexity was mediated by visit frequency. After controlling for visit frequency, the relationship reversed directions, with higher clinical complexity associated with lower odds of receiving recommended processes of care more often than expected. In addition, visit frequency is a strong predictor of process performance (Table 5). The odds of receiving recommended care for patients with clinical complexity in the 90th percentile (vs. 10th percentile) was lower by a factor of 0.92 (95% CI .88 to .96; p-value <.001). Controlling for visit frequency also reduced the effect of clinical complexity on the odds of receiving recommended care for the individual measures (Table 6).
Table 5.
Odds of Receiving Recommended Processes of Care More often than Expected, Controlling for the Number of Visits, from Multivariable Model
| Odds ratio | 95% CI | -value* | |
|---|---|---|---|
| Clinical complexity | 0.92 | 0.88 to 0.96 | <.001 |
| Age | 1.09 | 1.08 to 1.11 | <.001 |
| Age-square | 1.00 | 1.00 to 1.00 | <.001 |
| Non-white | 0.90 | 0.84 to 0.97 | 0.00 |
| Male | 0.91 | 0.86 to 0.96 | 0.001 |
| Income less than $15,000 | 0.73 | 0.65 to 0.82 | <.001 |
| Income $15,000 to $30,000 | 0.89 | 0.79 to 1.00 | 0.04 |
| Income $30,000 to $60,000 | 0.96 | 0.85 to 1.09 | 0.53 |
| Income greater than $60,000 | Omitted | ||
| Log of number of visits | 1.28 | 1.25 to 1.31 | <.001 |
*P-value based on hierarchical logistic regression results
Table 6.
Effect of Clinical Complexity on Odds of Receiving Recommended Care Controlling for Visit Number in Multivariate Models Controlling for Patient Characteristics*
| Odds ratio | 95% CI | -value** | |
|---|---|---|---|
| Overall process performance higher than expected | 0.92 | 0.88 to 0.96 | <.001 |
| Mammography | 0.60 | 0.49 to 1.11 | <.001 |
| Colon cancer screening | 0.97 | 0.92 to 1.00 | 0.23 |
| Influenza vaccination | 1.03 | 0.96 to 0.97 | 0.36 |
| Pneumococcal vaccination | 1.13 | 1.05 to 0.96 | <.001 |
| Lipids checked | 1.08 | 0.96 to 0.82 | 0.22 |
| ACE-I in heart failure | 0.74 | 0.63 to 1.00 | <.001 |
| Diabetic eye examination | 0.70 | 0.65 to 1.09 | <.001 |
*The relationship between clinical complexity and each performance measure is estimated separately using multivariate regression controlling for age, age-squared, race, sex, and income category
**P-value based on hierarchical logistic regression results
Clinically Complex Patients with Higher Performance are more Likely to be Satisfied with Care
Controlling for complexity, a one-unit increase in measured performance was associated with 1.16 times higher odds (95% CI 1.11 to 1.22; p-value <.001) of being satisfied with care. The interaction term between complexity and performance was positive and significant (OR 1.06, 95% CI 1.00 to 1.12; p-value 0.04); as complexity increased, the effect of achieving high performance on the odds of being satisfied also increased. For example, a one-unit increase in performance increased the odds of satisfaction by 1.12 at the 10th percentile of complexity and by 1.23 at the 90th percentile of complexity. These findings suggest that the performance measurement does not compete with unmeasured aspects of care, but rather improves satisfaction with care among complex patients.
Sensitivity Analyses
Excluding people with Charlson comorbidity score of 4 or greater had little influence on the magnitude or statistical significance of the results. Limiting the analyses to veterans who received all of their care at the VA also did not appreciably change the results. These results are available upon request.
DISCUSSION
We found that patients with higher clinical complexity had higher measured performance on common process measures used to assess the quality of outpatient care, an effect that was mediated by high frequency of outpatient visits. We also found that satisfaction with care was higher among clinically complex patients with high measured performance, suggesting that compliance with performance measures in clinically complex patients does not crowd out unmeasured care. While the effect of complexity on performance was small (with smaller relative risks),26 we examined performance measures that apply to millions of people nationally. Furthermore, with the aging of the US population, the number of patients who might be considered as “clinically complex” is high and growing.27 Thus, a population-based estimate of the effect size may be substantial.
Prior studies that have empirically examined the relationship between clinical conditions and processes of care8,17 have also found that as the number of comorbid conditions increased, measured performance did as well, with similar magnitudes to the differences that we found. Our work adds to this literature by showing that this relationship is mediated by visit frequency. Furthermore, we directly examine the impact of performance on satisfaction with care.
A common concern related to performance measurement has been that patients with multiple complex conditions often have priorities that lie outside of and directly compete with the priorities of performance measures. If this is the case, focusing attention on measured dimensions of care may diminish attention to unmeasured care, inadvertently harming care that is not covered by performance measurement. Our finding of higher patient satisfaction among clinically complex patients who receive more recommended services suggests that this is not the case. In contrast, our results suggest that high performance leads to higher levels of satisfaction with care, an effect that is stronger among clinically complex patients.
Prior work on the relationship between measured performance and satisfaction with care has largely been condition-specific evaluations which may not relate well to overall satisfaction with care.28–34 One prior study has examined the relationship between global ratings of health and measures of overall performance35 and found that higher global ratings of health care were not associated with higher performance. However, this study was based on 236 individuals, compared to the more than 35,000 patients we study here, and thus may have been underpowered to detect these differences across patients.
The finding that higher performance is associated with higher satisfaction with care is consistent with what is known about patient expectations, suggesting that patients who present for medical attention are highly dissatisfied when they have unmet expectations.36 This may be particularly true among clinically complex patients, who may have higher expectations of physician visits given a higher burden of chronic illness.
Our study has several limitations. First, our study cohort is a national sample of veterans, limiting the generalizability of our results. Second, it is possible that our results are confounded by provider characteristics that make it more likely that some patients will both have higher rates of diagnosis codes and be more likely to comply with performance measures. The hierarchical structure of the DCG model we use, however, is designed to ameliorate intentional coding proliferation by only counting the most severe condition within condition categories.19 Third, we use a single item assessment of patient satisfaction, which may be a limitation. However, results of multiple studies show that global ratings are reproducible37,38 and have evidence of construct validity as shown by high correlations with longer, multi-item scales and alternative measures.39–41 Furthermore, the global ratings performed similarly to the multi-item scales.42,43 Finally, as with any survey, the survey results may be biased by non-response despite the high response rate of 70%.
These results have important implications. Our findings provide good news, in that fears of serious “crowding out” of patient satisfaction do not appear to be substantiated among complex patients. Furthermore, these results reinforce the importance of performance measurement among clinically complex patients. Although prior work has questioned whether applying performance measurement to clinically complex patients results in higher overall quality of care, our results suggest that compliance with performance measures is a predictor of satisfaction with care for clinically complex patients. Our results also emphasize the importance of increased office visits to overcome the potentially negative effect of competing demands on compliance with performance measures. Our finding of increased performance with increased visit frequency suggests that increased contact increases the likelihood that patients will receive recommended care and that providers might schedule more frequent visits in an effort to meet performance expectations. These findings also highlight the importance of developing longitudinal performance measures. While many performance measures are generated over time (i.e. patients with diabetes who had a diabetic eye examination within the last 12 months), some measures of performance capture care given during a single episode of care. For example, some measures examine whether a physician responded appropriately to an elevated blood pressure at that office visit. As clinically complex patients have competing demands that limit a physician’s ability to address patient issues in a single office visit, measures that are based on patient care at one visit might penalize the physicians who care for clinically complex patients. Instead, measures that are based on care delivered over a longer period of time may be more appropriate.
As the United States moves toward widespread implementation of performance measurement, it is crucial to understand how these systems affect overall clinical care, particularly among clinically complex patients. Our study provides evidence that high performance is associated with improved patient satisfaction among clinically complex patients, and thus, provides support for the use of performance measurement to improve quality of care among clinically complex patients.
Acknowledgments
The authors thank Anne Canamucio, M.S., for her excellent assistance with data analyses, and Judy Shea, PhD, for her comments on this manuscript.
Funding for the study was provided by the Center for Health Equity Research and Promotion, Philadelphia VA and by the Atlantic Philanthropies, SGIM/ACGIM, the John A. Hartford Foundation, and ASP. Rachel Werner is supported in part by a VA HSR&D Career Development Award. Virginia Chang is supported in part by grant K12-HD043459 from the NIH/NICHD.
Conflict of Interest None disclosed.
References
- 1.Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003;41:I-30–8. [DOI] [PubMed]
- 2.Werner RM, Asch DA. Clinical concerns about clinical performance measurement. Ann Fam Med. 2007;5:159–63. [DOI] [PMC free article] [PubMed]
- 3.Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA. 2005;293:1239–44. [DOI] [PubMed]
- 4.Casalino LP. The unintended consequences of measuring quality on the quality of medical care. N Engl J Med. 1999;341:1147–50. [DOI] [PubMed]
- 5.Boyd CM, Darer J, Boult C, Fried LP, Boult L, Wu AW. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA. 2005;294:716–24. [DOI] [PubMed]
- 6.Tinetti ME, Bogardus ST Jr., Agostini JV. Potential pitfalls of disease-specific guidelines for patients with multiple conditions. N Engl J Med. 2004;351:2870–74. [DOI] [PubMed]
- 7.Min LC, Wenger NS, Fung C, et al. Multimorbidity is associated with better quality of care among vulnerable elders. Med Care. 2007;45:480–8. [DOI] [PubMed]
- 8.Higashi T, Wenger NS, Adams JL, et al. Relationship between Number of Medical Conditions and Quality of Care. N Engl J Med. 2007;356:2496–504. [DOI] [PubMed]
- 9.Kizer KW. The “new VA”: a national laboratory for health care quality management. Am J Med Qual. 1999;14:3–20. [DOI] [PubMed]
- 10.Kizer KW, Demakis JG, Feussner JR. Reinventing VA health care: systematizing quality improvement and quality innovation. Med Care. 2000;38:I-7–16. [DOI] [PubMed]
- 11.Jha AK, Perlin JB, Kizer KW, Dudley RA. Effect of the transformation of the Veterans Affairs health care system on the quality of care. N Engl J Med. 2003;348:2218–27. [DOI] [PubMed]
- 12.Doebbeling BN, Vaughn TE, Woolson RF, et al. Benchmarking Veterans Affairs Medical Centers in the delivery of preventive health services: comparison of methods. Med Care. 2002;40:540–54. [DOI] [PubMed]
- 13.Wright SM, Craig T, Campbell S, Schaefer J, Humble C. Patient satisfaction of female and male users of Veterans Health Administration services. J Gen Intern Med. 2006;21:S26–32. [DOI] [PMC free article] [PubMed]
- 14.Centers for Medicare and Medicaid Services. Physician Quality Reporting Initiative. 2007; Available at: http://www.cms.hhs.gov/pqri/. Accessed June 11, 2008.
- 15.Joint Commission on Accreditation of Healthcare Organizations. HEDIS & Quality Measurement. 2006; Available at: http://web.ncqa.org/tabid/59/Default.aspx. Accessed 6/11/2008.
- 16.Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to Medicare beneficiaries: a profile at state and national levels. JAMA. 2000;284:1670–6. [DOI] [PubMed]
- 17.Min LC, Reuben DB, MacLean CH, et al. Predictors of overall quality of care provided to vulnerable older people. J Am Geriatr Soc. 2005;53:1705–11. [DOI] [PubMed]
- 18.Ellis RP, Pope GC, Iezzoni LI, et al. Diagnosis-based risk adjustment for Medicare capitation payments. Health Care Financ Rev. 1996;17:101–28. [PMC free article] [PubMed]
- 19.DxCG. DxCG RiskSmart Stand Alone V2.1.1 User Guide. Boston, MA 2006.
- 20.Rosen AK, Loveland S, Anderson JJ, et al. Evaluating diagnosis-based case-mix measures: how well do they apply to the VA population? Med Care. 2001;39:692–704. [DOI] [PubMed]
- 21.Petersen LA, Pietz K, Woodard LD, Byrne M. Comparison of the predictive validity of diagnosis-based risk adjusters for clinical outcomes. Med Care. 2005;43:61–7. [PubMed]
- 22.Werner RM, Greenfield S, Fung C, Turner BJ. Measuring quality of care in patients with multiple clinical conditions: summary of a conference conducted by the Society of General Internal Medicine. J Gen Intern Med. 2007;22:1206–11. [DOI] [PMC free article] [PubMed]
- 23.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–83. [DOI] [PubMed]
- 24.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–9. [DOI] [PubMed]
- 25.Weeks WB, Bott DM, Bazos DA, et al. Veterans Health Administration patients’ use of the private sector for coronary revascularization in New York: opportunities to improve outcomes by directing care to high-performance hospitals. Med Care. 2006;44:519–26. [DOI] [PubMed]
- 26.Zhang J, Yu KF. What’s the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. Am Med Assoc. 1998;280:1690–1. [DOI] [PubMed]
- 27.Anderson G, Horvarth J. Chronic Conditions: Making the Case for Ongoing Care. Baltimore: Johns Hopkins Press; 2002.
- 28.Breen A, Breen R. Back pain and satisfaction with chiropractic treatment: what role does the physical outcome play? Clin J Pain. 2003;19:263–8. [DOI] [PubMed]
- 29.Edlund MJ, Young AS, Kung FY, Sherbourne CD, Wells KB. Does satisfaction reflect the technical quality of mental health care? Health Serv Res. 2003;38:631–45. [DOI] [PMC free article] [PubMed]
- 30.Gross R, Tabenkin H, Porath A, et al. The relationship between primary care physicians’ adherence to guidelines for the treatment of diabetes and patient satisfaction: findings from a pilot study. Fam Pract. 2003;20:563–9. [DOI] [PubMed]
- 31.Orlando M, Meredith LS. Understanding the causal relationship between patient-reported interpersonal and technical quality of care for depression. Med Care. 2002;40:696–704. [DOI] [PubMed]
- 32.Meredith LS, Orlando M, Humphrey N, Camp P, Sherbourne CD. Are better ratings of the patient-provider relationship associated with higher quality care for depression? Med Care. 2001;39:349–60. [DOI] [PubMed]
- 33.Druss BG, Rosenheck RA, Stolar M. Patient satisfaction and administrative measures as indicators of the quality of mental health care. Psychiatr Serv. 1999;50:1053–8. [DOI] [PubMed]
- 34.Cleary PD, McNeil BJ. Patient satisfaction as an indicator of quality care. Inquiry. 1988;25:25–36. [PubMed]
- 35.Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med. 2006;144:665–72. [DOI] [PubMed]
- 36.Jackson JL, Chamberlin J, Kroenke K. Predictors of patient satisfaction. Soc Sci Med. 2001;52:609–20. [DOI] [PubMed]
- 37.Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med. 1998;73:993–7. [DOI] [PubMed]
- 38.Cohen DS, Colliver JA, Marcy MS, Fried ED, Swartz MH. Psychometric properties of a standardized-patient checklist and rating-scale form used to assess interpersonal and communication skills. Acad Med. 1996;71:S87–9. [DOI] [PubMed]
- 39.Hays RD, Shaul JA, Williams VS, et al. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care. 1999;37:22–31. [DOI] [PubMed]
- 40.Shea JA, Guerra CE, Ravenell KL, McDonald VJ, Henry CA, Asch DA. Health literacy weakly but consistently predicts primary care patient dissatisfaction. Int J Qual Health Care. 2007;19:45–9. [DOI] [PubMed]
- 41.Mah JK, Tough S, Fung T, Douglas-England K, Verhoef M. Parents’ global rating of mental health correlates with SF-36 scores and health services satisfaction. Qual Life Res. 2006;15:1395–401. [DOI] [PubMed]
- 42.Weech-Maldonado R, Morales LS, Elliott M, Spritzer K, Marshall G, Hays RD. Race/ethnicity, language, and patients’ assessments of care in Medicaid managed care. Health Serv Res. 2003;38:789–808. [DOI] [PMC free article] [PubMed]
- 43.Shea JA, Guerra CE, Weiner J, Aguirre AC, Ravenell KL, Asch DA. Adapting a patient satisfaction instrument for low literate and Spanish-speaking populations: Comparison of three formats. Patient Educ Couns. 2008;in press. [DOI] [PubMed]