Abstract
Objective
To analyze the relationship between satisfaction and technical quality of care for common mental disorders.
Data Source
A nationally representative telephone survey of 9,585 individuals conducted in 1997–1998.
Study Design
Using multinomial logistic regression techniques we investigated the association between a five-level measure of satisfaction with the mental health care available for personal or emotional problems and two quality indicators. The first measure, appropriate technical quality, was defined as use of either appropriate counseling or psychotropic medications during the prior year for a probable depressive or anxiety disorder. The second, active treatment, indicated whether the respondent had received treatment for a psychiatric disorder in the past year. Covariates included measures of physical and mental health and sociodemographic indicators.
Principal Findings
Appropriate technical quality of care was significantly associated with higher levels of satisfaction. The strength of the association was moderate.
Conclusions
Satisfaction is associated with technical quality of care. However, profiling quality of care with satisfaction will likely require large samples and case-mix adjustment, which may be more difficult for plans or provider groups to implement than measuring technical indicators. More importantly, satisfaction is not the same as technical quality, and our results suggest that at this time they cannot be made to approach each other closely enough to eliminate either.
Keywords: Quality of care, satisfaction, mental health
Measures of patient satisfaction with health care are widely used by insurers, providers, and researchers due to their intrinsic value as measures of consumer preference and their relative ease of measurement. Such surveys may be used to evaluate health care plans and providers (Crofton, Lubalin, and Darby 1999). Satisfaction indices are also used for a variety of other purposes, including assessment of quality of health care and quality improvement (Cleary and McNeil 1988). However, given the widespread use of satisfaction surveys, surprisingly little work has been done to investigate the relationship between subjective patient satisfaction and objective measures of quality of care (Cleary and McNeil 1988). In particular, the possibility of treatment selection bias (McClellan and Newhouse 2000) in studies of the quality–satisfaction relationship has not been explored.
The definitions of quality of care and patient satisfaction have varied across past studies, and have sometimes been used interchangeably. In this paper we follow the definitions proposed by Donabedian (1980). He distinguishes three components of quality: (1) technical quality of care, (2) interpersonal quality, and (3) amenities. This study examines the first component, which he defines as “the extent to which health care meets pre-defined standards of acceptable or good care.”
While technical quality is based upon objective criteria, satisfaction is subjective (Donabedian 1980). Satisfaction reflects both the patient's subjective assessment of quality of care and expectations for it (Pascoe 1983). While satisfaction is often viewed as multidimensional (Zaslavsky et al. 2000; Harris et al. 1999), the moderate-to-high correlations found between measures of different quality dimensions (Zaslavsky et al. 2000) suggests the presence of an overarching quality domain.
Few studies have analyzed the relationship between objective quality and subjective patient satisfaction in health care or mental health care. Meredith et al. (2001) found a process measure of technical quality of care to be associated with patient satisfaction in mental health care for depression. In a study that investigated whether multiple administrative outcomes of inpatient mental health treatment correlated with several measures of satisfaction, the results were equivocal, with some of the satisfaction–administrative outcome pairs significantly correlated, while the majority were not (Druss, Rosenheck, and Stolar 1999).
There are several potential methodological pitfalls in using satisfaction as a marker of technical quality, the most problematic being possible selection bias. This bias occurs when greater morbidity is associated with higher quality health care and lower levels of satisfaction with health care, and morbidity is not adequately controlled (McClellan and Newhouse 2000). The conditions necessary for selection bias are likely to be the norm. Individuals with greater morbidity are more likely to be in treatment and, among those in treatment, more likely to receive a sufficient level of services (Regier et al. 1993; Kessler et al. 1997; Wang, Berglund, and Kessler 2000). This is important because, in community samples, not receiving appropriate care usually results from failure to obtain any care or sufficient care. Further, individuals with greater morbidity are less likely to be satisfied with health care (Hoff et al. 1999; Holcomb et al. 1998; Hermann, Ettner, and Dorwart 1998). Due to data limitations, sample size, or for the sake of convenience, investigators or administrators might not adequately adjust for case mix. The effect of the selection bias would be underestimates of the magnitude of the technical quality–satisfaction relationship.
The goals of this study were to validate the technical quality–satisfaction relationship and investigate the possible effects of selection bias on this relationship among individuals with anxiety or depressive disorders, and in a more psychiatrically heterogeneous group, those with any alcohol, drug, or mental (ADM) disorder. We hypothesized that technical quality of care would be positively associated with satisfaction, but that the relationship might be obscured in unadjusted models by selection bias. We further hypothesized that the selection bias would be greater among the group of individuals with any ADM disorder, due to the greater degree of psychiatric heterogeneity.
Methods
Sample
The sample is drawn from HealthCare for Communities (HCC), which is part of the Robert Wood Johnson Foundation's Health Tracking Initiative. Data are from the HCC phone survey, which was conducted during 1997–1998. The design of HCC is described in detail elsewhere (Sturm et al. 1999). The HCC respondents were a stratified probability sample of participants in the Community Tracking Study (CTS), a nationally representative study of the U.S. civilian, noninstitutionalized population during 1996–1997 (Kemper et al. 1996). The CTS includes a sample clustered within 60 randomly selected U.S. communities and a national sample. To improve power for analyses, HCC oversampled the following CTS respondent groups: the national sample, individuals with family income less than $20,000, individuals with high psychological distress on a scale consisting of two items from the 12-item Short Form Health Questionnaire (SF-12) (Ware, Kosinski, and Keller 1996) and individuals with any mental health specialty use during the prior year. Of the 14,985 respondents selected for HCC, 9,585 were reinterviewed for a response rate of 64 percent. These analyses focus on the sample of individuals with any anxiety or depressive disorder (n=1,290) and any alcohol, drug, or mental (ADM) disorder (n=1,943). In analyses, we weight the data to be representative of the population of the United States by using CTS data to adjust for the probability of selection, unit nonresponse, and nontelephone households. The sociodemographic profile of our sample is shown in Table 1.
Table 1.
Sample Characteristics
| Sample with Anxiety/Depressive Disorder (n=1,290) | Sample with Any ADM Disorder (n=1,943) | |||
|---|---|---|---|---|
| Percent | Standard Error | Percent | Standard Error | |
| Satisfaction | ||||
| Very satisfied | 21.2 | 1.4 | 21.4 | 1.2 |
| Satisfied | 46.5 | 2.0 | 48.9 | 1.5 |
| Neutral | 14.7 | 1.3 | 15.3 | 1.1 |
| Dissatisfied | 12.2 | 1.3 | 10.4 | 1.0 |
| Very dissatisfied | 5.3 | 1.1 | 4.0 | 0.7 |
| Technical Quality | ||||
| Not Appropriate | 67.0 | 1.7 | ||
| Appropriate | 33.0 | 1.7 | ||
| Active Treatment | ||||
| No | 61.7 | 1.4 | ||
| Yes | 38.3 | 1.4 | ||
| Gender | ||||
| Male | 34.5 | 1.8 | 49.4 | 1.6 |
| Female | 65.5 | 1.8 | 50.6 | 1.6 |
| Age | ||||
| 45 or younger | 40.4 | 1.8 | 36.0 | 1.4 |
| Older than 45 | 59.6 | 1.8 | 64.0 | 1.4 |
| Income | ||||
| 0–15K | 23.6 | 1.7 | 23.8 | 1.5 |
| 15–30K | 26.7 | 2.0 | 24.6 | 1.4 |
| 30–50K | 21.3 | 1.7 | 22.4 | 1.5 |
| 50–70K | 11.2 | 1.2 | 12.2 | 1.1 |
| 70–100K | 9.7 | 1.2 | 8.8 | 0.9 |
| 100+K | 7.4 | 1.0 | 8.1 | 0.9 |
| Race | ||||
| Non-Hispanic white | 72.5 | 2.0 | 69.7 | 2.0 |
| African American | 14.6 | 1.7 | 16.3 | 1.8 |
| Hispanic | 9.2 | 1.2 | 9.9 | 1.1 |
| Other | 3.7 | 0.6 | 4.1 | 0.6 |
| Employment Status | ||||
| Employed | 62.5 | 2.1 | 67.0 | 1.6 |
| Unemployed | 9.3 | 1.1 | 9.6 | 1.0 |
| Not in labor force | 28.2 | 2.0 | 23.4 | 1.5 |
| Insurance | ||||
| Uninsured | 18.1 | 1.4 | 19.1 | 1.3 |
| Insured | 81.9 | 1.4 | 80.9 | 1.3 |
| Education | ||||
| Less than HS graduate | 20.2 | 1.7 | 19.6 | 1.5 |
| HS graduate | 33.5 | 1.7 | 35.8 | 1.6 |
| Some college | 26.9 | 1.6 | 26.4 | 1.2 |
| College graduate | 19.5 | 1.7 | 18.3 | 1.3 |
Dependent Variable
Satisfaction
We used a five-level scale of overall satisfaction with the mental health care available for personal or emotional problems during the past 12 months. Because not all individuals with disorders received treatment, this variable reflects experience with treatment and respondents’ perceptions of potentially available treatment.
Independent Variables
Appropriate Technical Quality of Care
In our analyses of the satisfaction–technical quality relationship among adults with a 12-month anxiety or depressive disorder, we used a previously described indicator of technical quality (Young et al. 2001). Briefly, this indicator was developed for a hierarchy of disorders and was defined as use of either appropriate counseling or psychotropic medications during the prior year for a probable depressive or anxiety disorder. Appropriate counseling was defined as four or more visits with a mental health specialist or primary care provider that included counseling for mental health problems. Appropriate pharmacotherapy was based upon AHCPR and other published guidelines, and included parameters for the type of medication, dosage, and duration of treatment.
Active Treatment
The above measure of appropriate technical quality approximates quality of care for individuals with depressive or anxiety disorders. However, it would be difficult to fit a similar measure of technical quality across a broad range of psychiatric disorders. Therefore, to study the satisfaction–technical quality relationship among adults with any 12-month ADM disorder we developed a broader indicator of technical quality, active treatment. This variable measures active treatment, beyond assessment, for an ADM disorder in the past year including: use of inpatient, day treatment, or residential care; use of prescribed psychotropic medications daily for a month or more; or a period of potentially therapeutic outpatient treatment for ADM conditions, such as four or more outpatient visits or visits to a provider trained in counseling methods, improving skills in relationships or coping with loss, teaching ways to relax, encouraging enjoyable activities or taking responsibility for substance abuse problems, or teaching how to avoid recurrences. While the use of a broader definition of quality could potentially decrease the strength of the association between satisfaction and quality, it would lead to greater statistical power and more precise estimates.
Psychiatric Morbidity
Psychiatric morbidity covariates included measures of probable 12-month mental health disorders; major depression, dysthymia, generalized anxiety disorder, and panic disorder were assessed (Kessler et al. 1998) using short-form versions of the World Health Organization's Composite International Diagnostic Interview (CIDI) (World Health Organization 1995). Lifetime mania corresponded to a positive score on the CIDI stem, and psychosis corresponded to a report of ever having had an overnight hospital stay for psychotic symptoms or having received a diagnosis of schizophrenia from a physician (Sturm et al. 1999). Probable 12-month substance use/abuse disorders were assessed using the AUDIT (World Health Organization 1992). Two mental health composites were included in our analyses. The MCS12 uses items from the Short Form Health Questionnaire (SF-12) (Ware, Kosinski, and Keller 1996); the MHI-5 contains additional items on the respondent's mental health status in the past four weeks.
Physical Health
Indicator variables were included for one, two, or more chronic conditions, as well as the PCS12, an aggregate measure of physical health functioning from the 12-item Short Form Health Questionnaire (SF-12) (Ware, Kosinski, and Keller 1996).
Sociodemographics
Other covariates include: gender, income (0–15k, 15–30k, 30–50k, 50–70k, 70–100k, and greater than 100k), number in household, insurance status, race (non-Hispanic White, African American, Hispanic, other), employment status (employed, unemployed, not working and not looking for employment), age (between 18 and 45, greater than 45), education (not high school graduate, high school graduate, some college, college graduate), and marital status (married or living with partner, single).
Analyses
Our first set of models used the subsample of individuals with a 12-month anxiety or depressive disorder, excluding individuals with lifetime mania or psychosis. We first tested the association between technical quality of care and satisfaction in a bivariate model. We then investigated the possibility of selection bias by adding the covariates previously described to the model. Selection bias is suggested by both a large difference in the magnitude of technical quality coefficient across the two regression equations and statistical significance of the additional covariates. In our second set of models we performed a parallel set of analyses using the sample of individuals with a 12-month anxiety, depressive, or substance disorder, or lifetime mania or psychosis to investigate the relationship between satisfaction and active treatment.
We used ordered multinomial logistic regression techniques (Hosmer and Lemeshow 2000). Similar to logistic regression, statistical significance occurs when the odds ratio (OR) differs statistically from 1.00. However, in multinomial logistic regression the OR represents the increased (OR>1) or decreased odds (OR<1) of reporting a higher level of satisfaction (e.g., satisfied versus very satisfied) that is associated with a one-point increase in the independent variable. For example, for dichotomous variables (e.g., female=0 or 1), the OR represents the odds of a woman reporting a higher or lower level of satisfaction, relative to a man. When the range of values for the independent variable is larger, as with the physical and mental health components of the SF-12 (approximate range 15–66) and MHI-5 (range 0–100), the effect of a one-unit change and the magnitude that the OR deviates from 1.00 is relatively smaller, although statistical significance is not affected by the scale of the metric. Estimates were calculated using SUDAAN software, version 8.0.0, and take into account the complex survey design and survey clustering (SUDAAN 2001). The degrees of freedom in the denominator of our F-tests differ for covariates due to our variance corrections for imputed data.
Results
A large majority of respondents with an anxiety or depressive disorder were either satisfied or very satisfied with the care available to them for mental health problems (Table 1). In a bivariate model among individuals with an anxiety or depressive disorder, technical quality of care was a significant predictor of satisfaction (odds ratio=1.55, t=2.11, p=0.04) (Table 2). The OR in the full model was slightly increased and more highly significant (OR=1.64, t=2.51, p=0.01). Measures of psychiatric morbidity (F-stat (8, 323)=2.08, p=0.04) and employment status (F-stat (2, 329)=3.63, p=0.03) were also significant in the full model. Other covariates including gender (t-stat=0.70, p=0.48), age (t-stat=1.28, p=0.20), income (F-stat (5, 227)=1.77, p=0.12), number in household (t-stat=0.66, p=0.51), race (F-stat (3, 328)=1.69, p=0.17), education (F-stat (3, 328)=0.33, p=0.81), insurance (t-stat=0.99, p=0.32) and physical health (F-stat (3, 328)=0.04, p=0.99) were not significant.
Table 2.
The Relationship between Appropriate Quality of Care and Patient Satisfaction for Respondents with a 12-Month Depression or Anxiety Disorder (n=1,290)
| Unadjusted Model | Adjusted Model | ||
|---|---|---|---|
| OR (95% CI) | OR | (95% CI) | |
| Technical Quality | |||
| Appropriate | 1.55 (1.03, 2.34)* | 1.64 | (1.11, 2.41)* |
| Not Appropriate | 1.00 (—,—) | 1.00 | (—,—) |
| Gender | |||
| Male | 1.00 | (—,—) | |
| Female | 1.12 | (0.82, 1.52) | |
| Age | |||
| 45 or younger | 0.73 | (0.46, 1.18) | |
| Older than 45 | 1.00 | (—,—) | |
| Income | |||
| 0–15K | 1.02 | (0.46, 2.31) | |
| 15–30K | 1.11 | (0.59, 2.06) | |
| 30–50K | 1.22 | (0.69, 2.15) | |
| 50–70K | 1.35 | (0.80, 2.28) | |
| 70–100K | 1.73 | (0.85, 3.52) | |
| 100+K | 1.00 | (—,—) | |
| No. of people in household | 0.97 | (0.88, 1.07) | |
| Race | |||
| African American | 1.12 | (0.72, 1.74) | |
| Hispanic | 0.80 | (0.47, 1.37) | |
| Other | 0.69 | (0.45, 1.04) | |
| Caucasian | 1.00 | (—,—) | |
| Employment Status | |||
| Employed | 0.62 | (0.40, 0.97)* | |
| Unemployed | 0.51 | (0.29, 0.92) | |
| Not in labor force | 1.00 | (—,—) | |
| Insurance | |||
| Insured | 1.42 | (0.71, 2.86) | |
| Uninsured | 1.00 | (—,—) | |
| Education | |||
| Less than HS graduate | 1.06 | (0.54, 2.08) | |
| HS graduate | 0.98 | (0.48, 2.00) | |
| Some college | 0.85 | (0.47, 1.54) | |
| College graduate | 1.00 | (—,—) | |
| Physical Health | |||
| Chronic condition | |||
| 0 | 1.00 | (—,—) | |
| 1 | 1.04 | (0.68, 1.59) | |
| 2+ | 1.05 | (0.71, 1.55) | |
| SF-12 physical score | 1.00 | (0.97, 1.03) | |
| Mental Health Measures | |||
| SF-12 mental health score | 1.00 | (0.96, 1.03)* | |
| MHI-5 | 1.01 | (1.00, 1.02) | |
| Problem with alcohol | 0.73 | (0.49, 1.09) | |
| Problem with drugs | 0.74 | (0.33, 1.63) | |
| Major depression | 0.91 | (0.62, 1.32) | |
| Dysthymia | 0.87 | (0.60, 1.26) | |
| Panic | 0.85 | (0.57, 1.26) | |
| Generalized anxiety | 0.93 | (0.66, 1.31) | |
=variables signficant at 0.05 level as a group
A broader definition of quality, active treatment, was also significantly associated with satisfaction (OR=1.40, t=2.60, p=0.01) among respondents with any ADM disorder (Table 3). The OR was slightly increased in the adjusted model (OR=1.56, t=3.04, p=0.003). In this model, income (F-stat (5,200)=5.54, p<0.001), and psychiatric morbidity (F-stat (10, 488)=2.75, p=0.003) were also significantly associated with satisfaction, while gender (t-stat=0.94, p=0.35), age (t=1.96, p=.0504), number in household (t=0.06, p=0.95), employment status (F-stat (2, 496)=2.96, p=0.053), race (F-stat (3, 495)=2.21, p=0.09), education (F-stat (3, 495)=1.88, p=0.13), insurance status (t=1.53, p=0.13), and physical health status (F-stat (3, 495)=0.17, p=0.91) were not significant.
Table 3.
The Relationship between Active ADM Treatment and Patient Satisfaction for Respondents with Any 12-Month Disorder (n=1,943)
| Unadjusted Model | Adjusted Model | ||
|---|---|---|---|
| OR (95% CI) | OR | (95% CI) | |
| Active Treatment | |||
| Yes | 1.40 (1.09, 1.80)** | 1.56 | (1.17, 2.07)** |
| No | 1.00 (—,—) | 1.00 | (—,—) |
| Gender | |||
| Male | 1.00 | (—,—) | |
| Female | 1.13 | (0.88, 1.45) | |
| Age | |||
| 45 or younger | 0.68 | (0.47, 1.00) | |
| Older than 45 | 1.00 | (—,—) | |
| Income | |||
| 0–15K | 1.07 | (0.61, 1.87)*** | |
| 15–30K | 1.18 | (0.73, 1.90) | |
| 30–50K | 1.29 | (0.77, 2.17) | |
| 50–70K | 1.89 | (1.19, 3.02) | |
| 70–100K | 1.92 | (1.13, 3.26) | |
| 100+K | 1.00 | (—,—) | |
| No. of people in household | 1.00 | (0.92, 1.09) | |
| Race | |||
| African American | 1.09 | (0.78, 1.51) | |
| Hispanic | 0.66 | (0.44, 0.99) | |
| Other | 0.82 | (0.57, 1.17) | |
| Caucasian | 1.00 | (—,—) | |
| Employment Status | |||
| Employed | 0.62 | (0.42, 0.92) | |
| Unemployed | 0.77 | (0.46, 1.29) | |
| Not in labor force | 1.00 | (—,—) | |
| Insurance | |||
| Insured | 1.41 | (0.91, 2.19) | |
| Uninsured | 1.00 | (—,—) | |
| Education | |||
| Less than HS graduate | 1.24 | (0.75, 2.05) | |
| HS graduate | 0.98 | (0.60, 1.60) | |
| Some college | 0.81 | (0.52, 1.24) | |
| College graduate | 1.00 | (—,—) | |
| Physical Health | |||
| Chronic Condition | |||
| 0 | 1.00 | (—,—) | |
| 1 | 0.90 | (0.65, 1.25) | |
| 2+ | 0.98 | (0.71, 1.35) | |
| SF-12 physical score | 1.00 | (0.98, 1.03) | |
| Mental Health Measures | |||
| SF-12 mental health score | 1.00 | (0.98, 1.03)** | |
| MHI-5 | 1.01 | (1.00, 1.02) | |
| Problem with alcohol | 0.89 | (0.66, 1.21) | |
| Problem with drugs | 0.82 | (0.56, 1.21) | |
| Major depression | 0.75 | (0.59, 0.95) | |
| Dysthymia | 0.79 | (0.54, 1.15) | |
| Panic | 0.82 | (0.57, 1.18) | |
| Generalized anxiety | 0.93 | (0.65, 1.33) | |
| Psychosis | 1.38 | (0.81, 2.34) | |
| Mania | 1.03 | (0.76, 1.38) | |
=variables significant at 0.05 level, as a group
=variables significant at 0.01 level, as a group
=variables significant at 0.001 level, as a group
Discussion
Satisfaction measures have their own intrinsic merit as measures of consumer preference. Satisfaction is also theoretically a relatively easy way to obtain inexpensive markers of technical quality of care. However, it has been suggested patients have difficulty distinguishing technically appropriate care (Welton and Parker 1999; Pascoe 1983), and concerns have been raised about using satisfaction as a marker for technical quality of care (Cleary and McNeil 1988).
We have shown that satisfaction with available mental health services is significantly associated with two measures of appropriate care. Further, the association between satisfaction and quality was more robust to model specification than we anticipated. The strength of the association between satisfaction and appropriate technical quality of care was comparable in a bivariate model and in a model that included measures of mental and physical health and sociodemographic status. The broader definition of quality, active treatment, was also significantly associated with satisfaction in both unadjusted and adjusted models, although the strength of the relationship was slightly stronger in the adjusted model. Thus, although our measures of psychiatric health were significantly associated with satisfaction, their exclusion from the model did not lead to substantive changes in the quality coefficients. However, we believe it would be premature to state that case-mix adjustment is not needed in these types of analyses. Indeed, defining the relevant groups for analyses is a type of case-mix adjustment. Further study with other samples and other groups of disorders is necessary to clarify the utility of case-mix adjustment.
While the strength of the satisfaction–technical quality association was only moderate, our estimates might be larger with more perfect measures of technical quality or mental health status. Our case identification relied on a brief diagnostic screening instrument, which has been found to be highly specific but not very sensitive (Murphy et al. 2000), and due to the constraints of population surveys we were not able to control for all potentially relevant mental health disorders. Our measure of appropriate counseling, at least five minutes of counseling in four different sessions, is unsophisticated and contains no information on session content or process. While the definition for appropriate counseling has been shown to affect outcomes for depressive disorders in the Medical Outcomes Study, it has not been studied for anxiety disorders (Sturm and Wells 1995). We assessed technical quality through self-report of use of specific services; however, Katon et al. (1996) found moderate associations between self-reports of medication use and administrative data in an HMO (Katon et al. 1996). Due to these limitations, we feel that our estimate of the strength of the satisfaction–technical quality relationship should be viewed as a theoretical lower bound, although in practice it will be difficult for any health plan or provider to implement more sophisticated methodologies.
One other data limitation deserves comment. While satisfaction with care is multidimensional, our study used only a one-item global measure of satisfaction. However, this is likely to mirror “real world” applications of satisfaction measures where the mental health items in any general survey of satisfaction with health care will be limited. For example, the core CAHPS questionnaire has no mental health care items, and only three items in the optional supplement.
Our study demonstrates that technical quality of care correlates with satisfaction, but what does this mean? One explanation for our findings is that individuals are able to distinguish quality care for common mental disorders, but there are other plausible explanations (Orlando and Meredith 2002). For example, providers might be more apt to give quality care to individuals who are more easily satisfied, or clinicians who provide appropriate technical quality of care might also have better interpersonal skills, leading to greater patient satisfaction. Therefore, to better understand the causes of the quality–satisfaction correlation will require a more detailed investigation of the complex clinician–patient dynamic.
If the strength of the satisfaction–technical quality association using currently feasible techniques is only moderate, attempts to differentiate technical quality among health plans or provider groups using satisfaction as a proxy will likely require a large number of observations or large differences in satisfaction between plans. The robust relationship between satisfaction and quality is encouraging. However, our findings may not generalize to populations that were not well represented in this study, such as individuals with schizophrenia or bipolar disorder. Case-mix adjustment will still be necessary to define the relevant groups. Because estimation of technical quality using satisfaction as a proxy measure will likely require large samples and case-mix adjustment, it may be more difficult for plans or provider groups to implement than directly using indicators of quality. More importantly, satisfaction is not the same as technical quality, and our results suggest that at this time they cannot be made to approach each other closely enough to eliminate either.
Footnotes
Funded by the Robert Wood Johnson Foundation—HealthCare for Communities and the UCLA-RAND NIMH Research Center on Managed Care for Psychiatric Disorders (MH 54623).
References
- Cleary PD, McNeil BJ. “Patient Satisfaction as an Indicator of Quality Care”. Inquiry. 1988;25(1):25–36. [PubMed] [Google Scholar]
- Crofton C, Lubalin JS, Darby C. “Consumer Assessment of Health Plans Study (CAHPS). Foreword”. Medical Care. 1999;37(3, supplement):MS1–9. doi: 10.1097/00005650-199903001-00001. [DOI] [PubMed] [Google Scholar]
- Donabedian A. The Definition of Quality and Approaches to Its Assessment. Ann Arbor, MI: Health Administration Press; 1980. [Google Scholar]
- Druss BG, Rosenheck RA, Stolar M. “Patient Satisfaction and Administrative Measures as Indicators of the Quality of Mental Health Care”. Psychiatric Services. 1999;50(8):1053–8. doi: 10.1176/ps.50.8.1053. [DOI] [PubMed] [Google Scholar]
- Harris LE, Swindle RW, Mungai SM, Weinberger M, Tierney WM. “Measuring Patient Satisfaction for Quality Improvement”. Medical Care. 1999;37(12):1207–13. doi: 10.1097/00005650-199912000-00004. [DOI] [PubMed] [Google Scholar]
- Hermann RC, Ettner SL, Dorwart RA. “The Influence of Psychiatric Disorders on Patients’ Ratings of Satisfaction with Health Care”. Medical Care. 1998;36:720–7. doi: 10.1097/00005650-199805000-00011. [DOI] [PubMed] [Google Scholar]
- Hoff RA, Rosenheck RA, Meterko M, Wilson NJ. “Mental Illness as a Predictor of Satisfaction with Inpatient Care at Veteran Affairs Hospitals”. Psychiatric Services. 1999;50(5):680–5. doi: 10.1176/ps.50.5.680. [DOI] [PubMed] [Google Scholar]
- Holcomb WR, Parker JC, Leong GB, Thiele J, Higdon J. “Customer Satisfaction and Self-Reported Treatment Outcomes among Psychiatric Patients”. Psychiatric Services. 1998;49(7):929–34. doi: 10.1176/ps.49.7.929. [DOI] [PubMed] [Google Scholar]
- Hosmer DW, Lemeshow S. Applied Logistic Regression. New York: Wiley; 2000. [Google Scholar]
- Katon W, Robinson P, Von Korff M, Lin E, Bush T, Ludman E, Simon G, Walker E. “A Multifaceted Intervention to Improve Treatment of Depression in Primary Care”. Archives of General Psychiatry. 1996;53(10):924–32. doi: 10.1001/archpsyc.1996.01830100072009. [DOI] [PubMed] [Google Scholar]
- Kessler RC, Andrews G, Mroczek D, Ustun B, Wittchen HU. “The World Health Organization Composite International Diagnostic Interview Short-Form (CIDI-SF)”. International Journal of Methods in Psychiatric Research. 1998;7:171–85. [Google Scholar]
- Kessler RC, Frank RG, Edlund M, Katz SJ, Lin E, Leaf P. “Differences in the Use of Psychiatric Outpatient Services between the United States and Ontario”. New England Journal of Medicine. 1997;336(8):551–7. doi: 10.1056/NEJM199702203360806. [DOI] [PubMed] [Google Scholar]
- Kemper P, Blumenthal D, Corrigan JM, Cunninghan PJ, Felt SM, Grossman JM, Kohn LT, Metcalf CE, St Peter RF, Strouse RC, Ginsburg PB. “The Design of the Community Tracking Study: A Longitudinal Study of Health System Change and Its Effects on People”. Inquiry. 1996;33(2):195–206. [PubMed] [Google Scholar]
- McClellan MB, Newhouse JP. “Overview of the Special Supplemental Issue”. Health Services Research. 2000;35(5, part 2):1061–9. [PMC free article] [PubMed] [Google Scholar]
- Meredith LS, Orlando M, Humphrey N, Camp P, Sherbourne CD. “Are Better Ratings of the Patient–Provider Relationship Associated with Higher Quality Care for Depression?”. Medical Care. 2001;39(4):349–60. doi: 10.1097/00005650-200104000-00006. [DOI] [PubMed] [Google Scholar]
- Murphy JM, Monson RR, Laird NM, Sobol AM, Leighton AH. “A Comparison of Diagnostic Interviews for Depression in the Stirling County Study: Challenges for Psychiatric Epidemiology”. Archives of General Psychiatry. 2000;57(3):230–6. doi: 10.1001/archpsyc.57.3.230. [DOI] [PubMed] [Google Scholar]
- Narrow WE, Regier DA, Rae DS, Manderscheid RW, Locke BZ. “Use of Services by Persons with Mental and Addictive Disorders. Findings from the National Institute of Mental Health Epidemiologic Catchment Area Program”. Archives of General Psychiatry. 1993;50(2):95–107. doi: 10.1001/archpsyc.1993.01820140017002. [DOI] [PubMed] [Google Scholar]
- Orlando M, Meredith LS. “Understanding the Causal Relationship between Patient-Reported Interpersonal and Technical Quality of Care for Depression”. Medical Care. 2002;40(8):696–704. doi: 10.1097/00005650-200208000-00009. [DOI] [PubMed] [Google Scholar]
- Pascoe GC. “Patient Satisfaction in Primary Care: A Literature Review and Analysis”. Evaluation and Program Planning. 1983;6(3–4):185–210. doi: 10.1016/0149-7189(83)90002-2. [DOI] [PubMed] [Google Scholar]
- Regier DA, Narrow WE, Rae DS, Manderscheid RW, Locke BZ, Goodwin FK. “The De Facto U.S. Mental and Addictive Disorders Service System. Epidemiologic Catchment Area Prospective 1-Year Prevalence Rates of Disorders and Services”. Archives of General Psychiatry. 1993;50(2):85–94. doi: 10.1001/archpsyc.1993.01820140007001. [DOI] [PubMed] [Google Scholar]
- Sturm R, Gresenz C, Sherbourne C, Minnium K, Klap R, Bhattacharya J, Farley D, Young AS, Burnam MA, Wells KB. “The Design of HealthCare for Communities: A Study of Health Care Delivery for Alcohol, Drug Abuse, and Mental Health Conditions”. Inquiry. 1999;36(2):221–33. [PubMed] [Google Scholar]
- Sturm R, Wells KB. “How Can Care for Depression become More Cost-effective?”. Journal of the American Medical Association. 1995;273(1):51–8. [PubMed] [Google Scholar]
- SUDAAN. SUDAAN software for the Statistical Analysis of Correlated Data. Cary, NC: Research Triangle Institute; 2001. version 8.0. [Google Scholar]
- Wang PS, Berglund P, Kessler RC. “Recent Care of Common Mental Disorders in the United States: Prevalence and Conformance with Evidence-Based Recommendations”. Journal of General Internal Medicine. 2000;15(5):284–92. doi: 10.1046/j.1525-1497.2000.9908044.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ware J, Jr, Kosinski M, Keller SD. “A 12-Item Short-Form Health Survey: Construction of Scales and Preliminary Tests of Reliability and Validity”. Medical Care. 1996;34(3):220–33. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
- Welton R, Parker R. “Study of the Relationships of Physical and Mental Health to Patient Satisfaction”. Journal for Healthcare Quality. 1999;21(6):39–46. doi: 10.1111/j.1945-1474.1999.tb01003.x. [DOI] [PubMed] [Google Scholar]
- World Health Organization. The Alcohol Use Disorders Identification Test (AUDIT). Guidelines for Use in Primary Health Care. Geneva: World Health Organization; 1992. [Google Scholar]
- World Health Organization. Composite International Diagnostic Interview (Version 2.1) Geneva: World Health Organization; 1995. [Google Scholar]
- Young AS, Klap R, Sherbourne CD, Wells KB. “The Quality of Care for Depressive and Anxiety Disorders in United States”. Archives of General Psychiatry. 2001;58(1):55–61. doi: 10.1001/archpsyc.58.1.55. [DOI] [PubMed] [Google Scholar]
- Zaslavsky AM, Beaulieu ND, Landon BE, Cleary PD. “Dimensions of Consumer-Assessed Quality of Medicare Managed-Care Health Plans”. Medical Care. 2000;38(2):162–74. doi: 10.1097/00005650-200002000-00006. [DOI] [PubMed] [Google Scholar]