Abstract
Objective
To determine what aspects of patient satisfaction are most important in explaining the variance in patients' overall treatment experience and to evaluate the relationship between treatment experience and subsequent outcomes.
Data Sources and Setting
Data from a population-based survey of 804 randomly selected injured workers in Washington State filing a workers' compensation claim between November 1999 and February 2000 were combined with insurance claims data indicating whether survey respondents were receiving disability compensation payments for being out of work at 6 or 12 months after claim filing.
Study Design
We conducted a two-step analysis. In the first step, we tested a multiple linear regression model to assess the relationship of satisfaction measures to patients' overall treatment experience. In the second step, we used logistic regression to assess the relationship of treatment experience to subsequent outcomes.
Principal Findings
Among injured workers who had ongoing follow-up care after their initial treatment (n=681), satisfaction with interpersonal and technical aspects of care and with care coordination was strongly and positively associated with overall treatment experience (p<0.001). As a group, the satisfaction measures explained 38 percent of the variance in treatment experience after controlling for demographics, satisfaction with medical care prior to injury, job satisfaction, type of injury, and provider type. Injured workers who reported less-favorable treatment experience were 3.54 times as likely (95 percent confidence interval, 1.20–10.95, p=.021) to be receiving time-loss compensation for inability to work due to injury 6 or 12 months after filing a claim, compared to patients whose treatment experience was more positive.
Keywords: Patient satisfaction, quality improvement, treatment outcomes, workers' compensation
In recent years, increased emphasis has been placed upon improving the quality of health care in the United States in response to evidence indicating that quality for too many patients is not what it should or could be (Schuster, McGlynn, and Brook 1998; Chassin and Galvin 1998; Committee on the Quality of Health Care in America 2001; Kohn, Corrigan,and Donaldson 1999). A recent detailed study by McGlynn et al. (2003) identified serious widespread deficiencies in health care quality, raising further concern. Corresponding to this emphasis on quality, there has been increased interest in analyzing and monitoring patient satisfaction for purposes of quality improvement. Routine monitoring of patient satisfaction is now performed by state and federal agencies (Centers for Medicare and Medicaid Services), by accrediting bodies such as the National Committee for Quality Assurance, and by private and public health care purchasers. Myriad measures and survey instruments have been developed to assess patient satisfaction (McCracken et al. 1997; Harris et al. 1999; Stump et al. 1995). Reflecting diverse purposes, studies have examined patient satisfaction at the micro (office encounter) level using instruments such as the Patient Satisfaction Questionnaire (PSQ) (Ware et al. 1983), as well as at the macro health plan level using the Consumer Assessments of Health Plans Study (CAHPS) survey (Hays et al. 1999) and other similar instruments.
Although the importance of assessing patient satisfaction from a “customer service” perspective is now recognized, debate continues regarding the empiric basis for this activity and its importance from a clinical viewpoint (Williams 1994; Williams, Coyle, and Healy 1998; Sitzia and Wood 1997; Weaver et al. 1997). Surveys that gather satisfaction data based on a single office encounter cannot provide information on care coordination and related care processes that are known to be the source of quality problems (Schuster, McGlynn, and Brook 1998; Committee on the Quality of Health Care in America 2001). Surveys that gather data at the health plan level, such as CAHPS, while useful for monitoring consumer satisfaction, have limited value for quality improvement because of the generality of the data collected. For meaningful quality improvement purposes, population-specific assessments of satisfaction that gather information related to targeted care processes should be validated against important empirically derived outcomes. In injured worker populations, one's ability to return to productive work represents such an outcome.
The concept of an episode of care (Hornbrook, Hurtado, and Johnson 1985; Wingert et al. 1995; DeVet et al. 2002) provides a useful framework for assessing patient satisfaction. Many patients see multiple health care providers in different treatment settings in the course of having a condition diagnosed and treated. We conducted a patient satisfaction survey, organized around an episode of care framework, as part of a larger ongoing community-based initiative designed to improve quality and health outcomes for injured workers in Washington State (Wickizer et al. 2001, 2002,2004). The purpose of this survey was to identify sources of dissatisfaction that might provide insight into processes of care in need of improvement. Started in 2002, the quality improvement initiative is being pilot tested in two sites serving target areas in western and eastern Washington. The initiative will conclude in 2006.
In Washington State, the Department of Labor and Industries (DLI) administers the state workers' compensation system and is the sole regulator of workers' compensation health care for all covered nonfederal employees. We combined the survey data with DLI administrative claims data pertaining to disability status and then performed a two-step analysis. First, we analyzed the survey data to determine which aspects of patient satisfaction were most important in explaining the variance in patients' self-reported overall treatment experience. Second, we examined the relationship of patients' treatment experience to subsequent disability outcomes. We hypothesized that patients reporting more negative treatment experience would be more likely to be out of work and on workers' compensation disability at 6 and 12 months following the filing of their claim.
Methods
Washington State Workers' Compensation System
Workers who sustain occupational injuries or develop occupational illnesses are eligible to receive medical care through the workers' compensation system and may also receive disability payments (partial wage replacement) for injuries or illnesses resulting in lost work time (Carr 1998; 1989). Workers' compensation insurance is administered at the state level and provides first-dollar coverage (no deductibles, coinsurance, or copayments) for medical care. Workers' compensation programs vary considerably from state to state in their statutory authority, organization, and ability to direct patients to certain forms of care. Washington is a “worker choice” state. By law, injured workers have the right to choose any licensed physician, chiropractor, osteopath, podiatrist, dentist, optometrist, or naturopath for their care. Further, Washington State law requires employers to purchase workers' compensation insurance through a designated public state agency, although employers who meet specific standards can self-insure. Approximately two-thirds of the state's nonfederal workforce are insured through the state fund and a third of the workers are employed at firms that are self-insured. In fiscal year 2000, the DLI expended $472.4 million for medical care and an additional $683.3 million for temporary and permanent disability payments. These features of Washington's workers' compensation program—a uniform set of benefits covering two-thirds of the state's worker population and the guarantee of broad access to medical care services unrestricted by complex managed care arrangements—make it well suited for the current study.
Study Framework
The study's design framework is shown in Figure 1. The episode of care begins with an injury event and the filing of a claim, and may involve a series of sequential diagnostic and treatment activities. For purposes of this study, the episode of care ended with the interview occasion. We attempted to interview injured workers between four and five months after the injury claim was filed. The actual mean (and median) time between claim filing and interview date was 159 days (SD=34 days, range=79 to 260 days). We ultimately interviewed 804 workers to gather patient satisfaction information on the types of care processes shown in Figure 1. The majority (57 percent) of the injured workers, except for workers with carpal tunnel syndrome, initiated care the same day they were injured, and an additional 31 percent received care within one to three days. Workers with carpal tunnel syndrome, considered an occupational illness rather than an injury, usually took longer to initiate care. The majority (55 percent) of these workers took a month or more to initiate care for their condition after becoming bothered by the symptoms.
As shown in Figure 1, our study involved two sequential analytic steps. Step 1 involved examining the relationships between satisfaction measures related to the processes of care incorporated in Figure 1 and overall self-reported treatment experience. Step 2 involved analyzing the relationship of treatment experience to subsequent treatment outcomes on a subset of 385 patients who met certain inclusion criteria described below. The study's procedures and measures were approved by the University of Washington Institutional Review Board.
Sampling
From the DLI claims databases, we identified claims that were submitted between November 1, 1999, and February 28, 2000. We extracted claims by Z-16 injury codes representing three major conditions commonly treated through workers' compensation: carpal tunnel syndrome, low back pain, and upper and lower extremity fractures. Using a stratified random sampling procedure, we selected approximately 400 cases within each injury type for inclusion in the survey, as well as 400 additional cases representing a random sample of workers with other injuries, for a total of 1,600 claimants. We selected cases within three strata (low back pain, upper and lower extremity fractures, and other injuries) so that the final survey sample consisted of two time-loss cases (patients who missed at least four days of work due to their injury and were thus eligible to receive disability, or wage replacement, payments) for every non-time-loss case. It was not feasible to do this for carpal tunnel syndrome cases because these cases often take longer to receive authorization as a workers' compensation claim. We oversampled time-loss cases to ensure that the majority of respondents would have some ongoing treatment for their injuries or illnesses beyond an initial visit.
Satisfaction and Outcome Measures
The telephone interview survey averaged 35 minutes and included 83 questions about patient satisfaction, health care utilization, treatment experience, and recovery status. In addition, the survey included items specifically related to workers' compensation (e.g., knowledge of the workers' compensation system, whether the worker had any job modification to accommodate his or her injury, work status, and earnings after injury). For this study, we analyzed a selected set of survey items closely related to the elements depicted in Figure 1. We borrowed measures from satisfaction surveys that had been previously validated on general patient populations (Ware et al. 1983), and on workers' compensation patient populations (Kyes, Franklin, and Weaver 1997; Rudolph et al. 2002). The survey included: (1) two items pertaining to the difficulty in obtaining initial care and follow-up care (1=a lot of difficulty, 3=very little difficulty); (2) two items related to the perceived appropriateness of the timing of referral for specialty care (1=too soon after initiation of treatment, 2=not soon enough) and the quality of specialty care (1=very poor, 7=excellent care); (3) seven items pertaining to patient satisfaction with different aspects of provider care (1=very dissatisfied, 4=very satisfied); (4) one item regarding care coordination (1=very dissatisfied, 4=very satisfied); (5) one item concerning how thoroughly the health care provider understood the patient's job duties and work setting, a measure intended to assess the provider's occupational medicine expertise (1=very well, 4=not at all); (6) one item pertaining to the patient's overall treatment experience (1=very poor, 7=excellent); and (7) one item representing self-reported recovery status (1=completely recovered, 5=not recovered). In completing the interview, respondents were instructed to consider the provider from whom they received most of their care and to rate that care accordingly.
The seven provider satisfaction ratings addressed aspects of care within the domains of technical and interpersonal quality: provider ability to diagnose and treat injury, perceived thoroughness of the provider in giving treatment, provider ability to direct the patient to the most effective treatment for the injury, provider ability to listen and understand the patient's concerns, provider courtesy and respectfulness, explanation of the nature of the injury, and explanation of treatment needed for the injury. Since these ratings were correlated, it was not desirable to treat them as separate variables for purposes of analysis. Further, using scales instead of individual satisfaction items generally enhances data reliability (Ware and Hays 1988). The first three satisfaction items were averaged to create a “technical quality” index and the remaining four items were averaged to create an “interpersonal quality” index. Both summary indexes exhibited good internal consistency (technical quality index Cronbach's alpha=0.81; interpersonal quality index Cronbach's alpha=0.85). The Pearson correlation between the two summary index measures was .64 (p=.01). To make our results as relevant as possible for “best practice” health care delivery and to simplify the presentation of findings, we followed Rubin et al. (1993) and analyzed the satisfaction measures in binary form: “very satisfied” versus “less than very satisfied” or “excellent” versus “less than excellent.”
We included the following additional survey measures as covariates in the analysis: (1) general satisfaction with medical care received prior to the current injury; (2) satisfaction with current job; (3) provider type/treatment setting (worker's family physician, other primary care clinic, hospital emergency department, urgent care center, specialist physician, chiropractor, physical therapist); (4) age; (5) sex; (6) race; and (7) education.
We used DLI administrative claims data to construct an outcome measure representing workers out of work and receiving disability compensation at 6 or 12 months after claim filing. We combined cases on disability at 6 and 12 months because there were too few cases at each time to permit separate meaningful analyses. This outcome measure offered two important advantages: (1) it is highly accurate because disability claim payments are closely tracked and subject to audit; and (2) it enabled us to evaluate outcome subsequent to the time of interview, and thus to establish a temporal relationship between patient satisfaction and disability outcomes.
Survey Administration
The survey was administered via a computer-assisted telephone interview (CATI) over a five-month period beginning in April 2000. As many as 15 attempts were made to contact 1,559 of the 1,600 claimants identified (41 claimants had no telephone number). There were 804 (53 percent) that completed the interview, including 25 interviews in Spanish. There were 146 (9 percent) claimants that refused to participate and 609 (39 percent) could not be contacted. We examined whether nonrespondents differed from respondents in gender, age, type of injury, claim type (time-loss claim versus non-time-loss claim) or insurance status (self-insured versus state fund). Respondents were more likely to be female (45 percent versus 38 percent, p=.01) and to have carpal tunnel syndrome (27.2 percent versus 18.3 percent, p =.01). Respondents with fractures, back sprain, or other injuries were also more likely than were nonrespondents with these injuries to have a time-loss claim (69.9 percent versus 63.8 percent, p=.02). (As noted earlier, the sampling of time-loss claims did not include carpal tunnel syndrome cases.) There were no significant differences between respondents and nonrespondents with regard to age or proportion of self-insured cases.
Statistical Analysis
In the first step of our analysis, we used multiple linear regression to examine the association between individual satisfaction measures and overall treatment experience, measured on a seven-point scale. We entered variables in the regression model in block form in the following order: (1) demographic variables and satisfaction with prior medical care and current job, (2) type of injury, (3) treatment setting/provider variables, and (4) difficulty in obtaining care, satisfaction with care coordination, degree of health care provider understanding of the patient's job activities and work setting, and the summary technical and interpersonal quality satisfaction measures. Entering the variables in the model in block form enabled us to determine the percentage of variance in overall treatment experience explained by the satisfaction measures in contrast to the other variables. Of the 804 patients surveyed, 123 (15.2 percent) had an initial medical encounter only and no follow-up care. Sixty percent of the 681 (84.8 percent) patients receiving follow-up care reported having five or more provider visits. Because patients who received initial care only had limited contact with the treatment system and provided limited satisfaction information, we analyzed them separately from patients who had follow-up care.
We examined studentized residuals to check whether our model met the assumptions for linear regression. Although the residual plots revealed no serious departures from the assumptions, we specified two alternative regression models to test the robustness of the findings. We first collapsed the seven-point scale into three categories representing poor, moderate, and good treatment experience, and conducted ordinal regression. We then created a binary variable from the treatment experience scale (excellent or less than excellent treatment) and performed logistic regression. There was no meaningful change in the results based upon either of the two alternative models. Because the linear regression model used the full range of values contained in the seven-point treatment experience scale and provided more readily interpretable information regarding the relations between satisfaction measures and treatment experience, we retained that model for the analysis.
For the second step of the analysis, we performed multivariate logistic regression to assess the relationship of treatment experience to subsequent treatment outcome (receiving versus not receiving time-loss compensation for temporary disability 6 or 12 months after claim filing). We included in the model demographic factors and type of injury, along with two additional variables intended to control for possible confounding effects of disability status at the time of interview and recovery status. Subjects who were on disability at time of interview, or who were less recovered from their injury, might be more likely to both have a perceived negative treatment experience and be on disability later at 6 or 12 months. The two variables included in the model were: (1) self-reported recovery status (≤50 percent recovered versus 75 percent or fully recovered) and (2) on versus off disability at the time of interview. All analyses were performed using SPSS version 11.0 (SPSS Inc., Chicago, IL), and p<.05 (two-tailed) was considered statistically significant.
The outcome analysis was limited to workers who (1) were insured through the state fund (detailed data on time-loss compensation are not collected for self-insured claimants) and (2) completed the interview prior to 180 days following claim filing (to ensure that outcome was assessed after the time of interview). We excluded 268 (33.3 percent) cases based on the first criterion and 151 (18.8 percent) based on the second, leaving 385 subjects (48 percent) in the analysis. By limiting the time between claim filing and interview to a maximum of 179 days, we reduced the mean time to interview from 159 days (Figure 1) to 145 days. Nonetheless, 24 percent (92) of 385 cases were interviewed within 14 days (between 165 and 179 days) of the time the first disability outcome measure was assessed (180 days after claim filing). The results of the outcome analysis could have been influenced by the proximity of these cases to the 180-day limit. To test the sensitivity of our results to this, we restricted the time limit to 165 days and repeated the analysis. This reduced the mean time from claim filing to interview date from 145 days to 137 days.
Results
Descriptive Analyses
The majority of injured workers in the sample were white, male, between the ages of 35 and 54, and educated at least through high school (Table 1). Sixty percent of the sample received compensation for lost work time.Table 2 summarizes the satisfaction measures. Injured workers reported having little difficulty gaining access to care either for initial treatment or follow-up.
Table 1.
Measure | Number (%) | |
---|---|---|
Age | ||
18–24 | 62 (8.0) | |
25–34 | 173 (22.0) | |
35–44 | 257 (32.0) | |
45–54 | 221 (27.0) | |
55+ | 99 (11.0) | |
Gender | ||
Male | 450 (56.0) | |
Female | 354 (44.0) | |
Race | ||
White | 684 (85.0) | |
Black | 25 (3.0) | |
Asian, Pacific Islander | 26 (3.0) | |
Native American | 17 (2.0) | |
Other | 52 (7.0) | |
Education | ||
<12 years | 111 (14.0) | |
High school degree | 331 (41.0) | |
Some college or technical school | 251 (31.0) | |
College degree | 66 (8.0) | |
Postgraduate work or degree | 41 (5.0) | |
Injury | ||
Carpal tunnel syndrome | 219 (27.2) | |
Upper or lower extremity fracture | 202 (25.1) | |
Low back sprain | 197 (24.5) | |
Other injuries | 186 (23.1) | |
Form of insurance | ||
Self-Insured | 268 (33.3) | |
Insured through state fund | 536 (66.7) | |
Type of claim | ||
Time-loss claim | 482 (60.0) | |
Non-time-loss claim | 322 (40.0) |
Table 2.
Measure | Number (%) |
---|---|
Initial care obtained with little difficulty | 693 (86.2) |
Follow-up care obtained with little difficulty (n =681) | 602 (90.8) |
Timing of referral was appropriate (n =402)a | 282 (70.1) |
Specialty care rated as excellent (n =453) | 242 (53.4) |
Provider satisfaction ratings (very satisfied)b | |
Provider was able to determine what was wrong and what needed to be done to treat injury+ | 585 (72.8) |
Provider was thorough and gave careful treatment+ | 595 (74.0) |
Provider directed patient to most effective treatment for injury+ | 554 (68.9) |
Provider listened carefully and understood patient's concerns# | 619 (77.0) |
Provider was courteous and treated patient with respect# | 700 (87.1) |
Provider explained nature of injury# | 611 (76.0) |
Provider explained treatment needed for injury# | 577 (71.8) |
Technical quality index (very satisfied) | 569 (70.8) |
Interpersonal quality index (very satisfied) | 596 (74.1) |
Patient was very satisfied with how care was coordinated | 476 (59.2) |
Physician understood patient's job duties and work setting | 287 (35.7) |
Self-reported recovery status | |
Fully recovered | 226 (28.1) |
75% recovered | 293 (36.4) |
50% recovered | 125 (15.5) |
25% recovered | 50 (6.2) |
Not recovered | 101 (12.6) |
Patient rated overall treatment experience as excellent | 307 (38.2) |
On disability at 6 or 12 months after claim received (n =536) | 72 (13.4) |
aFifty-one patients self-referred to a specialist and hence did not provide information on timing of referral.
bRefers to the physician or health care provider who provided most of the care received by the patient for his or her injury.
+Item used to create technical quality index.
#Item used to create interpersonal quality index.
Seventy percent of the patients referred for specialty care rated the timing of the referral as appropriate. Most patients (approximately 70 percent) were very satisfied with their care, but less satisfied (59 percent) with how that care was coordinated. Fewer patients (36 percent) felt their health care provider had a good understanding of their job activities or work setting. Compared with the physician satisfaction measures shown in Table 2, a smaller percentage of patients (38 percent) rated their overall treatment experience as excellent. This discrepancy probably reflects differences in the measures. The treatment experience measure was based upon a seven-point scale (1=very poor, 7=excellent), whereas the physician satisfaction measures were based upon four-point scales (1=very dissatisfied, 4=very satisfied). The seven-point scale would be more discriminating. Combing the highest and next highest categories in the seven-point sale would increase the rating of “excellent” from 38 percent to 60 percent.
Approximately one-third of the sample reported being 50 percent or less recovered from their injury at time of interview, and 13.4 percent (72) of the 536 state-fund claimants were receiving time-loss/disability compensation due to their inability to work 6 or 12 months after filing a claim. Although not included in Table 2, the survey gathered data on work status at time of interview. Seventy-seven percent (622/804) of the injured workers had returned to work at time of interview. Twenty-five percent (156/622) of these workers reported being 50 percent or less recovered from their injury, and thus were probably working with some degree of ongoing symptoms.
Relationship of Satisfaction to Treatment Experience
Results of the regression analysis performed to determine relationships between the satisfaction measures and overall treatment experience are shown in Table 3. The regression coefficients (β) indicate a change in treatment experience, measured on a seven-point scale, associated with a one-unit change in the independent variable or with a difference in group status relative to a defined (omitted) reference group in the case of categorical variables.
Table 3.
Initial Care Only (n=123) | Follow-up Care (n=681) | ||||
---|---|---|---|---|---|
Satisfaction Ratings and Reports | β (SE) | P-Value | β (SE) | P-Value | |
Constant | 3.44 (0.61) | <.001 | 3.48 (0.32) | <.001 | |
Age | |||||
18–24 | −0.52 (0.49) | .29 | −0.26 (0.24) | .29 | |
25–34 | −0.20 (0.44) | .65 | 0.04 (0.18) | .82 | |
35–44 | −0.55 (0.47) | .21 | −0.03 (0.17) | .85 | |
45–54 | −1.22 (0.44) | .007 | −0.07 (0.16) | .67 | |
55 and older* | 0.00 | 0.00 | |||
Gender | |||||
Female | −0.14 (0.23) | .60 | 0.18 (.10) | .06 | |
Male* | 0.00 | 0.00 | |||
Race | |||||
Nonwhite | −0.50 (0.35) | .16 | 0.30 (0.14) | .04 | |
White* | 0.00 | 0.00 | |||
Education | |||||
Post-high school or college | −0.37 (0.27) | .18 | 0.02 (0.10) | .81 | |
High school degree or less* | 0.00 | 0.00 | |||
Satisfaction with prior care | |||||
Very satisfied | 0.88 (0.26) | .001 | 0.08 (0.10) | .41 | |
Less than very satisfied* | 0.00 | 0.00 | |||
Satisfaction with job | |||||
Very satisfied | −0.05 (0.27) | .84 | −0.12 (0.11) | .30 | |
Less than very satisfied* | 0.00 | 0.00 | |||
Type of injury | |||||
Back injuries | 0.05 (0.31) | .86 | −0.28 (0.15) | .07 | |
Carpal tunnel syndrome | −0.12 (0.40) | .76 | −0.06 (0.15) | .67 | |
Upper or lower extremity fractures | 0.28 (0.37) | .94 | 0.14 (0.14) | .33 | |
Other injuries* | 0.00 | 0.00 | |||
Provider type for initial care | |||||
Other MD/provider | 0.55 (0.32) | .09 | 0.22 (0.14) | .12 | |
Employer's medical clinic | 0.09 (0.81) | .90 | 0.19 (.22) | .39 | |
Hospital ER | 0.11 (0.37) | .77 | −0.12 (0.14) | .42 | |
Urgent care | 0.30 (0.42) | .45 | 0.09 (0.21) | .64 | |
Chiropractor | −0.32 (0.51) | .52 | −0.03 (0.31) | .92 | |
Patient's usual doctor* | 0.00 | 0.00 | |||
Provider type for follow-up care | |||||
Chiropractor | —a | — | 0.29 (.25) | .26 | |
Other primary care MD | — | — | −0.08 (0.19) | .68 | |
Specialist | — | — | −0.14 (0.16) | .38 | |
Physical therapist | — | — | −0.18 (0.17) | .30 | |
Other provider | — | — | −.39 (0.30) | .20 | |
Patient's usual doctor* | 0.00 | 0.00 | |||
Difficulty obtaining initial care | |||||
Very little difficulty | 1.34 (0.37) | <.001 | 0.29 (0.15) | .05 | |
Some or a lot of difficulty* | 0.00 | 0.00 | |||
Satisfaction with technical care | |||||
Very satisfied | 0.88 (0.35) | .01 | 1.13 (0.16) | <.001 | |
Less than very satisfied* | 0.00 | 0.00 | |||
Satisfaction with interpersonal care | |||||
Very satisfied | 0.35 (0.39) | .38 | 0.61 (0.14) | <.001 | |
Less than very satisfied* | 0.00 | 0.00 | |||
Provider understanding of job | |||||
A lot of understanding | 0.91 (0.30) | .003 | 0.20 (0.10) | .05 | |
Less than a lot of understanding* | 0.00 | 0.00 | |||
Difficulty getting follow-up care | |||||
Very little difficulty | —a | 0.38 (0.17) | .02 | ||
Some or a lot of difficulty* | 0.00 | ||||
Coordination of care | |||||
Very satisfied | —a | 0.61 (0.13) | <.001 | ||
Less than very satisfied* | 0.00 |
Omitted reference category.
Follow-up measure omitted for model representing initial care.
R2 for models representing initial care only (n =123) and follow-up care (n=681) equals .52 and .46, respectively. Satisfaction ratings explained 26% and 38% of the variance in treatment experience for these two models.
For injured workers who had initial care only (n =123), there was little relationship between demographic factors and treatment experience. However, workers aged 45 to 54 rated their treatment experience significantly worse (β =−1.22, p =.007) than workers aged 55 or older. Satisfaction with prior medical care was positively related (p =.001) with current treatment experience, but patients' job satisfaction was not related to treatment experience, nor was provider type. For this group of patients who received limited treatment for their injury, access to initial care was a strong predictor of treatment experience. Injured workers who reported having little difficulty obtaining initial care rated their treatment experience much more positively than workers who reported having some or a lot of difficulty accessing care (β =1.34, p <.001). Injured workers who were very satisfied with the technical care they received also rated their treatment experience more favorably (p=.01). Finally, ratings of the provider's understanding of the patient's job activities, an indicator of the provider's occupational medicine expertise, was a strong predictor of overall treatment experience (β =0.91, p =.003). The R2 for the model was .53, with the satisfaction measures accounting for 26 percent of the total variance in treatment experience, after controlling for demographics, satisfaction with prior medical care, and with current job, type of injury, and provider.
The same general pattern of results described above was observed for injured workers who received follow-up care (n =681). Satisfaction with prior medical care was not found to be a predictor of current treatment experience, however. Compared to whites, nonwhites reported having a somewhat more favorable treatment experience (β =.30, p=.04). Satisfaction with technical and interpersonal quality were strongly and positively associated with treatment experience (β =1.13 and 0.61, respectively, p<.001), as was satisfaction with care coordination (β =0.61, p <.001). Access to care, both initial and follow-up treatment, as well as provider occupational medicine expertise, remained positively associated with treatment experience (p≤.05). The R2 for the model was .46. As a group, the satisfaction measures explained 38 percent of the total variance in treatment experience, after controlling for demographics, type of injury, provider type, and other variables.
We tested a third regression model (not shown in Table 3) to examine the predictors of treatment experience among the subset of 402 patients who had follow-up care and a specialty referral. Satisfaction with the timing of the referral and the quality of specialty care were both positively related (p=.053 and p<.001, respectively) to overall treatment experience.
Relationship of Treatment Experience to Disability Outcome
The importance of the above findings for quality improvement depends on whether patients' overall treatment experience is related to outcome. This question is addressed in Table 4. Patients older than age 45 had a greater risk (OR=1.53, 95 percent CI, 0.99 to 2.37, p =.055) of being on disability 6 or 12 months after claim filing compared to younger patients, those younger than age 45. Patients with carpal tunnel syndrome were also at greater risk for being on disability 6 or 12 months after claim filing compared with patients having “other injuries” (OR=7.49, 95 percent CI 1.61 to 34.77, p =01). As expected, patients who were on disability at the time of the interview were much more likely to be on disability later (OR=52.2, 95 percent CI=19.78 to 142.94, p <.001) compared to patients not on disability at time of interview. Similarly, patients who reported being less recovered from their injury were more likely to be on disability and out of work than patients who reported being more fully recovered (OR=4.92, 95 percent CI=1.71 to 14.21, p =.003). Even after adjusting for these factors, patients' overall treatment experience was associated with the risk of subsequent disability. The risk of being on disability 6 or 12 months after claim filing was 3.54 times higher (95 percent CI=1.21 to 10.33, p=.021) for patients who rated their treatment experience less than excellent as compared to patients who rated their treatment experience excellent. While disability status and recovery status at time of interview were both strongly associated with subsequent disability outcome at 6 or 12 months, their effect on the relationship of treatment experience to disability outcome was modest. The unadjusted odds ratio for the treatment experience variable was 2.63 (p =.005).
Table 4.
Measure | Adjusted Odds Ratio (95% Confidence Interval) | P-Value | ||
---|---|---|---|---|
Education | ||||
High school degree or less | 1.00 | |||
Post–high school education | 1.24 (0.52–2.94) | .62 | ||
Race | ||||
White | 1.00 | |||
Nonwhite | 1.40 (0.39–4.88) | .61 | ||
Sex | ||||
Male | 1.00 | |||
Female | 0.83 (0.35–1.96) | .67 | ||
Age | ||||
<45 | 1.00 | |||
45 and older | 1.53 (0.99–2.37) | .055 | ||
Type of injury | ||||
Other injuries | 1.00 | |||
Upper or lower extremity fracture | 2.49 (0.45–13.67) | .29 | ||
Low back sprain | 2.37 (0.39–14.20) | .35 | ||
Carpal tunnel syndrome | 7.49 (1.61–34.77) | .01 | ||
On disability at time of interview | ||||
No | 1.00 | |||
Yes | 53.20 (19.78–142.94) | <.001 | ||
Recovery status at time of interview | ||||
Completely or 75% recovered | 1.00 | |||
50% recovered or less | 4.92 (1.71–14.21) | .003 | ||
Overall treatment experience** | ||||
Excellent | 1.00 | |||
Less than excellent | 3.54 (1.21–10.33) | .021 |
Excludes self-insured patients (n =268) and cases where interview occurred 180 days or more postclaim filing (n =151).
The unadjusted odds ratio for “treatment experience” is 2.63 (p =.005).
Effects of Exclusion Criteria and Sensitivity Analysis
To test for potential bias arising from the exclusion of workers who were self-insured or who had an interview completed after 180 days following the filing of a claim, we conducted chi-square analyses to determine: (1) whether self-insured patients differed from state-fund patients, and (2) whether among state-fund patients those who were interviewed prior to 180 days differed from those interviewed after 180 days in terms of provider satisfaction variables, satisfaction indexes, timing of referral, treatment experience, or recovery status. Among all (24) comparisons, only one was statistically significant. State-fund patients interviewed prior to 180 days (n =385) were less likely to report having a full or near full recovery as compared to state-fund patients interviewed after 180 days (n =151) (26.8 percent versus 39.2 percent, p=.005). However, it is unlikely that the exclusion of patients interviewed after 180 days introduced bias into the analysis. Among all cases (n =804), time to interview was not correlated with the technical quality satisfaction index (r =.02, p =.65), the interpersonal quality satisfaction index (r =.03, p =.47), or the measure of overall treatment experience (r =−.05, p =.18).
An additional concern regarding the results reported above arose because, as discussed earlier, some respondents were interviewed close to the time (180 days after claim filing) disability status was measured. We restricted the time between claim filing and interview to 165 days, which reduced the average time to interview from 145 days to 137 days, and repeated the logistic regression analysis (the number of cases analyzed decreased from 385 to 293). There was little change in the pattern of results. The odds ratio for the variable representing patients' treatment experience remained statistically significant (p =.018) and increased slightly in magnitude from 3.54 to 4.0.
Discussion
The widely cited Institute of Medicine study Crossing the Quality Chasm (Committee on the Quality of Health Care in America 2001) emphasizes the importance of improving patients' treatment experience. Our findings support this view. The injured workers in our study resided throughout Washington State (33 of its 39 counties) and obtained medical care for their work-related injuries and illnesses from a broad spectrum of community physicians and other health care providers. We found that satisfaction with technical quality and, to a lesser extent, interpersonal quality, as well as the patient's rating of care coordination, were strongly and positively associated with overall treatment experience. The ability of the patient to access treatment for his or her injury was also found to be an important factor associated with more favorable treatment experience. The choice of health care provider, whether primary care physician, specialist, or chiropractor, had little influence on patients' ratings of their treatment experience. However, the provider's understanding of what the patient did on the job, a proxy measure for provider expertise in occupational medicine, was positively related to patients' treatment experience ratings.
Many of these factors could be modified through quality improvement efforts. Our ongoing quality improvement initiative in Washington State addresses several of these factors and focuses especially on improving care coordination, reducing barriers to access, and increasing health care providers' occupational medicine expertise (Wickizer et al. 2001, 2002, 2004). The importance of improving processes of care is underscored by our finding regarding the relationship of treatment experience to disability outcome. Patients who rated their treatment experience less favorably were 3.5 times as likely to be out of work and receiving disability compensation for their injury 6 or 12 months after filing a claim, compared to patients who rated their treatment experience more favorably. While important, we consider this finding suggestive. It is unclear whether dissatisfaction leads to poorer outcomes, whether it reflects poor care that results in poorer outcomes, or whether the types of people most likely to have poor outcomes are most likely to be dissatisfied with care. Further research using additional longitudinal data is needed to better understand how patient satisfaction may influence outcomes.
What is needed within the field of occupational health care, and within the broader field of chronic illness care, is the development of approaches to improve secondary prevention. Injured workers who remain on disability longer than two to three months have substantially diminished prospects of returning to productive employment (Cheadle et al. 1994). The cost of disability in terms of years of productive life lost is enormous (Fulton-Kehoe et al. 2000). A significant portion of disability suffered by patients with occupational injuries or illnesses might be prevented if the quality of care, care coordination, and related care processes important for secondary prevention were improved.
Our findings should be placed in the context of other studies. We know of only one prior peer-reviewed report of a general patient satisfaction survey within a workers' compensation population. That study found somewhat lower ratings of satisfaction among workers' compensation patients in California (Rudolph et al. 2002), as compared to the ratings reported here. Washington affords workers' compensation patients greater choice in selecting a provider than California. Also, the California study was not population-based. Patients in our study did have choice in selecting a health care provider, but this seemed to have little influence on their treatment experience. Prior research of back pain patients has found greater patient satisfaction with chiropractic care than physician care (Hertzman-Miller et al. 2002) but back pain patients accounted for only about a quarter of our study sample.
Studies that have examined the relations between patient satisfaction and self-assessed health status have produced mixed results. In a study of elderly HMO patients, Hall, Milburn and Epstein (1993) found evidence in support of a causal pathway from self-perceived health to subsequent satisfaction but not vice versa. In another study Covinsky et al. (1998) reported data showing that among elderly hospitalized patients discharge health status and satisfaction were positively correlated independent of the change in health status. Marshall, Hays, and Mazel (1996) conducted a detailed analysis of data gathered through the Medical Outcomes Study (MOS) to examine the relationship of satisfaction to health status. The researchers found evidence linking baseline satisfaction to subsequent mental health but not physical health. Finally, Kane, Maciejewski, and Finch (1997) examined the relationship between health status and subsequent satisfaction among patients undergoing cholecystectomy. The relative change in outcomes was positively related to satisfaction but explained only a small percentage of the variance in satisfaction.
The limitations of our study should be noted. Our analysis was restricted to patients receiving medical care through the workers' compensation system. Although limited largely to three conditions—carpal tunnel syndrome, low back sprain, and upper and lower extremity fractures—these conditions account for a substantial proportion of morbidity within the workers' compensation system and even within the general medical care system (Hart, Deyo, and Cherkin 1995; Katz et al. 1998; Feuerstein et al. 1998). Furthermore, our survey had a response rate of 53 percent. It is possible the results might differ for study nonrespondents and for other work-related injuries and illnesses, but this is unknown.
Our study uniquely approached patient satisfaction in an injured worker population from a pragmatic, quality improvement perspective. Assessing satisfaction within an episode of care provides insight that may have meaningful applicability to quality improvement efforts. In contrast, satisfaction data obtained at the level of an encounter precludes patient assessment of experiences over time and with multiple contacts regarding technical quality, care coordination, and other related aspects of care. Approaching satisfaction data from a health plan level may also fail to capture this experience because it is too general. Patients' perceptions of their treatment experience at the episode level appear to have a relationship with their eventual outcome. Although assessment of patient satisfaction offers recognized utility for improvement of the customer service side of health care delivery, our finding warrants further exploration into dimensions of the patient's experience with care that could help target quality improvement efforts, such as topics for best-practice education, performance feedback, or outcomes monitoring. Finally, our finding of a positive relationship between treatment experience and subsequent outcomes underscores the potential value of current efforts designed to improve the experience of patients in their encounters with the health care system.
Acknowledgments
The authors thank Jerry Gluck for his assistance with file construction and programming. Appreciation is also expressed to two anonymous reviewers for their helpful comments and suggestions.
Footnotes
The survey on which this study was based was funded by the Washington State Department of Labor and Industries. Partial support for the preparation of this article was provided by the Robert Wood Johnson Foundation through a grant from the Health Care Financing and Organization (HCFO) Program.
References
- Carr JD. Workers' Compensation Systems Purpose and Mandate. Occupational Medicine. 1998;13(2):417–22. [PubMed] [Google Scholar]
- Chassin MR, Galvin RW. The Urgent Need to Improve Health Care Quality.” Institute of Medicine National Roundtable on Health Care Quality. Journal of the American Medical Association. 1998;280(11):2533–7. doi: 10.1001/jama.280.11.1000. [DOI] [PubMed] [Google Scholar]
- Cheadle AC, Franklin G, Wolfhagen C, Savarino J, Liu PT, Salley C, Weaver M. Factors Influencing the Duration of Work-Related Disability A Population-Based Study of Washington State Workers' Compensation. American Journal of Public Health. 1994;84(2):190–6. doi: 10.2105/ajph.84.2.190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Committee on the Quality of Health Care in America. Crossing the Quality Chasm. Washington, DC: National Academy Press; 2001. [Google Scholar]
- Covinsky KE, Rosenthal GE, Chren M, Justice AC, Fortinsky RH, Palmer RM, Landefeld CS. The Relation between Health Status Changes and Patient Satisfaction in Older Hospitalized Medical Patients. Journal of General Internal Medicine. 1998;13(4):223–9. doi: 10.1046/j.1525-1497.1998.00071.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Vet HC, Heymans MW, Dunn KM, Pope DF, ver der Beek AJ, Macfarlane GJ, Bouter LM, Croft PR. Episodes of Low Back Pain A Proposal for Uniform Definitions to Be Used in Research. Spine. 2002;27(21):2409–16. doi: 10.1097/01.BRS.0000030307.34002.BE. [DOI] [PubMed] [Google Scholar]
- Elling RH. Workers' Health, Safety, and Compensation in Historical and Cross-National Perspective. An Overview. Annals of the New York Academy of Science. 1989;572:240–53. doi: 10.1111/j.1749-6632.1989.tb13610.x. [DOI] [PubMed] [Google Scholar]
- Feuerstein M, Miller VL, Burrell LM, Berger R. Occupational Upper Extremity Disorders in the Federal Workforce Prevalence, Health Care Expenditures, and Patterns of Work Disability. Journal of Occupational and Environmental Medicine. 1998;40(6):546–55. doi: 10.1097/00043764-199806000-00007. [DOI] [PubMed] [Google Scholar]
- Fulton-Kehoe D, Franklin G, Weaver M, Cheadle AC. Years of Productivity Lost among Injured Workers in Washington State Modeling Disability Burden in Workers' Compensation. American Journal of Industrial Medicine. 2000;37(6):656–62. doi: 10.1002/(sici)1097-0274(200006)37:6<656::aid-ajim10>3.0.co;2-c. [DOI] [PubMed] [Google Scholar]
- Hall JA, Milburn MA, Epstein AM. A Causal Model of Health Status and Satisfaction with Medical Care. Medical Care. 1993;31(1):84–94. doi: 10.1097/00005650-199301000-00007. [DOI] [PubMed] [Google Scholar]
- Harris LE, Swindle RW, Mungai SM, Weinberger M, Tierney WM. Measuring Patient Satisfaction for Quality Improvement. Medical Care. 1999;37(12):1207–13. doi: 10.1097/00005650-199912000-00004. [DOI] [PubMed] [Google Scholar]
- Hart LG, Deyo RA, Cherkin DC. Physician Office Visits for Low Back Pain Frequency, Clinical Evaluation, and Treatment Patterns from a U.S. National Survey. Spine. 1995;20(1):11–9. doi: 10.1097/00007632-199501000-00003. [DOI] [PubMed] [Google Scholar]
- Hays RD, Shaul JA, Williams VS, Lubalin JS, Harris-Kojetic LD, Sweeney SF, Cleary PD. Psychometric Properties of the CAHPS Survey Measures. Consumer Assessment of Health Plans Study. Medical Care. 1999;37(3, supplement):MS22–31. doi: 10.1097/00005650-199903001-00003. [DOI] [PubMed] [Google Scholar]
- Hertzman-Miller RP, Morgenstern H, Hurwitz EL, Yu F, Adams AH, Harber P, Kominski GF. Comparing the Satisfaction of Low Back Pain Patients Randomized to Receive Medical or Chiropractic Care Results from the UCLA Low-Back Pain Study. American Journal of Public Health. 2002;92(10):1628–33. doi: 10.2105/ajph.92.10.1628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hornbrook MC, Hurtado AV, Johnson RE. Health Care Episodes Definition, Measurement and Use. Medical Care Review. 1985;42(2):163–218. doi: 10.1177/107755878504200202. [DOI] [PubMed] [Google Scholar]
- Kane RL, Maciejewski M, Finch M. The Relationship of Patient Satisfaction with Care and Clinical Outcomes. Medical Care. 1997;35(7):714–30. doi: 10.1097/00005650-199707000-00005. [DOI] [PubMed] [Google Scholar]
- Katz JN, Lew RA, Bessette L, Punnett L, Fossel AH, Mooney N, Keller RB. Prevalence and Predictors on Long-Term Work Disability Due to Carpal Tunnel Syndrome. American Journal of Industrial Medicine. 1998;33(6):543–55. doi: 10.1002/(sici)1097-0274(199806)33:6<543::aid-ajim4>3.0.co;2-r. [DOI] [PubMed] [Google Scholar]
- Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999. [PubMed] [Google Scholar]
- Kyes K, Franklin G, Weaver M. Reliability and Validity of Medical Outcome and Patient Satisfaction Measures among Injured Workers in Washington State A Pretest. American Journal of Industrial Medicine. 1997;31(4):427–34. doi: 10.1002/(sici)1097-0274(199704)31:4<427::aid-ajim8>3.0.co;2-s. [DOI] [PubMed] [Google Scholar]
- Marshall GN, Hays RD, Mazel R. Health Status and Satisfaction with Health Care Results from the Medical Outcomes Study. Journal of Consultation and Clinical Psychology. 1996;64(2):380–90. doi: 10.1037//0022-006x.64.2.380. [DOI] [PubMed] [Google Scholar]
- McCracken LM, Klock A, Mingay DJ, Asbury JK, Sinclair DM. Assessment of Satisfaction with Treatment for Chronic Pain. Journal of Pain Symptom Management. 1997;14(3):292–9. doi: 10.1016/s0885-3924(97)00225-x. [DOI] [PubMed] [Google Scholar]
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA. The Quality of Health Care Delivered to Adults in the United States. New England Journal of Medicine. 2003;348(26):2635–45. doi: 10.1056/NEJMsa022615. [DOI] [PubMed] [Google Scholar]
- Rubin HR, Gandek B, Rogers WH, Kosinski M, McHorney CA, Ware JE. Patients' Ratings of Outpatient Visits in Different Practice Settings. Journal of the American Medical Association. 1993;270(70):835–40. [PubMed] [Google Scholar]
- Rudolph L, Dervin K, Cheadle AC, Maizlish N, Wickizer T. What Do Injured Workers Think about Their Medical Care and Outcomes after Work Injury? Journal of Environmental and Occupational Medicine. 2002;44(5):425–34. doi: 10.1097/00043764-200205000-00011. [DOI] [PubMed] [Google Scholar]
- Schuster MA, McGlynn EA, Brook RH. How Good Is the Quality of Health Care in the United States? Milbank Quarterly. 1998;76(4):517–63. doi: 10.1111/1468-0009.00105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sitzia J, Wood N. Patient Satisfaction A Review of Issues and Concepts. Social Science and Medicine. 1997;45:1829–43. doi: 10.1016/s0277-9536(97)00128-7. [DOI] [PubMed] [Google Scholar]
- Stump TE, Dester PR, Tierney WM, Wolinsky FD. Measuring Patient Satisfaction with Physicians among Older and Diseased Adults in a Primary Care Municipal Outpatient Setting An Examination of Three Instruments. Medical Care. 1995;33(9):958–72. [PubMed] [Google Scholar]
- Ware JE, Hays RD. Methods for Measuring Patient Satisfaction with Specific Medical Encounters. Medical Care. 1988;26(4):393–404. doi: 10.1097/00005650-198804000-00008. [DOI] [PubMed] [Google Scholar]
- Ware JE, Snyder MK, Wright R, Davies AR. Defining and Measuring Patient Satisfaction with Medical Care. Evaluation Program Planning. 1983;6(3):247–63. doi: 10.1016/0149-7189(83)90005-8. [DOI] [PubMed] [Google Scholar]
- Weaver M, Patrick DL, Markson LE, Martin D, Frederic I, Berger M. Issues in the Measurement of Satisfaction with Treatment. American Journal of Managed Care. 1997;3(4):579–94. [PubMed] [Google Scholar]
- Wickizer TM, Franklin G, Mootz R, Turner JA, Fulton-Kehoe D, Plaeger-Brockway R, Drylie D, Smith-Weller T. An Innovative Quality Improvement Pilot to Reduce Disability among Injured Workers in Washington State: Early Experience of the Occupational Health Services Project. Milbank Quarterly. 2004 doi: 10.1111/j.0887-378X.2004.00321.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wickizer TM, Franklin G, Plaeger-Brockway R, Mootz R. Improving the Quality of Workers' Compensation Health Care Delivery The Washington State Occupational Health Services Project. Milbank Quarterly. 2001;79(1):5–33. doi: 10.1111/1468-0009.00194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wickizer TM, Franklin G, Plaeger-Brockway R, Mootz R, Drylie D. Improving the Quality of Occupational Health Care in Washington State New Approaches to Designing Community-Based Health Care Systems. Journal of Ambulatory Care Management. 2002;25(2):43–52. doi: 10.1097/00004479-200204000-00006. [DOI] [PubMed] [Google Scholar]
- Williams B. Patient Satisfaction A Valid Concept? Social Science and Medicine. 1994;38(4):509–16. doi: 10.1016/0277-9536(94)90247-x. [DOI] [PubMed] [Google Scholar]
- Williams B, Coyle J, Healy D. The Meaning of Patient Satisfaction An Explanation of High Reported Levels. Social Science and Medicine. 1998;47(9):1351–9. doi: 10.1016/s0277-9536(98)00213-5. [DOI] [PubMed] [Google Scholar]
- Wingert TD, Kralewski JE, Lindquist TJ, Knutson DJ. Constructing Episodes of Care from Encounter and Claims Data Some Methodological Issues. Inquiry. 1995;32(4):430–43. [PubMed] [Google Scholar]