Understanding Employee Awareness of Health Care Quality Information: How Can Employers Benefit?

Abstract

Objective

To analyze the factors associated with employee awareness of employer-disseminated quality information on providers.

Data Sources

Primary data were collected in 2002 on a stratified, random sample of 1,365 employees in 16 firms that are members of the Buyers Health Care Action Group (BHCAG) located in the Minneapolis–St. Paul region. An employer survey was also conducted to assess how employers communicated the quality information to employees.

Study Design

In 2001, BHCAG sponsored two programs for reporting provider quality. We specify employee awareness of the quality information to depend on factors that influence the benefits and costs of search. Factors influencing the benefits include age, sex, provider satisfaction, health status, job tenure, and Twin Cities tenure. Factors influencing search costs include employee income, education, and employer communication strategies. We estimate the model using bivariate probit analysis.

Data Collection

Employee data were collected by phone survey.

Principal Findings

Overall, the level of quality information awareness is low. However, employer communication strategies such as distributing booklets to all employees or making them available on request have a large effect on the probability of quality information awareness. Employee education and utilization of providers' services are also positively related to awareness.

Conclusions

This study is one of the first to investigate employee awareness of provider quality information. Given the direct implications for medical outcomes, one might anticipate higher rates of awareness regarding provider quality, relative to plan quality. However, we do not find empirical evidence to support this assertion.

According to a recent survey sponsored by the U.S. Chamber of Commerce, approximately 11 percent of every payroll dollar of U.S. businesses was spent on health care benefits in 2002 (U.S. Chamber of Commerce 2003). While price remains the dominant factor driving employers' health plan contracting decisions, there is growing interest among employers to evaluate the quality of health plans and providers. In the 1997 Robert Wood Johnson Foundation Employer Health Insurance Survey, nearly 39 percent of large firms considered health plan accreditation and 52 percent considered the board certification of physicians when making contracting decisions (Marquis and Long 2001).

Some employers have begun to collect and distribute health plan and provider quality information to their employees, with the hope that this information will result in better clinical outcomes if individuals seek treatment from higher quality providers.1 As a second objective, employers hope to improve market efficiency, whereby employees who are offered a choice of plans and providers use information on quality, price, and other attributes when making decisions about their health care.2

Economic theory often assumes that consumers are fully informed about the attributes of the goods and services that they purchase. When information is not known, consumers may search for and utilize both informal and formal sources of information. Characterizing medical care as a “reputation good,”Satterthwaite (1979) suggests that since physicians' services are differentiated, an evaluation of their attributes can be achieved only through experience. As such, consumers may rely on informal sources such as spouses, relatives, and friends to learn about providers (Pauly and Satterthwaite 1981).3 Recent studies by Hoerger and Howard (1995); Feldman, Christianson, and Schultz (2000); and Harris (2003) find empirical support for this claim.

During the past decade, there has been significant investment to develop standardized systems of quality measurement based on the aggregate experiences of multiple consumers that can be used by employers, government purchasers, and other consumer groups.4 Several studies have investigated consumers' use of these formal sources of quality information (e.g., health plan report cards) and have found that demographic characteristics, health status, and an individual's likelihood of switching plans affect use (Short et al. 2002; Braun et al. 2002; Schultz et al. 2001; Fowles et al. 2000). Our study most closely resembles Schultz et al. (2001), who examined health plan report card use in 1998 by employees in 19 firms that were members of the Buyers Health Care Action Group (BHCAG). Their analysis revealed that gender, education, and stated preferences about the importance of quality had a positive influence on report card use, while firm size was inversely related to report card use.

One factor that has not received much attention in earlier work is the role that employers can have in educating employees about their health benefits and quality. Employer communication methods, including distributing booklets, brochures, and newsletters, holding education sessions, and using web-based strategies, may significantly impact employees' awareness and use of quality information.5

Two empirical studies have focused on employer communication and employee awareness of benefits. Driver (1980) compared the use of traditional benefits communication methods (e.g., booklets, brochures, and bulletin boards) with more interactive communication between employees and human resources staff members, and found that two-way communication was superior with respect to knowledge retained. More recently, Feldman and Schultz (2001) examined the factors associated with employee use of flexible spending accounts (FSAs). They found that more active employer communication strategies, such as holding special meetings with employees to explain FSAs, were positively associated with employee use of these accounts.

In 2002, we conducted a survey of 1,365 employees in 16 large firms in the Minneapolis–St. Paul metropolitan area to gain a better understanding of their awareness and use of provider quality information. We develop a framework to analyze the factors that are associated with employee awareness of employer-disseminated quality information on providers, focusing particular attention on the impact of employer communication strategies.

Study Setting

The study setting is the Buyers Health Care Action Group (BHCAG), a health insurance purchasing and reform coalition comprising 30 employers in the Minneapolis, Minnesota, region.6 These self-insured employers have approximately 250,000 employees and dependents eligible for health insurance. In 2002, 24 BHCAG employers offered “Choice Plus,” a point-of-service health plan with out-of-network coverage. Employers may offer other managed care and indemnity plans, but some have selected Choice Plus as their sole health insurance plan. About 100,000 employees and dependents were enrolled in Choice Plus in 2002.

Prior to 1997, BHCAG contracted with a single health plan to operate Choice Plus. In 1997, BHCAG implemented a new approach designed to create a more competitive and cost-efficient system. The new approach utilizes direct contracting with provider networks. Under the new approach, consumers choose among “care systems,” which are integrated teams of primary care providers, affiliated specialists, hospitals, and allied professionals.7 One unique requirement of Choice Plus is that primary care providers (PCPs) are prohibited from affiliating with multiple care systems. About 95 percent of PCPs in the Twin Cities contract with BHCAG through a care system.

In 2001, BHCAG sponsored two programs for reporting care system quality. The first was a consumer mail survey of 16,000 Choice Plus enrollees that was conducted by an independent survey organization. About 325 adults who told about their own health care and 325 parents who told about their children's care were sampled from each care system. Individuals were asked about their experiences with clinics and medical care in the prior year. Questions included how they rated their clinic and doctor, whether their doctors communicated well, and whether they were treated with respect by the staff. Additionally, they were asked whether they got the help they needed to obtain referrals, coordinate care, and get care without long waits. Care systems were given an above average, average, or below average rating on each dimension. The consumer survey was an updated version of the report card used in prior years and analyzed by Schultz et al. (2001).

The second program, launched in 1999, is the “Excellence in Quality” awards, which recognizes care systems' achievements in both technical and service quality. The annual awards are based on four criteria: (1) receipt of good consumer survey scores, (2) delivery of preventive care services to a large majority of their patients, (3) proof that the care system improved quality and the outcomes of care in at least one important way, and (4) demonstration of the care system's commitment to patient safety. Three levels of award are given: gold, silver, and special recognition. The gold award recognizes the top performing care system and carries a prize of $100,000, while silver is for the next highest performance and carries a $50,000 prize. Special recognition awards are given to systems that performed exceptionally well in one area. Winning care systems gain the privilege of using the title in marketing efforts for the next two years and are identified in the consumer survey results booklet described below.

Employees could access the consumer survey results and the quality award information either by obtaining a special paper booklet published by BHCAG or by searching on the Choice Plus web site (http://www.choiceplus.com). Additionally, information about the quality award could be obtained from award-winning care systems' advertisements, physicians, or employers.

Conceptual Framework

Our framework is based on a search-theoretic model, whereby an employee's propensity to search for and become aware of provider quality information depends on the benefits obtained from acquiring the information and the search costs (Stigler 1961).8

Let B_i denote the benefit to the ith employee of quality information search. We hypothesize that the “productivity” of quality information is directly related to an employee's medical care utilization. Employees who have more frequent interactions with medical care providers may derive a larger benefit from increased knowledge about the specific quality of care delivered by the providers who are treating them, relative to employees who have few or no provider interactions. Other factors correlated with anticipated medical care utilization are also likely to affect the benefit derived from search and awareness.9 For example, younger female employees may exhibit greater concern about provider quality if they are planning to have children and will need prenatal care.

Employees who are less satisfied with their current care system and who are likely to switch providers during the next open enrollment period are also more likely to benefit from having quality information. Dissatisfaction with providers and medical care utilization are both correlated with employee use of health plan report cards (Short et al. 2002; Braun et al. 2002; Schultz et al. 2001; Fowles et al. 2000).

Let C_i denote the costs associated with the ith employee's search for quality information. Searching for quality information requires an expenditure of the employee's time and effort. We hypothesize that employers who directly communicate with employees about the quality information, using either traditional or web-based methods, can reduce this time cost and increase awareness.

Comprehension costs represent a second type of cost borne by an employee. More formal sources of quality information use particular methodologies that require the employee to expend effort to read and understand what is being measured. We hypothesize that employee education will be inversely related to the magnitude of this cost and positively related to awareness of quality information.

Using an implicit benefit–cost analysis, an employee's propensity to search for quality information depends on whether the expected benefit from having the information is greater than the cost of searching. The propensity to search is an unobserved variable, but it has an observed counterpart, denoted as A_i. We specify the following reduced-form model for the observed employee awareness (A_i):

$$Prob\left(A_i=1\right)=\gamma B_i+\gamma C_i+{\varepsilon }_i$$

The probability that an employee indicates awareness of the quality information is a function of factors that influence the benefits of search (B_i), factors that influence the costs of search (C_i), parameter vectors γ and λ, and a random error term ɛ_i that captures unobserved factors that influence employee awareness. We now turn to a discussion of the empirical specification.

Data and Measures

We surveyed employees from 16 BHCAG member firms that had selected Choice Plus as their sole health plan in 2002 or where it was a dominant plan among several choices.10 Employees were randomly selected within strata defined by family structure, employer, and whether the employee or family member had switched care systems between 2001 and 2002. Stratification by family structure ensured adequate representation of both single employees and families with their sole source of health benefits through BHCAG employers.11 Stratification by employer ensured adequate representation of employees from smaller companies. Employees who switched care systems from 2001 to 2002 were oversampled because we thought these employees would be more likely to search for information about their health benefit options.

Single employees responded for themselves. Within families, the interviewers asked to speak with the person who was most knowledgeable about health insurance and medical care. The respondent answered questions for him- or herself, the spouse or domestic partner (where applicable), and one randomly selected child (where applicable).12 The final sample sizes were 711 single employees and 654 families. Given potential differences between singles and families, we estimate separate empirical models for each sample.

We are interested in evaluating the set of factors that influence an employee's awareness of the consumer survey results and the quality award.13 We constructed binary measures equal to 1 if the person was aware of each information source (consumer survey and quality award) and 0 otherwise.

We include several explanatory factors that are predicted to influence the benefits of search. The health care experiences of individuals and families are captured with two measures. First, we include an indicator variable for whether an employee had any interaction with a medical care provider during the prior year.14 Second, we control for the health status of the employee. We expect that individuals in poorer health typically have more interactions with providers and tend to exhibit greater concern about the quality of care they receive. We measure health status by including a dummy variable for whether an employee had any of the following chronic conditions: asthma, diabetes, hypertension, cancer, heart disease, or depression.

Demographic variables, including age and whether the employee is female, are also included. Given a positive relationship between age and illness propensity, we expect that older individuals will derive greater benefit from having provider quality information. Other studies have found that women tend to be more concerned about all aspects of health care quality, and therefore we expect higher-quality information awareness, relative to males.15

We also include a measure of employee satisfaction with his or her care system during 2001. This measure is scaled from 0 to 10, with higher numbers implying greater satisfaction with one's care system. We hypothesize that individuals and families who were less satisfied with their care systems in 2001 will be more likely to seek information about the quality of alternative care systems during 2002 open enrollment.

Job tenure is included to capture an employee's level of uncertainty with respect to his or her health benefits. In other work, Feldman et al. (2000) found that employees with longer job tenure may be more likely to trust their employer's decisions and, therefore, be less likely to search. We also include a measure for the number of years the employee has lived in the Twin Cities. The BHCAG employees who have lived longer in the Twin Cities may have greater understanding of the providers that are being evaluated by the two quality initiatives and therefore perceive greater benefits from the quality information. On the other hand, however, longer tenure may mean that consumers have already formulated their views and may perceive less benefit from search.

We include two sets of measures corresponding to the cost of search. First, we surveyed a human resources representative from each of the 16 firms to better understand the communication strategies used for both general health insurance benefits information and the specific quality information. A set of four, non-mutually exclusive, binary variables was constructed to measure how employers distributed information about the consumer survey results and the quality award. These methods included distributing paper copies of the booklet to all employees during open enrollment, distributing paper copies of the booklet to those employees who requested them, posting relevant information on a web site or directing employees to a web site through an intranet or the Internet, or having a human resources representative available to answer questions. We expect that the use of any communication strategy will be positively related to employee awareness of quality information, relative to employers not using that strategy.

We also control for an employee's income, since this is likely to be correlated with the opportunity cost of time. Income was missing for some observations, so we set income equal to zero for these individuals and included a dummy variable in the models that was equal to one. We use education level to proxy for comprehension costs. Education is defined using a categorical variable corresponding to the following education levels: less than a high school diploma, high school graduate, some college or technical school, a four-year college degree, or a post-baccalaureate degree.

For the sample of families, we modified our specification to take account of multiple household members. In particular, we include a binary indicator variable for the presence of children. We also use a measure of family income and define our residence measure to correspond to the longest tenure of any household member in the Twin Cities. Finally, the health status and utilization measures equal one if any household member had a chronic condition or if any member had a doctor visit during the year, respectively.16

Results

Table 1 provides descriptive statistics. Approximately 33 percent of both single employees and families reported seeing the consumer survey results and 23 percent reported seeing the quality award information. However, only 15 percent of singles and 20 percent of families reported seeing both types of quality information.17 These percentages are lower than those found by Schultz et al. (2001), who had reported that 47 percent of singles and 52 percent of families had seen the report card. One possible explanation is that employees were more likely to search for information in 1998 because the care system program was only in its second year, and so, over time, the benefits of seeking information have decreased.

Table 1.

Descriptive Statistics

	Singles (N=711)		Families (N=654)
Variable	Mean	SD	Mean	SD
See consumer survey results	.347	.476	.341	.475
Hear how care system can win quality award	.228	.42	.244	.430
Female employee	.60	.49	.43	.497
Age of employee (years)	41.02	13.66	43.13	10.58
Twin Cities tenure (years)	24.37	17.42	28.83	17.07
Have kids	—	—	.679	.467
Education level	3.48	.902	3.54	.991
Chronic disease	.341	.47	.394	.489
Income 2001 ($)	43,670	24,184	85,190	54,213
Income missing	.179	.383	.235	.425
Job tenure (years)	8.79	8.96	10.21	8.97
Any doctor visit	.783	.414	.928	.259
Care system rating (0–10 scale)	8.04	1.19	7.81	1.69
Booklet distributed to all	.291	.455	.317	.466
Booklet available by request	.262	.440	.292	.455
Web-based information	.052	.222	.092	.29
Human resources representative	.154	.362	.163	.370

There are some notable differences between singles and families, which may influence their respective use of quality information. By our study definition, singles do not have children, in contrast to 68 percent of the families. Not surprisingly, families also had more interactions with providers. Nearly 93 percent had at least one doctor visit by a family member compared with only 78 percent of singles. Also, employees with families tended to be slightly older and have longer job tenure.

The 16 employers adopted various methods to disseminate the provider quality information. Four of the firms did not pursue any direct communication strategy, 11 firms used one of the listed methods described above, and one firm used two methods. We were particularly surprised that only one firm reported using web-based communications to educate employees about the quality information. Measured in terms of employees, approximately 30 percent of the singles and families received a paper booklet of the consumer survey results, while another 28 percent could request a booklet. Another 15 percent had a human resources representative available to answer questions. Nearly 32 percent of singles and 21 percent of families received no direct communication from their employer about the quality information.

Given potential differences between singles and families, we estimated the multivariate models of consumer survey and quality award awareness separately for each group.18 Because it is probable that common unobserved factors influence both employee awareness of the consumer survey results and the quality award information, we used bivariate probit analysis, which permits a nonzero correlation structure for the error terms and increases the efficiency of our parameter estimates. In fact, with estimates of ρ equal to .70 for singles and .86 for families in the baseline model, the assumption of zero correlation is strongly rejected. Since parameter estimates for probit models are not directly interpretable, we report marginal effects and standard errors in Table 2.19

Table 2.

Bivariate Probit Results (Marginal Effects Reported with Standard Errors in Parentheses)

	Singles		Families
	Consumer Survey Information	Quality Award Information	Consumer Survey Information	Quality Award Information
Female	.102***	.03	.046	−.026
	(.032)	(.034)	(.03)	(.039)
Age	.001	−.004**	.000	−.0003
	(.002)	(.002)	(.003)	(.002)
Education	.066**	.028	.074***	.049***
	(.031)	(.025)	(.022)	(.018)
Have kids	—	—	.008	−.034
			(.045)	(.041)
Chronic disease	−.01	.002	.065*	.009
	(.056)	(.022)	(.038)	(.038)
Income 2001	.0000	.0000	−.0000	−.0000
	(.0000)	(.0000)	(.0000)	(.0000)
Income missing	.103	.023	−.058	−.116*
	(.063)	(.051)	(.066)	(.063)
Twin Cities tenure	.002	.0024**	.003	.001
	(.0014)	(.0011)	(.002)	(.002)
Job tenure	−.004	.004	−.003	−.002
	(.003)	(.0024)	(.003)	(.002)
Any doctor visit	.074*	.089**	−.008	.020
	(.043)	(.042)	(.09)	(.069)
Care system rating	.001	.012	−.0005	.013
	(.013)	(.01)	(.014)	(.014)
Booklet distributed to all	.186***	.104*	.192**	.048
	(.064)	(.058)	(.076)	(.051)
Booklet by request	.245***	.079	.301***	.107*
	(.07)	(.065)	(.079)	(.059)
Web-based information	−.043	.01	.192***	.059
	(.07)	(.062)	(.064)	(.042)
Human resources representative	.005	.005	.071	.063*
	(.046)	(.036)	(.055)	(.038)
ρ	.70		.87
N	622		548

^*p<.10;

^**p<.05;

^***p<.01.

Employer communication strategies are directly related to employee awareness of the consumer survey results and quality award information. Single employees are 18.6 percentage points more likely to be aware of the consumer survey results when they receive a booklet directly from their employer and 24 points more likely when a copy of the booklet is available by request. The magnitude of the effects on quality award awareness is smaller. Distributing a paper copy of the booklet is associated with a 10.4 point increase in the probability of quality award awareness, while having the booklet available by request is associated with a 7.9 point increase. This latter effect, however, is imprecisely estimated.

We find a similar pattern for families when the employer distributes consumer survey booklets to all employees or to those who request them. While the relation between the use of web-based communication and awareness of the quality award is positive, the effect is significant only for families. In particular, the use of web-based communication increases a family's likelihood of being aware of the quality award by 19.2 points, holding all else constant.20

Employee characteristics also influence the probability of quality information awareness. Education exhibits the largest influence of the demographic factors. Moving up one education category (e.g., from high school graduate to some college) is associated with a 7.4-point increase in the probability of seeing the consumer survey results by families and a 6.6 point increase in the probability for singles. Our hypothesis regarding the influence of gender on awareness finds limited support. While single female employees are 10 points more likely to be aware of the consumer survey results relative to single males, we find no other statistically significant differences in our specification.

Employees' care system experiences during 2001 are related to their awareness of quality information. Single employees who visited the doctor during 2001 were almost nine points more likely to be aware of the quality award. This may be due in part to the fact that care systems that won an award could advertise this information in their physician clinics. We found no such effect for families, though this may be due to a lack of variation in the explanatory variable. Finally, our estimates produce no support for the hypothesized relationship between an employee's satisfaction with his or her care system in 2001 and the awareness of quality information.21

We performed several sensitivity checks. First, we tested for the presence of a nonlinear relationship between satisfaction and information awareness by adding a quadratic satisfaction term to the model. We also tested for whether there is a “threshold effect” of dissatisfaction on information awareness by using dummy variables corresponding to different cutoff points in the sample distribution of an employee's care system rating (e.g., lowest 10th percentile). In neither case did we find an effect.

As a second check, we evaluated whether information awareness is more salient for new employees by reestimating the models with a dummy variable corresponding to whether an employee had one year or less of job tenure. Again, we found no change in our results.

Third, we investigated whether the effectiveness of particular communication strategies differed by employee demographic characteristics. The trade press has identified an increasing number of employers who are developing communication strategies that target specific employee segments. To examine this issue with respect to benefits communication, we reestimated the models including interaction terms between each of the communication strategies and three demographic variables (female, age, and education). Unfortunately, due to high multicollinearity, we could not obtain estimates with sufficient precision to detect whether the effectiveness of communication strategies differed by employee demographics. The interaction terms in our model were not jointly significant and therefore we chose not to report these findings.

Implications and Concluding Remarks

Evaluating the quality of health plans and providers is becoming a more common practice among purchasers and consumers. For many employers, the impetus for intensive benefits education, including the provision of quality information to their employees, comes from the perception that a direct linkage exists between employee comprehension of the value of these benefits and satisfaction. However, providing quality information to employees comes at a significant cost. Employers incur fixed costs associated with the collection, analysis, and presentation of survey and claims data as well as variable costs associated with the distribution of materials. As with any investment, employers must weigh the costs of providing quality information against the potential value to employees from having it available to support enrollment and medical care decisions. The latter depends fundamentally on whether employees are aware of the information in the first place.

The results of our empirical analysis reveal three key findings. First, employer communication strategies appear to have a large effect on the probability that an employee is aware of the consumer survey results and the quality award. In particular, we find that distributing paper booklets to all employees or having booklets available by request have the largest impacts on awareness. We also find limited evidence that web-based communications may increase awareness. Overall, it is striking that given the large investment BHCAG member firms made in collecting the quality information, they were not more active with respect to educating employees about it. Research questions that should be addressed in future work include whether differences exist with respect to the effectiveness of written versus electronic communication strategies and whether the use of multiple communication methods may prove more effective than a single method. Unfortunately, we could not test the latter hypothesis due to observing too few employers that use multiple communication strategies.

Second, employee education is positively associated with quality information awareness. Comprehension costs are likely to be higher for less-educated workers. However, intensive employee education may offset some of those costs. Furthermore, if an employer has a heterogeneous workforce, it may consider targeting certain communication strategies toward particular employee segments. Further research is needed to understand the impact of such a strategy. In this new era of “consumer-driven” health care, empowering employees by providing them with detailed information is critical to help them become better consumers of health care. However, based on the findings of this research, how that information is delivered may be just as important.

Third, less than one-third of employees in our sample were aware of the consumer survey results and quality award information. This low level of awareness is of particular concern to employers. In fact, BHCAG member firms decided in 2002 to discontinue the large-scale survey program to measure employees' satisfaction with their care systems, due in part to concerns about the lack of consumer use and the high cost of the survey.

Though BHCAG's care system model is unique, the insights gained from this analysis have broader implications. While most studies have focused on health plan quality, the BHCAG experience is about provider quality information. One of the most significant challenges associated with implementing provider quality measurement systems is the cost associated with data collection. In most geographic markets, the number of provider groups exceeds the number of health plans by an order of magnitude. The result is a much larger burden of collecting data to measure provider quality. Samples of consumers associated with each provider group must be large enough to effectively measure and identify quality differences. Finally, given the direct implications for medical outcomes, provider quality information is likely to be more meaningful to consumers than health plan quality information. If true, then one would expect consumers to have higher rates of awareness regarding provider quality information. However, the results from this study do not provide empirical support for this conclusion.