What Do Consumers Want to Know about the Quality of Care in Hospitals?

Abstract

Objective

To guide the development of the Consumer Assessments of Healthcare Providers and Systems (CAHPS®) Hospital Survey by identifying which domains of hospital quality included in a survey of recent hospital patients, and which survey items within those domains, would be of greatest interest to consumers and patients.

Data Sources/Study Setting

Primary data were collected in four cities (Baltimore, Los Angeles, Phoenix, and Orlando), from a demographically varied mix of people of whom most, but not all, had recently been hospitalized or had a close loved one hospitalized.

Study Design/Data Collection Method

A total of 16 focus groups were held in these four cities. Groups were structured to be homogeneous with respect to type of health care coverage (Medicare, non-Medicare), and type of recent hospital experience (urgent admission, elective admission, maternity admission, no admission). They were heterogeneous with respect to race/ethnicity, gender, and educational attainment. In addition to moderated discussions, focus group participants completed a pregroup questionnaire and various paper and pencil exercises during the groups.

Principal Findings

A wide range of features were identified by participants as being relevant to hospital quality. Many were consonant with domains and items in the CAHPS Hospital Survey; however, some addressed structural features of hospitals and hospital outcomes that are not best derived from a patient experience survey. When shown the domains and items being considered for inclusion in the CAHPS Hospital Survey, participants were most interested in items relating to doctor communication with patients, nurse and hospital staff communication with patients, responsiveness to patient needs, and cleanliness of the hospital room and bathroom. Findings were quite consistent across groups regardless of location and participant characteristics.

Conclusions

Consumers and patients have a high degree of interest in hospital quality and found a very high proportion of the items being considered for the CAHPS Hospital Survey to be so important they would consider changing hospitals in response to information about them. Hospital choice may well be constrained for patients, but publicly reported information from a patient perspective can also be used to support patient discussions with facilities and physicians about how to ensure patients have the best hospital experience possible.

The Centers for Medicare and Medicaid Services (CMS) is engaged in several initiatives to publicly report measures of health care provider performance. As early as 1999, CMS began publicly reporting CAHPS® patient experience survey and Health Employer Data and Information Set (HEDIS) measures about health plans. In later years, public reporting was expanded to end-stage renal disease facilities, nursing homes, and home health agencies. More recently, CMS has begun to publicly report quality information on hospitals, both to help consumers make more informed decisions and to increase the public accountability of hospitals. As part of this effort, the CAHPS Hospital Survey is being developed to measure hospital performance from the patient perspective. The CAHPS Hospital Survey was developed according to CAHPS survey design principles and procedures (Crofton, Lubalin, and Darby 1999) and is focused on measuring aspects of care that the patient is in the best position to evaluate; thus, it does not focus on having the patient evaluate technical aspects of care. Other articles in this supplement contain information about the development of the instrument and its psychometric properties, and provide complete lists of domains and items under consideration at various points in the survey development process (Keller et al. 2005; O'Malley et al. 2005).

The primary purpose of the CAHPS Hospital Survey is to produce information that is useful to consumers in comparing the quality of hospitals. CMS is required to publicly report all data collected through the survey. It is therefore important that the survey contains only items that are important to consumers and that they would find useful for comparing hospitals. Rather than make assumptions about what would be important to consumers, it was deemed critical, in the survey development process, to hear directly from consumers about this issue (Cleary and Edgman-Levitan 1997; Shaller et al. 2003; Sofaer and Firminger 2005). We present here the results of qualitative research using focus groups (Morgan 1998; Sofaer 1999, 2002) that were used to elicit input from consumers about how they define hospital quality and what they would like to hear from other patients about their hospital experiences.

Several patient satisfaction questionnaires are already being administered in hospitals, primarily for quality improvement (e.g. Press-Ganey 2004; National Research Center/Picker Group 2004). It was therefore also important to CMS that HCAHPS complement rather than duplicate existing survey efforts, that it not be too long, and that it only ask the questions most important to consumers with a need to compare hospital quality. Focus groups are considered a highly appropriate method to elicit information from relevant individuals about the importance not only of general domains but of individual items included in a survey. They are also an effective method for assessing the kind of information that a group such as consumers would find most useful in comparing hospital quality.

Once the CAHPS Hospital Survey is finalized and implemented nationally, scores on the items in the survey will be displayed on the Hospital Compare website, which is available both through http://www.medicare.gov and through http://www.dhhs.gov. Additional planned consumer testing will provide more in-depth feedback on how data from the CAHPS Hospital Survey should be displayed on the Hospital Compare website, how to best describe and explain these data to the lay public, and how consumers might use the results.

METHODS

A total of 16 focus groups were conducted in two rounds, held 6 months apart, in a total of four geographic locations. Focus groups lasted for 2 hours. The first round of six groups was conducted in October 2003. Three were held in Baltimore, MD and three in Los Angeles, CA. The second round of 10 focus groups was held in March 2004, five in Phoenix, AR and five in Orlando, FL. These markets were chosen because of the availability of a wide range of hospital facilities, the variety of health care coverage options available, and the ability to recruit culturally diverse participants.

As shown in Table 1, participants in the groups were recruited to vary in terms of age, gender, race/ethnicity, and type of health care coverage. In the first round of groups, all participants were people with Medicare. In the second round, six groups were limited to people with Medicare but four were designed for people with coverage other than Medicare. We deliberately included a substantial proportion of participants with low education levels (i.e. high school education or less) as a proxy for literacy skills, to ensure that the survey items and subsequent reports would be understandable and salient to the broadest range of potential users.

Table 1.

Demographic Characteristics of Focus Group Participants

	Total Number of Participants (N =153)	Percentage of Participants
Gender
Male	52	34.0
Female	100	65.3
Missing	1	0.7
Race/ethnicity
White	96	62.8
Black/African American	32	20.9
Asian	1	0.7
Native Hawaiian/other Pacific Islander	1	0.7
American Indian/Alaskan Indian or Native	2	1.3
Unknown	21	13.7
Hispanic or non-Hispanic
Hispanic	32	20.9
Non-Hispanic	92	60.1
Unknown	29	19.0
Level of education completed
Eighth grade or less	5	3.3
Some high school, but did not graduate	7	4.6
High school graduate or GED	52	34.0
Some college or 2 year degree or vocational school	57	37.3
Four year college graduate	16	10.5
More than 4 year college degree	14	9.2
Unknown	2	1.3

In each city, the groups were organized by the extent and kind of experience of participants with respect to a recent hospitalization, as we wanted to identify whether and in what ways recent experience (or lack of experience) with a hospital admission might influence participants' definitions of hospital quality and the salience and comprehensibility of survey items. Groups were organized as follows:

• Four Medicare elective admission groups (one in each location): participant, or a close loved one, had to be a Medicare recipient 65 years old or older who had been hospitalized within the past 12 months for elective care.

• Four Medicare urgent admission groups (one in each location): participant, or a close loved one, had to be a Medicare recipient 65 years old or older who had been hospitalized within the past 12 months for emergency care.

• Two non-Medicare elective admission groups (Phoenix and Orlando): participant, or a close loved one, had to be between the ages of 35 and 64 and have been hospitalized within the past 12 months for elective care.

• Two non-Medicare urgent admission groups (Phoenix and Orlando): participant, or a close loved one, had to be between 35 and 64 years of age and have been hospitalized within the past 12 months for emergency care.

• Two maternity admission groups (Phoenix and Orlando): women between the ages of 18 and 50 who had delivered a child at a hospital within the past 12 months and spent at least one night in the hospital.

• Two Medicare groups with no recent hospital experience (Los Angeles and Baltimore): participant had to be a person with Medicare 65 years old or older without a hospital admission in the last 12 months and without a close loved one with a hospital admission in the last 12 months.

Groups with participants who had experienced a recent hospitalization began with a warm-up exercise in which group members were asked to describe how they came to be admitted, with probes addressing such issues as previous experience with the hospital, how they were admitted, and the role of their physician in the choice of hospital. All focus group sessions then included a free-response discussion in which people were asked to brainstorm about what qualities they associate with a good, or high quality, hospital, and contrastingly, what qualities they associate with a bad, or poor quality hospital. In Baltimore and Los Angeles, this initial phase also included a “nominal group process.” In this kind of process, people are asked a question, given quiet time to think about and write down their answers, and then respond in turn to the moderator's request for their answers.

Following the open-ended discussion, each group was presented with a series of handouts, which contained items taken directly from the content domains of the original 66-item CAHPS Hospital Survey field test questionnaire. In Baltimore, where the first groups were conducted, handouts were organized into six domains, based on preliminary statistical analysis of the field test survey (Keller et al. 2005). Subsequently, the number of domains in the survey was modified to reflect the results of additional psychometric and factor analysis using CAHPS Hospital Survey field test data (Keller et al. 2005). Therefore, in all other locations handouts referred to seven content domains, as follows: (1) doctors' communication with patients; (2) nurse and hospital staff communication with patients; (3) hospital staff responsive to patient needs; (4) hospital staff help patients manage pain; (5) avoiding problems with medications; (6) hospital room comfort and cleanliness; and (7) hospital staff help patients avoid problems after they leave.

In Phoenix and Orlando, participants were asked to mark each item listed in the handouts as follows: (1) a star, to signify an aspect of quality that was so important to them they might change hospitals if scores for that domain were poor; (2) a strike-through line if that aspect of quality was not important to them at all; or (3) no mark at all if that aspect of quality was not important enough to influence them to change hospitals. A discussion of responses followed.

Next, respondents were asked to select the two most important items from the list by circling the item number. To get a sense of the relative importance of the seven domains, participants were also asked to complete the same marking exercise with a handout that listed only the domain names.

Moderators found it easy to engage the participants in discussions on hospital quality. Nevertheless, the exercises in these groups were cognitively challenging for many participants. We asked people to make a large number of rather fine-grained assessments (of items and domains) within a fairly short (2 hours) time frame. It was important for moderators to observe the proceedings carefully and watch for signs of burnout in participants, so they could modify aspects of the methodology to reduce participants' cognitive burden. Over the course of the 16 groups, we slightly revised the instructions that the moderator gave to the participants to clarify what judgments were required. We also slightly modified the printed materials (listing items and domains) to make the information easier for the user to understand. Most important, we repeatedly encouraged participants to ask questions about anything they did not understand. Although some adjustments were made, our approach was sufficiently consistent to ensure that findings from across groups could be compared or aggregated as originally intended in the research design.

FINDINGS

Findings from Focus Group Discussions

Free Response Consumer Definitions of Hospital Quality

In all 16 groups, the moderator began with an open-ended discussion of the characteristics participants associated with hospital quality. No suggestions or examples were given to participants. A wide range of characteristics was brought up, including both domains included in the CAHPS Hospital Survey instrument and domains that are not. In 15 of the 16 groups, participants mentioned communication with physicians, nurses, and all hospital staff, with many indicating this was the most important characteristic for them. Across the groups, specific aspects of communication quality mentioned included staff showing respect, compassion and kindness; being polite; providing reassurance in a frightening situation; spending enough time with patients; and involving patients in decisions as much as they wanted. Also in 15 of the 16 groups, participants mentioned cleanliness of both the hospital room and bathroom. The discussions made clear that for many people, cleanliness is important not just in itself, but because a dirty facility is one in which a patient might get an infection. In the first round of groups, which included only people with Medicare, another topic mentioned in all six sessions was having staff that responded rapidly to the patient's needs. In the second round of groups, which included a wider range of participants, staff responsiveness was spontaneously mentioned in about half the groups.

CAHPS Hospital Survey domains that were not mentioned at all, or mentioned considerably less often, included pain management, medications, planning for discharge, and other aspects of the hospital environment such as the privacy or the temperature of the room. However, in some groups, when the full range of domains from the CAHPS Hospital Survey was presented by the moderator, participants expressed recognition of the importance of virtually all the domains. As one said: “How could we have forgotten these?”

Several topics were raised by participants that are not included in the CAHPS Hospital Survey. These included the reputation of the hospital; the expertise of the physicians and nurses; the hospital's experience, expertise and specialization in the condition for which the patient is being hospitalized, as well as having the most up to date equipment for that condition; availability of brand name medications the patient was used to taking; the ratio of nurses to patients; getting help in walking; the location of the hospital and ease of access to the hospital, including access by public transportation; hospital-acquired infections; whether a patient advocate is available; having good food; help in screening out too many visitors; being asked about advanced directives; and “coming out alive and well.” Some participants worried about getting unnecessary procedures while others worried about being discharged too soon. Finally, in nearly half the groups, care received in teaching hospitals from people who participants thought of as “students” was viewed as a sign of poor quality. As one participant commented, “They have a lot of trainees [at Hospital X], so that's gonna impact the kind of service you get. How knowledgeable are they? They're all trainees.”

Items within Domains Viewed as Most Meaningful

In most but not all groups, the moderator led a discussion about which items, within each domain in the field test survey, participants viewed as most meaningful and appropriate to include on the final survey. Such a discussion was held in all of the Los Angeles, Phoenix, and Orlando groups and in one of the three Baltimore groups. In Baltimore, this discussion was limited to survey domains that had been identified by members as being of greatest interest. In Los Angeles, participants agreed that there should be at least one item from each of the seven domains presented, and their discussion focused on survey domains that group had not initially identified as most important, in order to determine which items from these domains should be retained, and which were candidates for elimination. We report by domain, across groups.

Doctors' Communication with Patients

Of the five items in this domain the ones that were most often chosen as important to include were having doctors (1) listen carefully to patients and (2) explain things so patients could understand them. One respondent reasoned, “If they're not listening they might miss something.” Another commented, “When you don't understand something, it's really terrible,” while another noted that for them clear explanations were “a total sign of respect, because we don't …have the education they do and [we need] to understand what is going on with your body…put it in layman terms.”

Across most groups, participants thought the item on doctors spending enough time with the patient was the least important one within this domain, although participants in the Baltimore group did not agree. Participants in the second round of focus groups felt that this factor was important but expressed little hope that patient feedback on this issue would motivate doctors to change their behavior. Here are some representative quotes:

Doctors listen carefully to patients. In my mind, it's the other way around. We have to listen carefully to them, because they come in fast and then they're out in a second.

The [doctors] blaze in and they blaze out

They're only there once a day for five minutes.

Nurse and Hospital Staff Communication with Patients

Within this domain, three items were most likely to be considered important, across the groups: nurses (1) listening carefully to patients; (2) treating them with courtesy and respect; and (3) explaining things carefully. However, as there is conceptual overlap between courtesy and respect and listening carefully (someone who is not listening can hardly be considered courteous or respectful) some participants thought the courtesy item was less important to include. Listening carefully had a special significance to participants as an indicator of quality. As one participant noted, “If [nurses] are not listening carefully, they might miss something. I might mention some little symptom that means nothing to me but is very important.” With respect to explanations, several participants commented that they rely on nurses to explain what the doctors have said to them and to communicate treatment plans:

A lot of times you go in there and the doctor says things that go right over your head. And the nurse comes in, and they can more or less say what the doctor said or what he meant in laymen's terms.

When they [nurses] listen … sometimes you're scared. And they might just say, “Let me get you a pamphlet, or a little more information.” So they do listen.

On the other hand, having nurses spend “enough” time with the patient was not considered a useful item. As with the same item regarding doctors, participants were not sure what “enough” might mean. Those who interpreted this term broadly thought it implied “hand-holding” or “coddling,” which they did not want or need. As one said, she wanted nurses to spend the necessary time “and then go away.” For those who thought the item meant spending enough time to do required tasks, it was important.

Another item within this topic that was the subject of considerable discussion across the groups was: “Doctors, nurses, and other hospital staff introduce themselves to patients the first time they come into the room.” Many initially said this was unimportant, as hospital staff wear name tags. However, others said it was very important and was an issue of safety. They indicated that lots of different staff come into a patient's room for different reasons; if patients do not know who they are and why they are there, patients might get the wrong medicine or an unnecessary treatment. Participants also thought introductions were a necessary part of coordination of care and communication, patients need to know the names of the doctors and nurses treating them so they can refer to them in subsequent discussions with other doctors or nurses about their care.

Hospital Staff Responsiveness to Patient Needs

Within this domain, there was considerable consensus that an item on having the call button answered as soon as wanted was extremely important. Many people noted that if patients are using the call button, it is for something important. As one said, “When I push the call button, I'm not asking for the TV guide. I need help.” Several respondents noted that this single item subsumed others in the list, such as getting help getting to the bathroom or keeping clean, as patients were likely to use the call button to get assistance with those matters.

It was critically important to participants in both rounds of groups to have immediate assistance when they needed to use the bathroom or a bedpan. Several noted that because of childbirth, surgery, or medications, a person's need to use the bathroom immediately may increase. Respondents also thought having help with toileting was a matter of “personal dignity.”

Though many also considered the item about staff protecting patients' privacy important, they recognized the fact that staff may not have complete control over this in the hospital environment:

Privacy isn't as important to me as the service is. I can get over being embarrassed but I can't get over bad service.

Participants did, however, identify certain situations when privacy was especially critical: when issues of life and death were being discussed; during labor and childbirth; and when someone was getting a sponge bath.

In the second round of groups, getting assistance with a bath or shower as soon as wanted was considerably less important than assistance with toileting. One respondent said “waiting 10 minutes for a shower wouldn't kill” him, while several other respondents mentioned that their family members were able to help them with their personal bathing. On the other hand, being able to be personally clean was of considerable importance to many participants in the first round of groups.

Hospital Staff Helping Patients Manage Pain

Although participants did not often bring up pain management as an indicator of quality in open-ended discussions, they nevertheless responded positively to the specific items included in this survey domain. When discussing these items during the first round of focus groups, participants had a difficult time choosing a single item they preferred above the others. In some groups, they leaned to the most general item, whether your pain is well controlled, while other groups preferred the item stating that doctors, nurses, and other staff do everything they can to help patients with their pain.

Several aspects of pain and pain management in the hospital setting emerged in the group discussions. Several groups mentioned concerns of becoming “dependent [on] or acclimated [to]” or even “hooked [on]” pain medication, making them nervous about the item “Doctors, nurses, and other hospital staff respond quickly when patients ask for pain medicine.” While most people think of pain management in terms of medication, a few participants did mention nonpharmaceutical ways to alleviate pain such as the use of extra pillows. Several respondents mentioned the notion that pain can be an indicator of other problems, and that pain can exacerbate existing problems. Others explained that to them pain meant something was seriously wrong with their bodies.

Avoiding Problems with Medication

During the first round of focus groups, participants had widely differing ideas about which items in this domain were most important to them. Some wanted to make sure staff checked on allergies to medications, some were concerned that they be told the side effects of any new medications, and others wanted someone to explain why new medications were being given. It was hard for participants to reach consensus about which one or two were most important.

In the second round, individual participants recognized that problems with medication can be serious, i.e., they “can kill you,” and several told stories about receiving, or almost receiving, the wrong medication.

Nevertheless, the items in this domain were, in general, less highly rated, perhaps because none of them speaks specifically to medication errors. Some participants objected to the item about staff checking for allergies before a new medication was given, because they thought that information should be readily available in the chart. Others noted that it might be annoying to be asked about allergies by each different staff member who might give you your medications, but that it was preferable to being given a medication to which you were indeed allergic. Across these items, getting information about new medications before they were actually administered was considered critical.

Some participants were particularly interested in knowing whether they were getting generic drugs, which they considered less desirable than brand name medications. Respondents in the maternity group were especially sensitive to drug-related problems that would affect their ability to breast feed.

Hospital Room Comfort and Cleanliness

In this domain, there was a high degree of consensus that the most important item was “The hospital room and bathroom are kept clean.” Cleanliness is viewed as important both in terms of elimination of bacteria that could lead to infection and because it says something about the competence and commitment of the hospital staff as a whole. One respondent rhetorically asked, “Who wants a dirty bathroom?” while another wondered aloud, “If the room is dirty, how dirty are your caretakers?” Yet another noted that if there is dirt you can see, imagine the invisible bacteria and germs you cannot see. One member of a maternity group said “You have a baby, you don't want him picking up germs.”

In comparison, concerns about quiet and a comfortable temperature pale. Respondents expressed concern over the room temperature, but offered solutions like access to blankets, comforters, and fans rather than being in control of the temperature. Many respondents recognized that the room temperature may be set for the entire floor or wing of the hospital. The level of importance on having quiet at night varied by group, but one point on which they all agreed was that unnecessary noise was not acceptable; for example, noise from a television or radio to which a staff member may be listening during the night shift. Apart from examples such as these, participants recognized that hospitals are busy and sometimes loud places. As one respondent put it, he was in a hospital “and not a hotel.”

How Well Does the Hospital Staff Help Patients Avoid Problems after They Leave the Hospital?

In discussing this domain, some participants in each group wondered why a hospital would care about problems experienced by patients after discharge. Some participants saw a hospital's inquiries about home care or identification of helpful community resources as simply a reassuring sign of the institution's commitment to healing. However, situations described by other group participants nearly always convinced others of the value of discharge information and the role that the hospital could or should take:

I think it's very important to discuss aftercare. My wife was in [X Hospital]. She had flesh-eating disease. And while she was there, they started to have a staph infection going through the hospital. So they told my wife …‘You got to go home. And we're going to line up an intern to come out and visit you.’ Well, the next day, the intern did show up … And it so happened that the thing had started growing again, you could literally see it grow before your eyes … and he called the ambulance and they rushed her back to the hospital. She was in surgery for six hours. If they didn't have that technical aftercare, who knows, she may have passed away.

Getting information in writing before going home was seen as very useful. As one participant explained, “checking out is overwhelming,” and being able to refer to the written instructions later is helpful. Of the four items in this domain, the ones mostly commonly highlighted as very important, across the groups, were getting written information about activities they can and cannot do after leaving the hospital, and getting such information about symptoms or health problems to look out for.

Domains Viewed as Most Valuable

During the second round of 10 groups, after participants indicated using the marks described above the importance of each domain as an indicator of quality, moderators asked participants which domains they would choose if they could only receive information about two of the seven. Across these 10 groups, participants found it very difficult to limit their choices. Many participants jokingly pleaded to be able to choose “at least three out of the seven!” The results of this forced choice exercise indicate that participants favored these four categories:

• Doctor communication skills

• Responsiveness of hospital staff

• Comfort and cleanliness of the hospital environment

• Nurse and hospital staff communication skills

over these three:

• Pain management

• Avoiding problems with medication

• Avoiding problems after leaving the hospital

Findings from Paper and Pencil Exercise

As noted in “Methods,” at the outset of the second round of focus groups, participants were asked to place stars next to items from the CAHPS Hospital Survey (excluding overall ratings) that they thought were so important that they would consider changing hospitals in response to them, and lines through items they thought were not at all important. As participants varied in terms of whether their health care coverage would limit their hospital choice, we explicitly requested participants to put aside the nature of their current coverage and complete the exercise under the assumption that they could switch hospitals if they chose.

Participants found a high proportion of survey items to be highly important, and a low proportion not at all important. Taking as a denominator the number of opportunities to rate an item (94 participants times 28 items, or 2,632), we can calculate three rates. The first is the rate at which items were starred, which is 84 percent. The second is the rate at which lines were drawn through items, which is 3 percent. The third rate is that at which no response was given, indicating some, but not a great, level of importance, which is 13 percent. In addition, there were five items that every focus group participant thought was at least somewhat important, including (1) “doctors treat patients with courtesy and respect”; (2) “doctors explain things in a way patients can understand”; (3) “doctors, nurses and other hospital staff do everything they can to help patients with their pain”; (4) “the hospital room and bathroom are kept clean”; and (5) “patients get information in writing about symptoms or health problems to look out for after they leave the hospital.”

DISCUSSION AND CONCLUSIONS

Consumer Interest in Hospital Quality and Quality Information

The experience reported here indicates that it is quite easy to engage consumers of all types—younger or older, Medicare or non-Medicare, urgent or elective care seekers—in discussions about hospital quality. All the groups were able to provide a wide range of responses to the open-ended question: “What factors characterize high- and low-quality care in hospitals?” Discussions were quite lively and participants did not shy away from asking questions of other group members or challenging them if they disagreed with a point of view.

One reason for the high degree of engagement may be that, compared with other health care entities (such as health plans), consumers find it easy to attribute both positive and negative hospital events to the hospital itself (the one exception being their circumstances once they get home). For example, people are often reluctant to hold health plans responsible for ensuring that preventive care (like annual mammograms or vaccinations for children) is delivered (McGee, Sofaer, and Kreling 1996). In contrast, participants in our groups found it very easy to hold the hospital responsible for the behavior of those who work there. That is, if bedrooms and bathrooms are not kept clean, if nurses fail to deliver medication on time, if staff physicians rush through examinations or explanations, then someone in the hospital hierarchy is responsible for seeing that the ensuing problems are either corrected or dealt with.

The most important finding from these groups is that those who participated—both former patients and those who may be patients in the future—want information about hospital quality. In more than one group, participants asked where they could get this information “right now.” In fact, at least one participant realized that the groups themselves were part of an effort to provide this information, saying in response to the same question, “They're going to create it. That's why they're spending big bucks to figure out what to say in it.” This desire for comparative information about hospital quality, especially in this group of people who chose to participate in the focus groups, may be another reason we had no difficulty in engaging people in a substantive conversation on what to a lay person may be viewed as a relatively arcane matter—selecting items for inclusion in a government survey.

Consumer Views on Key Aspects of Hospital Quality Assessed in the CAHPS Hospital Survey

As is the case with defining quality in other health delivery settings, people see communication as a cornerstone of high quality care (Inui and Carter 1985; Roter, Hall, and Katz 1987; Safran et al. 1998; Darby 2002; Teutsch 2003). Participants understood that appropriate and timely communication among and between patients, physicians, nurses, and other hospital staff and caregivers can both maximize the likelihood that patient needs will be met and greatly reduce the potential for errors related to diagnosis, medication, and treatment. When communication among all parties works the way it should, participants feel that they are respected and “heard.” This in turn lays the foundation for the best possible clinical outcome for the patient.

In addition to good communication, these former patients clearly viewed responsiveness of staff and a clean, well-kept hospital environment as key elements of high-quality care. While participants seemed aware that, as one said, “this is a hospital, not a hotel” and understood that they were there to be cared for but not catered to, they thought it was fair to demand responsiveness to requests for medication, assistance with bathing, and most of all, assistance with toileting. From a provider's point of view, the clinical skill of hospital staff or cutting edge technology may be the factors that connote high quality. Patients recognize the importance of those factors, as revealed in their responses to our initial open-ended question about hospital quality, but they also believe that even the most technically competent hospital staff need to be organized or empathetic enough to respond quickly to a request for assistance, especially with a function as basic as toileting. Similarly, patients view a clean and well-kept environment as a basic and essential element of a high-quality hospital.

Consistency across Participants from Diverse Backgrounds

As described in “Methods,” we went some effort to recruit a highly diverse group of people for these discussions. Participants varied in location, age, gender, race/ethnicity, education level, type of health care coverage, and type of hospitalization. We found remarkable consistency in response across groups. This may imply that there is substantial agreement among different segments of the population concerning the most important aspects of hospital quality. If true, this consistency would limit the extent to which reports on scores from the CAHPS Hospital Survey would need to be extensively tailored to different population groups.1

The Need for Further Research and Testing of Comparative Hospital Quality Reports

Previous research makes clear that, while these groups have provided considerable guidance for both survey developers and quality reporters, further formative and summative research will be needed. Formative research will be needed to develop the most effective possible report for consumers; summative research will be needed to confirm that, in fact, consumers will access, review, understand, and use a report on these survey-based measures.

As noted earlier, although participants were clearly both willing and able to engage in the focus group exercises, these activities represented a cognitive burden requiring concentration and thought. This has relevance to the task of developing effective hospital quality reports using the CAHPS Hospital Survey data. Specifically, the focus group experience underscores the importance of iterative testing of different elements of any comparative quality report, as well as testing the entire report in draft form (McGee et al. 1999; Barr et al. 2002). Presenting draft data displays and chunks of texts to potential users, asking for their feedback, and incorporating changes based on this information is the most certain path to producing a report that people will understand and use (Kanouse, Spranca, and Vaiana 2004).

Presenting the importance of particular measures—or more generally the importance of hospital quality—in language familiar and accessible to potential users is one path to making the report “click” with users. We noticed that one participant's explanation of the value of a piece of information often convinced another to change his or her mind. This suggests that verbatim quotes from group participants, if included in reports, may quickly communicate the value of a piece of information to a potential user.

One comment that frequently emerges in the debate about the value of quality information for consumers is this: Even if consumers say that they want quality information, what guarantee is there that they will use it? Few people have embraced and immediately used quality information about, for example, health plans (Marshall et al. 2000; Schauffler and Mordavsky 2001). There is evidence, however, that consumers do not consider health plans as having a significant impact on the quality of health care they receive. The fact that our focus group participants found it so easy to come up with examples that supported their assessment of an item's value and found it difficult to eliminate items and domains may, however, signify that consumers will be more interested in comparative data about hospitals than they have been in health plan reports. On the other hand, some consumers may not perceive that they have a choice of hospital, either because there is only one facility close to home, or because their health plan limits their choices, or because their physician(s) are affiliated with only one hospital. However, public reports of comparative hospital quality may have positive effects on hospital quality itself through multiple pathways, rather than the single pathway of consumer choice, in the aggregate, shifting “market share” from “worse” to “better” institutions. Even when consumers do not believe they have much choice, or even any choice, of hospital they can still use the findings in public reports to begin conversations with their physicians about the relative quality of different hospitals and even ask their physician to do whatever they can to ensure that their personal hospital experience will be a good one.

In sum, much remains to be learned about whether and how consumers will use comparative hospital quality information, especially information drawn from patient experience surveys like the CAHPS Hospital Survey. It will be imperative to conduct rigorous studies of who does and does not look at, understand and use the data from this survey that will be placed on the Hospital Compare website, how they use it, and what consequences that has both for hospital quality improvement and ongoing patient experiences. These studies can only be conducted, however, when data that have been vetted to be meaningful and comprehensible to consumers are presented to them by a credible agency, using evidence based report formats, and effectively promoted to ensure people are aware of the availability of the information and the potential benefits of using it (Sofaer 1997; Shaller et al. 2003).