How Health Care Organizations Are Using Data on Patients' Race and Ethnicity to Improve Quality of Care

Abstract

Context: Racial and ethnic disparities in the quality of health care are well documented in the U.S. health care system. Reducing these disparities requires action by health care organizations. Collecting accurate data from patients about their race and ethnicity is an essential first step for health care organizations to take such action, but these data are not systematically collected and used for quality improvement purposes in the United States. This study explores the challenges encountered by health care organizations that attempted to collect and use these data to reduce disparities.

Methods: Purposive sampling was used to identify eight health care organizations that collected race and ethnicity data to measure and reduce disparities in the quality and outcomes of health care. Staff, including senior managers and data analysts, were interviewed at each site, using a semi-structured interview format about the following themes: the challenges of collecting and collating accurate data from patients, how organizations defined a disparity and analyzed data, and the impact and uses of their findings.

Findings: To collect accurate self-reported data on race and ethnicity from patients, most organizations had upgraded or modified their IT systems to capture data and trained staff to collect and input these data from patients. By stratifying nationally validated indicators of quality for hospitals and ambulatory care by race and ethnicity, most organizations had then used these data to identify disparities in the quality of care. In this process, organizations were taking different approaches to defining and measuring disparities. Through these various methods, all organizations had found some disparities, and some had invested in interventions designed to address them, such as extra staff, extended hours, or services in new locations.

Conclusion: If policymakers wish to hold health care organizations accountable for disparities in the quality of the care they deliver, common standards will be needed for organizations’ data measurement, analysis, and use to guide systematic analysis and robust investment in potential solutions to reduce and eliminate disparities.

For several decades, researchers have documented disparities in the quality of health care received by minority patients in the United States (AHRQ 2009; Smedley, Stith, and Nelson 2003). Although researchers define disparities in different ways, a common theme is that they are unjust differences, not determined by clinical need or patients’ preferences (Smedley, Stith, and Nelson 2003), and can be remedied by actions at several levels of the health care system (King et al. 2008; Smedley, Stith, and Nelson 2003). Health care organizations’ collection of accurate race and ethnicity data from patients is an essential first step to identify and eradicate disparities at this level (Ayanian 2008; King et al. 2008; Lurie and Fremont 2006; Nerenz 2005).

Many regulations govern the collection and reporting of race and ethnicity data for Medicare and other federally funded programs (Youdelman and Hitov 2001), but it is not clear how consistently the data have been collected and whether the health care organizations themselves use the information or simply report it, as required. In 2009 the Institute of Medicine (IOM) strongly recommended consistent standards for collecting and reporting data on patients’ race, ethnicity, and language (Ulmer, McFadden, and Nerenz 2009). In addition, the recent federal reforms of health care and health information technology (IT) provide for widening the data collection by extending investment in electronic health records to health care providers that receive public funds (Siegel and Nolan 2009). To date, however, only pilot programs and limited implementation of data collection have been part of quality improvement initiatives for various health care settings in different states and cities (Jones, Trivedi, and Ayanian 2010; Jorgensen et al. 2010; Lurie et al. 2008; Sequist et al. 2006; Siegel et al. 2007; Weinick et al. 2007).

Health care organizations that collect race and ethnicity data from their patients are more likely than those that do not collect such information to try to reduce disparities and improve quality of care (Hasnain-Wynia and Baker 2006). Recent research studies, however, suggest that organizations that both collect and use patients’ race and ethnicity data are still relatively uncommon (Hasnain-Wynia and Baker 2006; Lurie et al. 2008; Weinick et al. 2007; Weinick, Flaherty, and Bristol 2008; Wilson-Stronks and Galvez 2007). Local and national best-practice guidance for data collection methods for hospitals and other organizations have been developed (Hasnain-Wynia et al. 2007; U.S. Department of Health and Human Services 2001; Weinick, Flaherty, and Bristol 2008), but there is an absence of consistency about which categories of data are recommended for collection (Ulmer, McFadden, and Nerenz 2009).

Although some recent research has focused on identifying and overcoming the challenges of collecting race and ethnicity data, much less attention has been paid to how organizations use this information. Among the problems identified are too small a sample of minority patients to generate meaningful comparisons or overly general categories to adequately capture a broad range of racial and ethnic groups within a patient population (Nerenz, Hunt, and Escarce 2006). There is concern, too, about potentially negative consequences of linking racial and ethnic data to quality improvement initiatives that use public reports of performance (Werner, Asch, and Polsky 2005) or pay-for-performance programs (Casalino et al. 2007), because this link may financially penalize health care organizations that disproportionately care for minority populations.

Our qualitative study explored the experiences of health care organizations that had collected patients’ race, ethnicity, and language data. Specifically, we examined how organizations used these data to design and implement interventions to improve quality and reduce disparities. These health care organizations included U.S. hospitals, community health centers, and health plans. We focused on the experience of organizations that could be considered atypical (since rigorously collecting and using this information is uncommon), and our aim was to better understand the potential benefits, costs, and challenges that might be encountered if collecting these data becomes a more widespread practice in the United States, through mandates, financial incentives, incorporation into accreditation criteria, or voluntary quality standards.

Organizations identify and address disparities using an approach that is conceptually similar to the framework outlined by Kilbourne and colleagues in their description of research on disparities (Kilbourne et al. 2006). They identified an initial “detection phase” when organizations or researchers define disparities, identify populations, and develop measures before beginning an intermediate phase of understanding the root causes and then concluding with a phase of designing and implementing interventions to reduce disparities. We were particularly interested in finding out how organizations defined and measured disparities and, when stratified by race and ethnicity, what indicators of performance or quality they considered most useful. We also explored the subsequent use of these data to understand the causes of disparities and to design interventions to address them. Our approach was based broadly on the Institute of Medicine's definition of health care disparities, that a disparity is a difference in the quality of care not explained by access, clinical needs, preferences, or appropriateness of care (Smedley, Stith, and Nelson 2003).

Methods

Our case study approach used qualitative methods, as they are well suited to identify and describe emerging themes in settings on which the empirical or theoretical work is limited (Miles and Huberman 1994). This approach also allowed us to explore an event or topic from the perspectives of different actors in complex organizations (Sofaer 1999). An in-depth case study approach enables the examination of an individual organization to illuminate key factors (Ragin 1999), rather than attempting to generalize from the individual case to the general population.

Our study used purposive sampling (Patton 2002), in which we chose organizations that had collected self-reported data on race and ethnicity from patients and had stratified at least one area of performance by race and ethnicity. We then generated the sample using a snowballing technique. Our content experts were identified from peer-reviewed literature and other published reports, and they were asked to recommend organizations that they believed would meet the two preceding inclusion criteria. Thorlby contacted these organizations by email or telephone to find out whether they met the criteria and were willing to participate in the study. The research team used these potential case study sites to compile a list of various organizational types and geographic locations. Last, we invited them by email or telephone to participate, on the condition that neither they nor their staff would be publicly identified.

At each site, we asked a range of individuals, including senior managers, senior clinicians, and data analysts, for interviews (up to five per site). Thorlby conducted the interviews in person using a semistructured interview guide. The interviews generally lasted for forty-five to sixty minutes and were taped using a digital audio recorder, transcribed, and coded. Thorlby created the first set of codes using a framework of themes developed earlier from the interview questions and from a content analysis of themes emerging from the first five informant interviews (Miles and Huberman 1994; Ritchie and Spencer 1994). Ayanian and Jorgensen independently applied these codes to another sample of five interview transcripts, which produced a revised set of codes, along with broad thematic categories, which Thorlby then reapplied to the existing transcripts and applied to the remaining transcripts. The study protocol was approved by the Harvard Medical School Committee on Human Studies.

Findings

Sample Characteristics

Eight of the eleven organizations we contacted agreed to participate in the study: three hospitals, three health plans, and two community health centers in six U.S. states, which are described in Table 1. To preserve the organizations’ anonymity, we did not name the state but did indicate the region where the organization was located or had its headquarters.

TABLE 1.

The Eight Organizations Participating in Our Study

Case 1	Hospital, nonprofit, with affiliated community health centers (New England)
Case 2	Hospital system, nonprofit (Southwest)
Case 3	Pediatric hospital, nonprofit (West Coast)
Case 4	Commercial health plan affiliated with hospital system (Midwest)
Case 5	Commercial health plan with predominantly regional membership (New England)
Case 6	Commercial health plan with regional and national membership (Mid-Atlantic)
Case 7	Federally qualified community health center (New England)
Case 8	Network of federally qualified community health centers (Midwest)

Although these organizations differed in size, structure, and available resources, some common themes emerged from their experience of collecting and analyzing race and ethnicity data. We grouped the interview findings into the following themes: the mechanics of data collection (categories of data and methods of collection); the purpose of data analyses; the approach to defining and measuring disparities using these data; the results of the analyses and the organizations’ responses to the findings; reflections on the potential benefits to the organizations of collecting and analyzing the data; and, finally, views of the future direction of policy with regard to the collection of racial and ethnic data.

Mechanics of Data Collection

All the organizations recognized that their efforts to reduce disparities had to be built on a firm foundation of accurate information. For example, the interviewees knew that they had to collect self-reported race and ethnicity directly from the patients. Three of them explicitly mentioned using the Hospital Research and Educational Trust (HRET) toolkit (Hasnain-Wynia et al. 2007), and two referred to the national standards on Cultural and Linguistically Appropriate Services (CLAS) (U.S. Department of Health and Human Services 2001). As we will show, however, collecting accurate information posed particular challenges for health plans and community health centers.

Hospitals and Community Health Centers

All the organizations reported problems with obtaining self-reported data, but the hospitals were able to collect and analyze the information more easily, perhaps because they had more resources to draw on than did the community health centers. Senior staff at all three hospitals explained that even though patients were accustomed to being asked for personal data (e.g., their age or insurance status), they were aware that patients were not asked to report their race or ethnicity. Staff therefore had often assigned patients to a category by means of “visual observation,” which is considered highly unreliable. Two of the hospitals (cases 1 and 3) had designed a script for asking patients these questions and had the admitting and registration staff attend training programs on this subject.

All the organizations needed to change their IT systems to capture the categories of race and ethnicity, add them to the patients’ records, and link them to a range of quality measures. Hospital 1 tried to ensure the quality of the information by reviewing samples of telephone interactions between staff and patients. Hospital 2 reported that it was in the process of training its staff and modifying its IT system. Managers and analysts expressed some lingering doubts about whether the patients had always specified their race or ethnicity or whether a staff member had merely inferred it. Managers in two of the hospitals (cases 1 and 3) noted that feedback from staff revealed their discomfort with asking patients about their race and ethnicity because the staff members were concerned about negative reactions from patients. Persistence with training and clarity about the expected use of the data had largely assuaged these concerns.

Although subject to federal data-reporting requirements regarding race and ethnicity, the community health centers in the study had not collected this information directly from patients in the past. As the hospitals had done, they had trained staff to enter the information collected from the patients into the system so that it could be linked with clinical records. The staff at one community health center (case 7) regarded their organization as “cutting edge” compared with other community health centers because they had had an electronic medical record since 2005. Nonetheless, they found collecting race and ethnicity data to be very time-consuming because of the time needed to interact with patients and input the data into the computer system. In fact, because of a shortage of resources, one community health center network (case 8) had used a summer intern to design and run the training.

Health Plans

Managers at each of the three health plans reported major challenges in collecting data directly from patients. All three health plans had collected only a limited amount of information directly from members because of their reluctance to ask for it at the time of enrollment. The health plans said this was because their members were sometimes worried that the information would be used to refuse or limit coverage.

There is a suspicion among those who most need to be identified that there's something untoward we'd be doing with the data. So there's a trust factor here. There's certainly a lot of talk about the lasting impression of the Tuskegee experiment in this country, that many blacks worry that if they share that information, they'll be experimented on, that it will be used in a way that's not to their advantage. (manager, health plan)

All three organizations had supplemented the little information collected directly from patients, either by using proxy methods to estimate the probability that a person belonged to a racial or ethnic group based on their zip code or surname (cases 5 and 6) or by cooperating with affiliated providers and employers (case 4). A manager at health plan 5 explained that these methods were not as good as self-reported data but did allow the health plan to identify the patients’ likely race or ethnicity and thereby target interventions geographically (based on those patients’ providers), rather than target the patients directly.

Racial, Ethnic, and Socioeconomic Categories

Our study found some variation in the categories of data collected from the organizations. At a minimum, all organizations used the Office of Management and Budget (OMB)'s five racial categories and dichotomous definition of ethnicity as “Hispanic or non-Hispanic.” Three of the organizations (a hospital, a health plan, and a community health center) reported that the OMB's definition of ethnicity (Hispanic or non-Hispanic) caused problems for some of their patients, who often were confused about having to report both their race and their ethnicity. For example, a manager at a community health center (case 7) observed that people with a Hispanic background answered yes to the ethnicity question but then refused to assign themselves to a racial category (because they considered Hispanic to be their race as well). This problem was resolved by changing the order of the questions, so that race was asked before ethnicity. Case 8 (a community health center) had particular difficulties with those patients from outside the Americas, who did not understand the word Hispanic. The staff responded by offering additional translations of the word. Two organizations also expanded their list of ethnic categories for patients to choose from, as they felt that the existing options were too narrow to reflect their rapidly changing populations. One of the organizations (case 1) found that patients did not always understand even these larger categories. One manager cited a debate over patients who listed “Jewish” as their ethnicity, with some in the organization arguing that this should be classified as a religion rather than an ethnicity. But the patient's own choice prevailed: “The thing is with self-reporting, that's what you get, is how people report, and there's no right or wrong” (senior nurse manager, hospital).

Language and socioeconomic data were less consistently collected. Case 1 had gathered data on race, an expanded set of thirty-three ethnic categories, language, and educational attainment, as required by a recent state mandate. Case 1 described its language data collection as needing more work, because its preexisting categories needed constant expansion as new languages surfaced. Case 1 also revealed that requests for information about educational attainment had met with some resistance from patients and that the available categories were confusing for people who had been educated abroad. Many of the other cases used much simpler “English / non-English language” categories and information about the payer (e.g., Medicaid) as a proxy for socioeconomic status. A manager at a hospital (case 1) talked about possibly being able to collect other data at registration, for example, whether a person was homeless or had mental health or substance abuse diagnoses, but none of the organizations had routinely tried to collect or analyze data about these other characteristics.

As we pointed out earlier, the health plan cases supplemented the limited data that they had collected directly from patients with proxy estimations of educational attainment and income based on the patients’ zip codes. Case 4 (a health plan) had access to race data through its enrollees’ employers, and it also gathered information about whether or not an enrollee belonged to a union.

Data Analysis

The organizations described the two main ways in which they used the data on patients’ race and ethnicity. At the most basic level, this information enabled the organizations to know exactly who was using their services: “to educate our leadership as to who our patients were, because heretofore, they really weren't aware” (clinical director, hospital). In some cases, particularly the larger hospitals in urban areas with rapidly changing immigrant populations, these data allowed them to gauge the size of these ethnic groups, about which they had previously had only hazy knowledge. Information about language was particularly useful, as it enabled the organizations to better estimate the demand for translation services.

The most common use of the data was establishing the presence or absence of disparities in the quality of care. Sometimes this use was prompted by anecdotal reports of dissatisfaction with services from a particular group, for example: “Our starting place really was with the sickle cell community coming to us and saying—you are not serving us well; our children come in; they're in pain; you're slow to respond; you don't listen” (director of nursing, hospital). Backing up these anecdotal reports with data was seen as a way to make decision making more robust, “to deal with things less on an opinion-based approach [and] more on an analytical based approach” (CEO, hospital).

More often, individuals within the organizations were familiar with the research literature on health disparities and wanted to find out whether those disparities were evident in their own organizations. Indeed, most organizations had begun to use their race and ethnicity data for this purpose, by stratifying those performance indicators by race/ethnicity that the organizations already were using to monitor their overall quality of care. The better-resourced organizations (e.g., all three hospitals) had stratified many more quality indicators by race and ethnicity than had the community health centers, which had analyzed only one or two clinical areas.

Defining and Interpreting Disparities

Our respondents offered different definitions of disparity. Some understood it primarily as a disparity in the quality of care that their organization could reverse. The Institute of Medicine's review of disparities (Smedley, Stith, and Nelson 2003) was frequently mentioned in this regard:

We'd gone out and started reading. So in 2002, the Institute of Medicine published Unequal Treatment, and that was around the same time that we first presented this data to our management. So we couched it within that framework: We said, the IOM just published this report. They've said that even among people with insurance, this is a problem. (senior manager, health plan)

Although our interviewees were familiar with the Institute of Medicine's review, none of them cited its full definition of a disparity as a difference not explained by access, clinical needs, appropriateness, or preferences. Instead, organizations varied considerably in how they decided when a difference became a disparity, how they interpreted the causes of disparities, and whether the disparity required action by the organization. Even those in the same organization frequently disagreed on what was meant by a disparity. For example, one senior manager of a hospital system believed that disparities were the product of the uninsured's poor access to care and believed that the disparities in the quality of care would disappear once the patients had entered the hospital system, arguing that the organization should therefore focus on improving access. “So when one thinks about the health disparities issues, the biggest opportunities are essentially—they're not under the lamppost. They're not within our formal systems of care within this organization, for the most part” (senior manager, hospital).

By contrast, an administrator in the same hospital believed that disparities could arise from the interaction between patients and clinicians, leading to poorer outcomes for minority patients: “Just kind of a lack of a cultural sensitivity that different ethnic groups will present to physicians with phrasing their complaints in different terminologies [so] that unless you're culturally attuned to how different ethnic groups express themselves, you don't pick up what their complaint is” (administrator, hospital). There were other examples of differing views about the likely cause of disparities. For example, a medical director of a community health center (case 7) believed that two groups’ differences in hypertension outcomes were likely to have some sort of biological cause, whereas her managerial colleagues thought that the regularity of visits to the clinic or compliance with medication were more likely explanations.

Selection of Quality Indicators

Many interviewees described disparities as unwarranted differences in the quality of care received by different groups that could be addressed by the organization itself. Thus they believed that standard process-of-care measures were the most valuable in this regard and noted that there should be no variation in the administration of recommended tests or procedures according to racial or ethnic group. The community health centers, hospitals with primary care facilities, and health plans also considered some of the ambulatory outcome measures, such as blood pressure control, to be useful.

The three hospitals had selected the “core” hospital quality measures issued by the accrediting body, the Joint Commission. These included measures of performance relating to acute myocardial infarction, heart failure, pneumonia, and surgical infections. The health plans had used performance measures from the Healthcare Effectiveness Data and Information Set (HEDIS) pertaining to the care of patients with chronic disease, immunization, and screening rates.

The community health centers had stratified their measures by race and ethnicity in regard to childhood immunization rates and also the management and intermediate outcomes of diabetes. All the hospitals (cases 1, 2, and 3) also had stratified the findings of their patient experience surveys by race and ethnicity (and, for case 3, language as well).

Most of the participants stated that the choice of performance indicators was based on nationally recognized measures that were important to the leadership: “Well, I liked HEDIS because HEDIS was evidence based, and it was a standardized way of measuring health care outcomes and having a benchmark” (senior manager, health plan).

Although most of the organizations agreed on the desire to locate and resolve disparities, closer questioning revealed a very heterogeneous approach to analyzing them. Some organizations always used the “white group” as their comparator (cases 1 and 3), while others, such as case 5, used the best-performing group as the comparator, regardless of whether it was the white group. All the cases reported difficulties with small numbers; for example, cases 1, 2, and 3 had published analyses in which the racial and ethnic categories had been merged into a single “nonwhite” category, sometimes containing more than one year's data, in order to generate sufficiently large sample sizes. One of the respondents expressed some misgivings about aggregating the data into these large categories, fearing that it might lead them to “miss something” by treating different groups the same.

The organizations also had different approaches to establishing whether the magnitude of a difference made it a disparity, which is summarized in Table 2.

TABLE 2.

Establishing Magnitudes of Difference That Created a Disparity

Case Number	Type of Organization	Magnitude of Difference	Adjustment for Mediating Factors	Action Taken as a Result of Analysis
1	Hospital + health centers	Any statistically significant difference	No	Investment in enhanced ambulatory care services
2	Hospital system	Variable	No	Targeted screening campaigns in affiliated ambulatory care services
3	Hospital system	Variable	No	Further research with patient groups
4	Health plan	>10 percentage point difference if statistically significant	No	Investment in improved access for patients
5	Health plan	≥7 percentage point difference regardless of significance	No	Investment in improved outreach and access for patients
6	Health plan	Any statistically significant difference	Yes, adjustment using income and other variables	Targeted screening campaigns, cultural competency training for staff
7	CHC	Any statistically significant difference	No	Further research to understand disparity
8	CHC	Any statistically significant difference	No	Further research in progress

Four organizations (two community health centers, a hospital, and a health plan) treated any statistically significant difference between two groups as a disparity. There were some exceptions. Case 4 (a health plan) took a 10 percentage point difference between two groups as evidence of a disparity, while case 5 (a health plan) decided that a difference equal to or greater than 7 percentage points was a cause for concern, regardless of statistical significance. Case 5 settled on this figure through negotiation with analysts and managers, based partly on scrutiny of the literature and partly on pragmatic considerations: “If you use too small a difference, then you may find a disparity everywhere, in any direction. And you can't work on everything all at once” (manager, health plan).

The other organizations did not agree on what magnitude of disparity was a cause for concern; their approaches varied depending on what was being observed. In cases 2 and 3 (both hospitals), the respondents reported a more iterative process to determine whether a difference was a disparity, particularly when the numbers were very small or when larger data sets showed a small but statistically significant difference within an organization with a good performance overall. Interviewees at both organizations described how they distinguished between a statistically significant difference and a “meaningful” difference:

We had just such huge data sets that at that point, it became important to distinguish between statistical significance and meaningful difference for a particular quality indicator. Maybe you had African Americans at 92 percent and whites at 94 percent; it was like a p value of less than .05. So if that was your rule of thumb, is that something you really want to focus your efforts on? Something where one subpopulation is getting it 94 percent of the time and the other is getting it 92 percent of the time? Probably not. (data analyst, hospital)

The case study sites also had mixed views of whether the data should be adjusted for confounding factors, such as income, education, clinical need, or patient preference. While some participants thought that ideally the analysis should take into account these other variables, only one organization (case 6, health plan) used multiple logistic regression to isolate the “race effect” from the influence of socioeconomic status.

I think you're almost guaranteed to find disparities if you don't adjust, simply because people with lower socioeconomic status are more likely to have inappropriate care. So you want to take that factor out, and we want to get down to fact. Is it, basically, the color of a person's skin that is driving disparities? (data analyst, health plan)

Organizations offered different explanations for not adjusting their data in this way. The analysts in case 2 (hospital) stated that they did not have enough information about income or education to allow that kind of analysis. The community health centers focused on the practical challenge of analytic capability but also argued that socioeconomic deprivation affected all their patients equally. “Well, because most of our patients are—90 percent of them—are living under 200 percent of the poverty level. So you could never stratify by socioeconomic [status] in this organization. You don't have the higher-income people and the middle-income people. We don't have them” (senior manager, community health center).

The data analysts from health plans and hospitals also mentioned the conceptual challenges of presenting adjusted data, in the form of odds ratios, to committees of senior management, who found them too complex. One hospital system analyst (case 2) explained that they had originally started with “a much more rigorous approach” but had abandoned this approach because of the difficulties presenting information to managers who were unaccustomed to seeing data in this format. Several interviewees pointed out that they were competing with other organizational objectives, such as patient safety, for management's attention and that these senior managers “have attention only for quick-hitting messages” (manager, hospital).

In these cases, the analysts and managers had tried to compensate for the lack of statistical adjustment by presenting bivariate data analyses in segments, first analyzed by race and then by language preference, gender, and payer status. In only one case (hospital 1) did they argue that the data might be salient without being adjusted for confounders:

I think our view here is that we want to find differences, and basically, it matters to people's lives. Yes, there may be certain factors that disproportionately affect one population versus the other. For example, language: we found disparities in one of the populations that may have limited English proficiency. Some could argue, well, if they spoke English, maybe there wouldn't be a disparity. Well, that's a research question, not a reality question, and I think we need to do what we can to make sure that those people are getting the services that they need. (senior manager, hospital)

Results of Data Analysis

The three hospitals (cases 1, 2, and 3) had not found any disparities in their core measures (according to their rules of measurement), although case 1 had discovered disparities in two HEDIS measures (diabetes control and colorectal cancer screening) in its affiliated primary care clinics, and case 2 had found a disparity in colorectal cancer screening in its primary care physician network. The three health plans (cases 4, 5, and 6) detected disparities for only a minority of HEDIS measures (2 of 9, 2 of 20, and 5 of 25 measures, respectively). The community health centers had analyzed only a few indicators. Case 7 had found disparities in blood pressure control among its non-Hispanic black patients, while case 8 had found disparities in the number of patients receiving immunizations.

The reaction to the presence or absence of disparities varied according to the type of organization. Some hospital-based interviewees expressed relief and surprise at the absence of disparities in their hospital. But they admitted that compliance with these core measures were the focus of a great deal of organizational quality improvement effort and that their hospitals already were performing well by national standards.

The interviewees at the health plans were surprised at finding disparities, especially when they regarded their organizations as high performing: “You think about these problems occurring in a low-income population, an uninsured population, maybe a Medicaid population. In an HMO, where people had almost first-dollar coverage for health services, we expected that anything we were hearing about in the area of disparities were not in my backyard” (manager, health plan).

Responses to Data Analyses

When the data analyses had found disparities, several interviewees described a process of inquiry into their cause. The managers in case 4, a health plan, remembered their initial surprise at finding a disparity between African American and white women's screening for breast cancer. They then conducted a focus group with the physicians and surveyed the patients in order to pinpoint the reasons. The survey enabled them to rule out their main hypothesis—that transportation costs were the main barrier—and showed that the women did not come because of the inconvenient appointment times, which the managers persuaded the provider to change. Case 1 (a hospital) had a similar problem with poor diabetes control affecting Latino patients at one of its affiliated community health centers.

So we developed this bilingual coaching model where we have the patient meet with a diabetes coach, who becomes their advocate around diabetes. And calls them at home, meets with them for an hour, and brings them in for visits. Sees them sometimes when they're there for their doctor's visits, checks up on how they're doing. But primarily really looks at what are the barriers that this patient faces to getting their diabetes under good control. (senior clinician, hospital)

Several other organizations had used the findings from their data analysis to invest in new services or to improve existing services when they found disparities or deficits in quality. These initiatives included mobile screening services, enhanced interpreter services, and lower copayments for preventive care.

Not all the cases were able to pinpoint a cause that could be addressed through an intervention. For example, case 7 (a community health center) tried to investigate a disparity in blood pressure control among non-Hispanic black patients but did not find any differences in rates of visits or prescriptions and, after a fairly intensive search, was left somewhat puzzled.

We really wanted to do something about this black disparity in blood pressure control, but what do you do? You can't just start doing anything, because that makes no sense; you have to figure it out; that's why we started to chip away, to hypothesize, now what might be causing this disparity? How do you address it if you don't really know what is the underlying reason? You can't address it. (senior manager, health center)

In case 1 (a hospital), finding no disparities led to a different kind of action. Despite uncovering no gaps in their core hospital measures of quality, the respondents talked about searching for new indicators that their hospital was not yet routinely collecting: “So what we're trying to do now is we're trying to think more of where does the disparity literature show there are disparities and how can we develop measures that are more in sync with where the disparity literature shows there are disparities” (senior manager, hospital).

These “sentinel” indicators derived from the research literature included disparities in pain management in the emergency department or disparities in renal transplantation. Other cases were refining their existing measures. For example, case 3 reported that its patient and family experience surveys were being translated into several languages and that it was planning to experiment with new formats to increase the response rate, which the hospital perceived to be low among some minority groups.

Drivers and Future Developments

On the basis of the race and ethnicity data, the leaders of all the organizations decided that equity should be a major organizational focus requiring the same precision in data collection and analysis as used for other dimensions of quality, such as safety and timeliness. Several participants referred to the importance of the Institute of Medicine's definition of quality (IOM 2001), which listed equity as one of six core components of quality. A vice president at one of the hospitals stated that this report helped its leaders understand that organizations delivering health care should be as comfortable with their duty to “eliminate any disparities based on patient attributes—age, gender, socioeconomic status, race” as they were with “providing safe care or dealing with problems of overuse and underuse.”

The participants felt that in regard to benefits, there were no direct competitive or financial advantages from concentrating on disparities. But some respondents did refer to the future benefits of a good reputation among patients. In fact, one hospital had begun marketing itself as “the hospital of choice” for Spanish speakers, and its work on equity had given it confidence to do this. One of the health plans reported that some large national employers asked questions about diversity and equity as part of the bidding process for contracts: “Those companies understand diversity, understand disparities, and we've got to be able to be front and center on that if we're going to get those accounts” (senior manager, health plan).

Not all the organizations felt confident about advertising their work in this area to their patients and the wider community. Case 1 (a hospital) was the only organization that made the results of its data analysis public, by posting the findings of their analyses on the organization's website. By contrast, case 3 (a hospital) felt that it was still a long way from making its work in this area public, partly because it still was trying to determine the cause of the disparities and did not want to mislead patients and the public: “Can we say with confidence to an African American family that when they come here, they will feel welcomed and heard? I don't know that we can say that necessarily, and so do you want to be raising expectations when you can't fulfill that?” (senior manager, hospital). The respondents from the health plans also mentioned that the data might lead them to identify disparities and therefore become more efficient by reducing hospitalizations and length of stay. In addition, all the organizations emphasized the importance of collaborative learning networks, access to grant funds (to support analysts and research projects), and the incentive of national award programs to advertise their work (if successful).

Some aspects of regulation were seen as beneficial, particularly in regard to collecting data. Some organizations (cases 1, 2, and 5) reported that city and state mandates required them to collect data on race, ethnicity, and language. They said that even though their efforts had preceded the mandates, they nevertheless felt that the mandates added legitimacy to the issue, which had been useful in making the case for investing in training and information technology.

A common perception was that more regulation in the area of disparities and provision of interpreter services was likely in the future. For some, this legitimized their current efforts, as they felt that they would be “ahead of the game.” Others were concerned that the regulation might be too specific and would encroach on their autonomy:

I think that this is something that, again, is more susceptible to data analysis, hypothesis development, and testing, and for organizations to understand it within their own context. If you try to apply a regulatory framework to that, I think that you're more likely to make mistakes because of a regulatory framework's inability to appreciate context. (CEO, hospital)

The two community health centers were concerned as well that additional regulations or mandates might not be adequately funded and therefore would be difficult to implement.

Discussion: Implications for Policymakers

The evidence from these interviews illustrates the challenges for these organizations as they started on the trajectory described by Kilbourne and colleagues (2006), from detecting disparities to understanding their causes, before designing and implementing remedies for those disparities.

The first barrier that the organizations faced was collecting data from patients. The experience of the hospitals in this study, whose investment in training and IT infrastructure seems to have overcome the challenges of collecting data, is mirrored by the experience of hospitals in a state that recently passed a mandate requiring providers to collect race, ethnicity, and language data: that is, similar challenges appear to have been surmounted (Jorgensen et al. 2010).

Conversely, the difficulties faced by the community health centers reflect the differences in available IT systems and staff to collect and input the data. A 2009 survey found that a majority of community health centers lacked fully functioning electronic health records (National Association of Community Health Centers 2009). These resource constraints could improve in the future as more resources are being made available through the Affordable Care Act to expand federally qualified community health centers. It is not yet clear, however, whether the increased demand for services (through the expansion of coverage) will exceed the extra investment (Fiscella 2011). Community health centers also should benefit from the plans to improve the quality and use of IT systems, made possible by the 2009 American Recovery and Reinvestment Act. This act requires vendors of IT systems wishing to access all federally funded health care providers (including community health centers) to include the collection of race and ethnicity data in their programs (Bilheimer and Klein 2010).

But even if hospitals and community health centers did collect comprehensive data, there still would be a huge gap in ambulatory and primary care for many Americans, as community health centers serve only some of the population, and most people do not often go to the hospital. Although many Americans do have a primary care physician, the literature suggests that little information is directly collected by primary care physicians (Wynia, Ivey, and Hasnain-Wynia 2010) and this is likely to be an enduring challenge, given the resource constraints faced by physicians in small or solo practices, which account for a large proportion of primary care in the United States.

This gap could be filled by better information from health plans, but as our study found, health plans face a different kind of challenge, stemming not so much from limited resources but from reluctance to ask enrollees for their personal data. A recent survey of health plans found that slightly more than 50 percent of health plans reported that they did collect race and ethnicity data, but it is not known whether they were self-reported (Rosenthal et al. 2009). Earlier work with health plan leaders suggested that worries about the acceptability and legality of collecting data from members were widespread (AHIP 2006). To what extent these barriers can be surmounted also is not known. Focus groups with health plan enrollees in Massachusetts reflected unease with revealing ethnicity and race to health plans, principally over fears that it would adversely affect access to care (Hasnain-Wynia, Taylor-Clarke, and Anise 2010). Nevertheless, some health plans have succeeded in collecting data directly from their enrollees. Furthermore, partnerships between health plans and national quality organizations, such as the National Health Plan Collaborative, have boosted health plans’ initiatives to reduce disparities, but it is notable that these pioneering health plans collected only limited (no more than 30 percent) information directly from their members and that the rate of growth was too slow to reach anything near complete coverage (Lurie et al. 2008).

Mandates for health plans to collect data directly from patients might be a productive way to correct this. One plan in our study is based in a state that enacted a mandate for both hospitals and health plans to collect race, ethnicity, and language data (Weinick et al. 2007). Research on health plan members conducted before the mandate was passed showed that despite the concerns about revealing race and ethnicity data, people understood the potential value of this information for quality improvement purposes. This finding suggests that, as with hospital providers, data collection is feasible (Hasnain-Wynia, Taylor-Clarke, and Anise 2010). In this case, the state is allowing health plans to very gradually increase the proportion of their member records that contain full race and ethnicity data (Health Care Quality and Cost Council 2009).

In the meantime, health plans will likely have to continue to use proxy methods like geocoding and probabilistic estimation. Although these methods have been judged to be a useful alternative for understanding the health needs of local areas or populations (e.g., to help in the targeting of a screening campaign), they are not reliable for individual patients (Ulmer, McFadden, and Nerenz 2009).

The variability in categories of data collected by these organizations underscores the need for the standardization of race, ethnicity, and language data categories, as recommended by the Institute of Medicine (Ulmer, McFadden, and Nerenz 2009). This goal is particularly important if many organizations’ data are to be pooled in order to see regional patterns of disparities or are to be shared among organizations, since policymakers increasingly seek models of care (e.g., medical homes or accountable care organizations) that cross organizational boundaries.

These standardized categories need to be flexible enough, however, to reflect the demographic profiles of different areas. Some of the organizations in our study wanted to know the differences within broad categories, for example, in order to distinguish people of different African nationalities from African Americans. The Institute of Medicine has argued for detailed categories that “roll up” into larger categories (Ulmer, McFadden, and Nerenz 2009), but comparisons that require population denominators derived from the U.S. Census for some racial or ethnic groups will not be easy because of their small size (Bilheimer and Klein 2010). Even if health care organizations are able to identify a robust set of categories, collecting data from patients needs to be an ongoing process because patients’ identities are not rigid or fixed, as the organizations in our study found.

The variability in how these organizations defined and measured disparities raises even more questions about standardization. Even though the arguments to standardize the categories and methods of data collection have been clearly articulated, less has been written about the potential challenges that face organizations attempting to analyze their own data, particularly if they take different analytical approaches in their search for disparities. We found organizations using different approaches to decide which size of sample to use, which comparator groups to use, what magnitude of difference to use, and whether to adjust for potentially confounding factors. While some of this variation was based on the presence or absence of resources, it also reflected different attitudes of researchers and regulatory bodies toward the nature and causes of disparities. Official agencies, such as the Agency for Healthcare Research and Quality (Hebert, Sisk, and Howell 2008), also recommend or use different definitions and measurements. Although the research literature has conventions regarding definitions and statistical methods for controlling for mediating factors, there is no consensus on how much relative weight to attach to socioeconomic status and race/ethnicity or how best to measure socioeconomic disadvantage and other environmental factors that affect minority populations (Bilheimer and Klein 2010; Hebert, Sisk, and Howell 2008; Kawachi, Daniels, and Robinson 2005; LaVeist 2005; Williams et al. 2010).

If there is methodological variability within the research community, does it matter that there is also methodological variability within health care organizations? Health care organizations may gain insights from more detailed analysis, especially to disentangle potential causes of disparities, as they evaluate the causes of an apparent disparity between two groups and decide whether to target socioeconomic factors (such as copayments) or factors that relate to racism or culture. Indeed, variable methods might lead to one organization investing resources to solve a problem that is not considered a disparity by another. While some variability of this kind might be acceptable to organizations that are not supported by public funds, the allocation of federal and state funds to others is likely to require more accountability and therefore to need more methodological consistency. Policymakers might consider drawing up guidelines for use by leaders and analysts in health care organizations, thereby building on best practices in the academic measurement and analysis of disparities (Bilheimer and Klein 2010) and on examples of locally produced guidance for hospitals (Weinick, Flaherty, and Bristol 2008).

As some of our cases demonstrate, though, those persons analyzing the data also were negotiating within each organization to decide what should be considered a significant disparity, based on their knowledge of patient feedback and local communities. Accordingly, any future guidance might need to balance standardization with autonomy so that organizations could determine themselves which disparities to remedy first.

More analytical consistency also would be valuable from a regulatory standpoint. The 2010 Patient Protection and Affordable Care Act (PPACA) signals an ambition to pay closer attention to monitoring disparities within federally funded health care organizations. This act defines a “disparity population” as one that experiences “a significant disparity in the overall rates of disease incidence, prevalence, morbidity, mortality or survival rates in the population compared to the health status of the general population,” which may be expanded to include “significant disparity in the quality, outcomes, cost or use of healthcare services or access to or satisfaction with such services as compared to the general population” (sec. 845e).

More regulation and standards would also have to consider which quality indicators should be stratified by race and ethnicity and to what extent health care organizations can be held to account for any resulting disparities. As the examples in our study illustrate, some disparities (such as differences in screening uptake) could be rectified by the organization, whereas the cause of others was not so clear-cut, for example, the differences in hypertension control at a community health center. The idea of holding hospital providers responsible for what happens to patients when they have left the hospital is gaining traction with policymakers, such as in relation to reducing readmissions for Medicare patients (Jencks, Williams, and Coleman 2009). Applying a similar logic to reducing disparities will need to be carefully considered. Disparities are caused by a complex interaction of factors, only some of which are likely to be within the control of health care organizations (Ayanian 2008; Chin et al. 2007). Holding providers accountable for disparities in health outcomes that are not perceived to be within clinicians’ power to control could therefore lead to resistance from them (Thorlby et al. 2011).

As the experiences of some of our cases show, these organizations had very different approaches to taking action once a disparity had been identified. Some of them undertook further analyses—involving research with communities or physicians—but others moved more swiftly to invest in interventions to solve the disparity. The research evidence regarding what interventions worked best to reduce disparities is still emerging and is of varying quality (Chin et al. 2007; Sequist et al. 2010). This raises the question of whether there should be common guidelines for additional research that organizations should conduct on the possible causes of a disparity and whether interventions should have an evidence base before they are implemented. Any such guidelines would also need to allow organizations some autonomy to determine what is reasonable or, indeed, feasible to address, given their resources.

A recurring theme in interviews with organizational leaders was the tension between the desire to preserve the autonomy of organizations to find their own solutions to disparities and the desirability of stronger external regulations and the need for more standardized quality measures. Part of this tension stems from ambiguity about whether the search for equitable care at this stage of its development should be primarily a health care innovation or a regulation. A major review of the substantial body of literature on health care innovations found several attributes that an innovation needs to have in order to be successful (Greenhalgh et al. 2005). These include the need to deliver perceived advantages to the organization and the capacity for organizations to change and to adopt the innovation as part of the implementation process. The experience of these organizations suggests that both of these key attributes are present but are not well developed.

The benefits—for example, reputational advantage among peers—of collecting race and ethnicity data were obvious to some people in each organization, but action on disparities was not seen as bestowing obvious advantages to public reputation. Most of these health care organizations were wary of making their performance reports available to the public. Nevertheless, research has demonstrated that public reporting of hospital process measures has led to improved quality and patient outcomes (Fung et al. 2008; Werner and Bradlow 2010). While the public reporting of equity measures is an important goal, it will require both a consensus on the appropriate definition of a disparity and some certainty that the measures are an adequate reflection of equitable care.

All the organizations in our study were experimenting with the collection, analysis, and application of race and ethnicity data. Their experience should be seen as an important element in the drive for better and more systematic evidence of what works to reduce disparities (Chin and Goldmann 2011). Policymakers and researchers should learn from these organizations in order to inform the future direction of regulation and development of guidance in this important area.

Conclusion

The organizations we chose for our project were further along in their data collection and analysis than others and thus understood the potential disparities in health care. Earlier research suggests that most comparable organizations are generally not as far along in the search for disparities in their services (Hasnain-Wynia and Baker 2006; Lurie et al. 2008; Weinick et al. 2007; Wilson-Stronks and Galvez 2007). If the federal government or the states wish to mandate other organizations that are less engaged (or not engaged at all) with the issue of racial and ethnic disparities, then the experience of the organizations in our study points to several key lessons for policymakers and organizational leaders.

These cases offer evidence that it is feasible for health care organizations to collect and analyze race and ethnicity data from their patients to uncover any disparities and quality problems. But even though data collection and analysis is possible, it is expensive (particularly for community health centers) and difficult for both hospitals and health plans. Greater regulation may be appropriate to generate the momentum to tackle disparities in organizations that are less motivated to collect information, but the regulation may need to be accompanied by more resources, particularly for community health centers.

If regulation is used to encourage or require the collection of race and ethnicity data to identify and address disparities, the variability of approaches adopted by the organizations in this study suggests that some common standards for analysis may be needed. Although we do not know whether the experiences of these cases would be replicated if data collection and analysis were more widespread, the heterogeneity of their approaches to the collection and analysis of racial and ethnic data suggests there may be substantial variation in the way that organizations measure and take action on health disparities. If regulators or government agencies want to encourage consistent action and investment by health care organizations in reducing disparities, some common definitions of disparities and methods of measurement for health care organizations may be useful, perhaps building on the experience of the HRET toolkit (Hasnain-Wynia et al. 2007).

A thornier issue raised by these cases is whether there should be common thresholds for intervention and action, as some organizations have invested in solutions to disparities that other organizations would not have considered a disparity at all. Policymakers thus need to find the right balance between regulation and the preservation of health care organizations’ autonomy to take the initiative on disparities. Finally, the experiences of the pioneering organizations in our study provide important lessons that can be disseminated more widely to other comparable health care organizations.