On the Perils of Conducting Observational Research on Racial Health Disparities

Racial disparities in cardiovascular health outcomes are well documented, as is the presence of unequal treatment of patients in racial and ethnic minority groups in the health care setting. ¹ , ² Frequently, these disparities are brought to light using large administrative claims data sets. Such data sets are powerful research tools, often containing clinical and demographic information on tens, if not hundreds of thousands, of individual patients. The use of administrative claims data sets in health services research is increasing.

An example of a project using one such data set (Optum Clinformatics, a large commercial insurance data set marketed for research purposes) was recently published in the Journal of the American Heart Association (JAHA), examining racial disparities in the treatment of non–ST‐segment–elevation myocardial infarction. ³ The authors demonstrate that Black and Hispanic individuals are less likely to be treated according to an invasive strategy when compared with White individuals. These findings are noteworthy given the preponderance of evidence supporting an invasive strategy as the standard of care for patients presenting with non–ST‐segment–elevation myocardial infarction, ⁴ as well as a steady increase in non–ST‐segment–elevation myocardial infarction incidence over time. ⁵

However, some caution is warranted when interpreting these results, as well as the results of any observational study attempting to document racial disparities using administrative claims data. The challenges associated with ascertaining race in large observational and administrative data sets are well‐known. ⁶ In many cases, the methods used to derive race and ethnicity variables are unclear, and the variables themselves have been incompletely validated. ⁷ And such difficulties are arguably even more relevant in commercial (as opposed to publicly available, government‐led) administrative claims data sets, given their proprietary nature.

Commercial administrative claims data sets often use automated algorithms to derive race and ethnicity data. Optum Clinformatics, for instance, previously used software known as E‐Tech to construct its race and ethnicity variables, “incorporating racial and ethnic neighborhood composition from the US Census, residential zip code, and first and last name.” ⁸ But such algorithms are prone to error. A study of the “diagnostic accuracy” of E‐Tech in determining a subject's race noted that the software misclassified “52% of all [B]lack participants…as [W]hite,” and, in doing so, systematically “underestimated [B]lack race.” ⁹ Perhaps even more troubling, it appears that Optum may no longer be using E‐Tech software, leaving researchers wanting to work with its data set with no published validation of the “proprietary algorithm” that Optum now uses to impute race and ethnicity data. ⁷

Returning to the example I cite at the beginning of this piece, then, an algorithm that routinely underestimates Black race could significantly confound the relationship between race and appropriate treatment of non–ST‐segment–elevation myocardial infarction. And with commercial administrative claims data sets, such as Optum Clinformatics, only increasing in popularity, when it comes to studying racial disparities, these sorts of methodological limitations, which are considerable, should elicit certain questions; chief among them, what is the overall utility of documenting racial and ethnic health disparities in privately held data sets that, “representative” though they may be, remain inaccessible to most researchers and ascertain race via algorithms that rely on crude proxies, such as name and geography (proxies that, given rapidly changing demographic patterns in the United States, ¹⁰ , ¹¹ may be outliving their usefulness)?

Indeed, many disparities researchers are already attuned to these difficulties, often providing certain caveats pertaining to observational findings, citing, for example, an inability “to rule out misclassification of demographics such as race, ethnicity, or…social risk factors,” that “race and ethnicity are highly heterogeneous and may not be adequately captured by the categories used in [the] analysis,” and that “administrative data are an indirect estimation, and as such…racial and ethnic groups may not be consistent with those identified as self‐report.” ³ But qualifications such as these are significant, and should do little to assuage the concerns of the reader; I would contend that they deserve more than passing mention in articles whose core conclusions often rely on models in which race and ethnicity are, in fact, incorporated as the primary exposure of interest.

The question of how race is “determined” in administrative claims data sets is a vexing one. Contrary to being a naturally existing category in any meaningful sense, race and ethnicity are, as is frequently acknowledged, “nuanced social constructs.” ³ And although self‐report is widely regarded as the most useful form of data collection on race in the health care setting, ¹² a large body of social science literature suggests that, in an everyday sense, race is even more complex, depending, as I have written elsewhere, “to varying degrees in different contexts, on an individual's self‐conception and on society's labeling of that individual.” ¹³

And if this more nuanced and contingent view of race is true, if race is not simply “out there,” waiting to be discovered and categorized (or, for that matter, is not simply a function of how individuals themselves perceive, and therefore report, their own identity ¹⁴ ), then what are computerized algorithms, which glean something called “race” from a collection of “ethnically unique” names and geographic areas, actually detecting? Clearly, there is a need to improve race and ethnicity collection and reporting practices in administrative claims data sets, especially commercially available ones; I would argue that methodological transparency is a necessary first step.

As health services researchers increasingly rely on nonpublic data sources in which the construction of key variables is undertaken by proprietary algorithms whose inner workings go largely unquestioned, clarification of concepts such as race (concepts that, historically, we have taken for granted as a priori facts) will become even more vital.

Source of Funding

None.

Disclosures

None.