Disparities in Documentation: Evidence of Race-Based Biases in the Electronic Medical Record

Abstract

Personal implicit biases may contribute to inequitable health outcomes, but the mechanisms of these effects are unclear at a system level. This study aimed to determine whether stigmatizing subjective terms in electronic medical records (EMR) reflect larger societal racial biases. A cross-sectional study was conducted using natural language processing software of all documentation where one or more predefined stigmatizing words were used between January 1, 2019 and June 30, 2021. EMR from emergency care and inpatient encounters in a metropolitan healthcare system were analyzed, focused on the presence or absence of race-based differences in word usage, either by specific terms or by groupings of negative or positive terms based on the common perceptions of the words. The persistence (“stickiness”) of negative and/or positive characterizations in subsequent encounters for an individual was also evaluated. Final analyses included 12,238 encounters for 9135 patients, ranging from newborn to 104 years old. White (68%) vs Black/African American (17%) were the analyzed groups. Several negative terms (e.g., noncompliant, disrespectful, and curse words) were significantly more frequent in encounters with Black/African American patients. In contrast, positive terms (e.g., compliant, polite) were statistically more likely to be in White patients’ documentation. Independent of race, negative characterizations were twice as likely to persist compared with positive ones in subsequent encounters. The use of stigmatizing language in documentation mirrors the same race-based inequities seen in medical outcomes and larger sociodemographic trends. This may contribute to observed healthcare outcome differences by disseminating one’s implicit biases to unknown future healthcare providers.

Introduction

Structural racism is increasingly recognized as an important factor in medical outcomes, though many view structural racism as an intangible and difficult-to-identify entity separated from seemingly objective medical practices. In reality, all healthcare providers live within the societies in which structural racism exists and have implicit biases that may affect how they do their work. The effects of provider beliefs and structural racism are primarily considered in the context of direct patient-provider or patient-health system interactions with a focus on individual patients [1–3]. However, an insidious and far-reaching method of bias transmission does exist—the electronic medical record (EMR)—and this avenue has received far less scrutiny.

The role of medical documentation in specific patient subsets or single-center studies has noted a biased skew towards negative terms being used in under-represented minority populations [4–6]. Multi-hospital healthcare system studies of how medical records may transmit racial bias are difficult to find, though a recent study showed an increase in diagnostic errors in notes with stigmatizing language present [7]. In this study, we hypothesized that stigmatizing language, i.e., ad hominem descriptions or documentation of stigmatizing language with limited medical relevance, is over-represented in racial minority populations’ EMR data across a healthcare system. We also hypothesized that these biases, positive or negative, may transcend single encounters and be perpetuated via EMR to other professionals in future encounters. In this report, we aim to highlight the pervasiveness of this language and the risks that patient characterizations may have on informing future care.

Methods

Data and Study Population

An institutional review board (IRB)–approved cross-sectional EMR chart review was undertaken for patients of all ages who received care and had opted-in to the use of their data for research purposes as part of their clinical care. A waiver of consent was obtained from the IRB for this retrospective research. Encounters were included if the note writer used words with potentially stigmatizing connotations, categorized as positive or negative based on common usage, to characterize a patient. Clinical notes were collected by the University of Minnesota’s informatics team and populated into Natural Language Processing-Patient Information Extraction for Research (NLP-PIER) software for analyses [8]. These words were selected based on their perceived regularity in the authors’ clinical observations and are included in Table 1. Specific curse words were combined as a group for analyses, as were the terms swearing, profanity, and cursing (SPC) and their conjugations. The reason for separating them is a perceived shock value of seeing specific curse words, whereas the words “swearing,” “cursing,” or “profanity” are less likely to elicit the same powerful reaction. Notes from five hospitals within the University of Minnesota/M Health Fairview were reviewed. These included the system’s primary academic center, its primary pediatric hospital, an urban community-based hospital, and two suburban hospitals, one of which provides both pediatric and adult ED and inpatient care. There were no exclusions in regard to the clinical role of the note writers.

Table 1.

Electronic medical record search terms

Negative terms	Positive terms
Angry	Brave
Argumentative	Reliable
Disrespectful	Respectful
Hostile	Polite
Noncompliant	Compliant
Specific Profanity: “Bitch,” “Damn,” “Fuck,” “Shit”a	Resilient
“Swearing,” “Profanity,” or “Cursing”b	Empathetic
	Ambitious
	Courageous

^aThese profane terms were analyzed as a single group (BDFS)

^bThese terms and their conjugations were analyzed as a single group (SPC)

The initial date of the encounter and the earliest date (“note date”) of each respective word in a given encounter were analyzed. Only the first instance of the selected words in each encounter was used to prevent inflation of word counts through documentation shortcuts like “copy-forwarding.” However, some patients did have multiple encounters included; in these patients, one or more of the stigmatizing terms occurred again in a later encounter. This allowed for the analysis of the perpetuation of stigmatizing language across encounters. All notes were reviewed in full by authors ZI, SH, BK, and/or AB to confirm the appropriate word context, excluding words, and/or encounters in which NLP-PIER selected misspelled words, correctly used terms that did not describe a person in that context (e.g., “HIPAA-compliant”), threatening language that may have legal ramifications, and quoted sentences from non-medical individuals in an encounter other than the patient (e.g., family members). Other collected data included patient age at the start of the encounter, race and gender based on EMR reporting (if available), and the hospital setting for the note (ED or inpatient). To streamline analyses, patients who were reported as having two racial/ethnic backgrounds were grouped with the under-represented race that was most prevalent in the included population (e.g., Black/AA plus White included in the Black/AA group). While we recognize race is a social construct, this analytic decision, given that we were reviewing personal documentation and potential bias, is consistent with psychological research that shows that members of both racial majorities and minorities are more likely to categorize bi-racial individuals in the under-represented minority group [9]. Ages were rounded to full completed year of life, with infants noted as 0 years if they had not reached their first birthday.

Statistical Approach

Patient and encounter characteristics were summarized using (1) counts and percentages or (2) means and standard deviations. The occurrence rates for each term or group of terms were compared between groups using logistic regression models fit using generalized estimating equations (GEE) to account for within-patient correlation for patients with multiple encounters. Results are reported using odds ratios with 95% confidence intervals and p-values. GEE logistic regression models with group, time, and group-by-time interaction were used to examine whether differences in rates of occurrence of terms between groups differed between the time periods before and after June 1, 2020. A two-sided p-value < 0.05 was regarded as statistically significant. Analyses were conducted using R version 4.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results

The initial dataset included 29,184 encounters for 22,739 patients. After screening using NLP-PIER, the final analyses included 12,238 encounters for 9135 patients. For comparison, the included hospitals cared for 260,427 unique patients during this time. The patients’ mean age at the start of each encounter was 52 years (standard deviation 22.3 years; range newborn, 104 years). Pediatric patients (< 21 years old) made up 10.3% of the total dataset (937/9135); of these, 33% of these encounters were for patients < 10 years old. Table 2 shows patient demographics. For the included hospitals, 177,252 (68.1%) were White, 46,867 (16.8%) were Black/AA, and 20,100 (7.7%) chose not to answer. All other reported races were ≤ 3% of the total and were not included for analyses. A total of 76,645 patients (29.4%) were < 21 years. Final analyses did not include the terms brave, courageous, empathetic, or reliable due to single-digit incidences in the dataset after the NLP-PIER review.

Table 2.

Demographics of study population

Category	N (%)
Total patients	9,135 (100)
Total encounters	12,238 (100)
Mean age at first encounter in years (standard deviation)	52 (22.3)
Age at first encounter in years (%)
0–20	937 (10.3)
21 +	8198 (89.7)
Gender
Female	4166 (45.6)
Male	4790 (52.4)
Unknown	179 (2)
Race
White	6730 (73.7)
Black/African American	1369 (15)
American Indian/Alaska Native	213 (2.3)
Asian	294 (3.2)
Middle Eastern	1 (0)
Native Hawaiian or Other Pacific Islander	21 (0.2)
Mixed race (3 or more reported)	8 (0.1)
Choose not to answer	429 (4.7)
No information available	70 (0.8)
Ethnicity
Not Hispanic/Latino	7813 (85.5)
Hispanic/Latino	266 (2.9)
Not reported	1056 (11.6)

Negative terms were found more frequently than positive terms (6707 vs 5969, respectively). Significant differences were observed by race. There was a significantly higher likelihood that a negative term was describing a Black/AA patient (OR 1.74) than a white patient. Conversely, positive terms were significantly more likely to be seen in documentation for White patients (OR 0.60). “Noncompliant” was more likely documented for Black/AA patients (OR 1.66), and “compliant” was more often seen in White patients (OR 0.73). Similarly, most of the negative terms, including curse words (universally quoting or paraphrasing patients), were associated with descriptions of Black/AA patients, whereas multiple positive terms occurred more frequently for White patients (Table 3).

Table 3.

Comparisons of the frequency of word use by race

	White	Black/AA	OR	95% CI	p-value
Angry	1506 (17.14%)	364 (17.63%)	1.03	(0.9, 1.19)	0.64
Argumentative	421 (4.79%)	104 (5.04%)	1.05	(0.8, 1.39)	0.71
Disrespectful	68 (0.77%)	51 (2.47%)	3.25	(2.14, 4.92)	< 0.001*
Hostile	323 (3.68%)	80 (3.87%)	1.06	(0.75, 1.48)	0.75
BDFS	852 (9.7%)	247 (11.96%)	1.27	(1.03, 1.56)	0.03*
SPC	653 (7.43%)	190 (9.2%)	1.26	(0.99, 1.61)	0.06
BDFS and SPC	1320 (15.02%)	370 (17.92%)	1.23	(1, 1.52)	0.04*
Noncompliant	2402 (27.33%)	794 (38.45%)	1.66	(1.42, 1.95)	< 0.001*
Compliant	3104 (35.32%)	590 (28.57%)	0.73	(0.64, 0.84)	< 0.001*
Ambitious	6 (0.07%)	7 (0.34%)	4.98	(1.55, 15.94)	0.007*
Resilient	230 (2.62%)	46 (2.23%)	0.85	(0.6, 1.19)	0.34
Polite	1551 (17.65%)	236 (11.43%)	0.60	(0.51, 0.71)	< 0.001*
Respectful	101 (1.15%)	30 (1.45%)	1.27	(0.82, 1.95)	0.28
Negative terms	4953 (56.36%)	1430 (69.25%)	1.74	(1.5, 2.03)	< 0.001*
Positive terms	4530 (51.55%)	805 (38.98%)	0.60	(0.52, 0.69)	< 0.001*

To evaluate our hypothesis about the “stickiness” of first impressions, we evaluated the extent to which patients with positive descriptors in their first encounter in the dataset have positive terms in one or more subsequent encounters, and we did the same for negative terms. Fifty percent (1551/3103) of patients had concordant descriptive terminology—positive followed by positive or negative followed by negative—in a subsequent encounter in this dataset. In the group with a positive term at their first encounter, 67% had a positive term at a subsequent encounter (OR 3.1, 95% confidence interval 2.5–3.9). Conversely, in patients with a negative term at their first encounter, 84% had a negative term at a subsequent encounter (OR 6.6, 95% confidence interval 5.1–8.5). This suggests that negative terms were about twice as likely to persist as positive terms, though “neutral” encounters that may have occurred during this time frame could not be accounted for in this dataset.

George Floyd’s murder (May 25, 2020) occurred within 7 miles of four of the five hospitals included in the study. This event led to profound racial unrest nationally and initiated more open discussions about racial inequities and programs aimed to combat them. Given these impacts, a post hoc analysis of whether the magnitude of the overall racial differences in the EMR may have been affected by this event was conducted, rounding to a pre-and post-date cutoff of June 1, 2020, where a small interaction p-value provides evidence for a change in magnitude of the differences. For most terms, no significant differences were observed between time frames. However, there were trends toward widening gaps in the documentation of profanity (0 → 5%) and negative terms overall (10 → 15%) for Black/AA individuals, though these did not meet statistical significance (Table 4).

Table 4.

Comparison of word use by race in relation to George Floyd’s murder

	Pre-June 1, 2020		Post-June 1, 2020		Interaction p-value
	White	Black/AA	White	Black/AA	Race × time period
N a	3881	903	4903	1161
Angry	691 (17.8)	170 (18.8)	815 (16.6)	194 (16.7)	0.65
Argumentative	204 (5.3)	48 (5.3)	217 (4.4)	56 (4.8)	0.76
Disrespectful	29 (0.7)	20 (2.2)	39 (0.8)	30 (2.6)	0.82
Hostile	153 (3.9)	29 (3.2)	170 (3.5)	51 (4.4)	0.20
BDFS	374 (9.6)	87 (9.6)	478 (9.7)	160 (13.8)	0.06
SPC	282 (7.3)	74 (8.2)	371 (7.6)	116 (10.0)	0.48
BDFS and SPC	564 (14.5)	132 (14.6)	756 (15.4)	238 (20.5)	0.09
Noncompliant	1142 (29.4)	346 (38.3)	1260 (25.7)	448 (38.6)	0.12
Compliant	1355 (34.9)	265 (29.3)	1749 (35.7)	325 (28.0)	0.45
Ambitious	2 (0.1)	4 (0.4)	4 (0.1)	3 (0.3)	0.60
Resilient	108 (2.8)	22 (2.4)	122 (2.5)	24 (2.1)	0.88
Polite	651 (16.8)	95 (10.5)	900 (18.4)	141 (12.1)	0.74
Respectful	46 (1.2)	14 (1.6)	51 (1.0)	16 (1.4)	0.98
Negative termsb	2244 (57.8)	613 (67.9)	2709 (55.3)	816 (70.3)	0.08
Positive termsb	1948 (50.2)	352 (39.0)	2578 (52.6)	453 (39.0)	0.44

^aN represents the number of encounters, with the percentages in parentheses for each term

^bSum of positive and negative terms surpasses total encounters as some encounters had both positive and negative terms

Designates a significant p-value (≤ 0.05)

BDFS, combined analysis of “bitch,” “damn,” “fuck,” and “shit”; SPC, “swearing,” “profanity,” or “cursing”

Discussion

Electronic documentation is ostensibly used to share factual descriptions of medical events and treatments rendered. However, human beings are writing medical notes, and everyone has biases that influence their interpretations of events and interactions with others. The digital permanence of the EMR allows for a more insidious and wide-spread transfer of implicit biases to future unknown readers. This may be true even if the latter clinical context is very different from the initial documentation. Thus, the potential for race-based biases being perpetuated through the EMR was the focus of the study. Here, we report that EMR descriptions of patients (or parents) using stigmatizing terminology is racially imbalanced, consistent with a nascent body of literature on the topic [6, 10–12].

Prior studies have shown that race is associated with differential outcomes potentially linked to EMR documentation, such as emergency department triage scores [13], diagnostic delays [14], social risk factor screening [15], treatments rendered, cancer outcomes [16, 17], and the use of goals-of-care discussions and advanced care planning [18, 19]. Here, we observed the more frequent use of negatively stigmatized terms for Black/AA patients, with the opposite being true for White patients. In addition, negative terms were more likely to recur in subsequent encounters than positive terms. The tendency to preferentially weight negative experiences or reports over positive ones in new or sparse situations (e.g., meeting a patient for the first time) may have been affected by previous documentation in the EMR [20, 21]. Negative stimuli have been shown to produce specific patterns of brain activation as compared with neutral stimuli, particularly in brain regions associated with memory and social cognition [22, 23]. It is conceivable that the anchoring to negative wording may affect clinical interactions with patients through these subconscious triggers.

Our analyses were intentionally limited to ED and inpatient documentation and the first occurrence for the target words in an encounter. This was based on a presumption that most of the note writers had limited or no familiarity with the patient and/or family and first impressions have significant impact. Sun et al. showed that stigmatizing language is less likely to be found in outpatient documentation, possibly due to note writers knowing the patients better than acute care providers [10]. There may be the presumption of generally agreed-upon norms of respectful or argumentative behavior, but these are very much influenced by cultural norms, in-group “social priming,” and lived experiences, with more variability than is often credited [24, 25]. In addition, heuristics can accelerate clinical decision-making and documentation but also rely heavily on cognitive biases and cultural norms [26]. The racial imbalance described in our study is likely driven primarily by a combination of these factors rather than explicit biases; nevertheless, this should not negate their impacts.

The racial differences observed in using specific curse words deserve further evaluation. The frequency at which specific curse words were documented was surprising, even after excluding threatening quotes where their inclusion can be reasonably argued. We found no other studies which have described similar findings. Curse words, whether heard or read, have a taboo element that can stimulate physiologic arousal [27]. We feel that quoting specific curse words only adds shock value without medical importance but may influence future medical personnel in how they interact with patients and families. This prejudicial risk seems amplified by the racial differences we observed. We feel that the avoidance of curse word documentation in general could reduce stigma and the risk of presumed antagonistic behavior during future patient encounters.

Intriguingly, we did see non-significant trends towards increased documentation regarding cursing and negative terms overall after George Floyd’s murder. The reasons are unclear, but possibilities include an increased awareness of racial tensions amplifying the cultural biases previously described. Alternatively, this observation may also represent a defensive approach to documentation by medical staff in case a negative exchange occurred. The ultimate reasons could not be determined as part of this research.

We feel that these results represent an under-recognized but important source of hidden bias in the medical community, but the study does have limitations. The cross-sectional language-focused approach and broad patient population does not allow us to determine whether clinical outcomes were affected. Despite this, and perhaps more importantly because of this limitation, these results suggest that the stereotypes that are well-described in broader society have leached into the medical culture in a disease-agnostic fashion. Another argument could be made that the word selection was not comprehensive and this is a small subset of the overall document count during this timeframe. No word list will be truly comprehensive for this topic. We prioritized the use of common but subjective terms (e.g., compliant and noncompliant),[25] the addition of positive terms to provide more balance, and the use of curse words that seem medically irrelevant but have emotional valence. Also, no single person is realistically reading all of the documentation of a particular patient, much less multiple patients. However, it only takes a few trigger words to prompt implicit bias. Given the fact that full documentation review is difficult, it paradoxically makes those single words or phrases more potent. Finally, our data reflects documentation in a single hospital system to streamline data collection, so we cannot confirm these findings with other systems’ EMRs. Despite these factors, it would be naïve to think this is simply a system issue. Given previous research on stigmatizing language, medical documentation is an oft-overlooked forum to implicit biases unfettered across space and time.

Our analyses highlight an uncomfortable reality about medical practice—that we all contribute our implicit biases, positive or negative, to other medical providers not just through verbal interactions but through nonverbal means. This may be compounded by regulations about EMR transparency for patients and families, a topic that has polarized many medical providers. Patients and families can now see these documents. They may observe stigmatizing terms being used but paradoxically may be more powerless to intervene, having no standardized method to respond in kind via electronic documentation. It is, therefore, imperative that in this age of EMR accessibility, we address the importance of documentation to reduce racial bias.

Evidence-based methods for inducing this change need further investigation. We believe that system-wide statements about reducing stigmatizing language are not enough—they often produce minimal sustainable improvements, are reduced to platitudes, and become easy to ignore it as other people’s problems. They may even produce backlash in our current culture. One suggestion, which should be rigorously studied to understand the extent of effectiveness given the inherent biases of artificial intelligence (AI), would be to use AI tools that could recognize certain stigmatizing terms and suggest non-stigmatizing alternative text or removal of the word or phrase altogether. This may be one avenues to have a broader impact on bias mitigation, though we are optimistic that other methods exist. Ultimately, systemic methods to reduce implicit bias in digital communication could have more profound effects than any one patient-provider interaction may hope to do.

Data Availability

Data is not publicly available. Access to the data may be available upon reasonable request to the corresponding author and clearance by the institutional data team.