Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Mar 1.
Published in final edited form as: Acad Emerg Med. 2016 Feb 22;23(3):297–305. doi: 10.1111/acem.12901

The Impact of Cognitive Stressors in the Emergency Department on Physician Implicit Racial Bias

Tiffani J Johnson 1, Robert W Hickey 1, Galen E Switzer 1, Elizabeth Miller 1, Daniel G Winger 1, Margaret Nguyen 1, Richard A Saladino 1, Leslie R M Hausmann 1
PMCID: PMC5020698  NIHMSID: NIHMS751151  PMID: 26763939

Abstract

Objectives

The emergency department (ED) is characterized by stressors (e.g. fatigue, stress, time-pressure, and complex decision-making) that can pose challenges to delivering high quality, equitable care. Although it has been suggested that characteristics of the ED may exacerbate reliance on cognitive heuristics, no research has directly investigated whether stressors in the ED impact physician racial bias, a common heuristic. We seek to determine if physicians have different levels of implicit racial bias post-ED shift versus pre-shift, and to examine associations between demographics and cognitive stressors with bias.

Methods

This repeated measures study of resident physicians in a pediatric ED used electronic pre- and post-shift assessments of implicit racial bias, demographics, and cognitive stressors. Implicit bias was measured using the Race Implicit Association Test (IAT). Linear regression models compared differences in IAT scores pre- to post-shift, and determined associations between participant demographics and cognitive stressors with post-shift IAT and pre- to post-shift difference scores.

Results

Participants (n=91) displayed moderate pro-white/anti-black bias on pre-shift (M=0.50, SD=0.34, d=1.48) and post-shift (M=0.55, SD=0.39, d=1.40) IAT scores. Overall, IAT scores did not differ pre-shift to post-shift (mean increase=0.05, 95% CI −0.02,0.14, d=0.13). Sub-analyses revealed increased pre- to post-shift bias among participants working when the ED was more overcrowded (mean increase=0.09, 95% CI 0.01,0.17, d=0.24) and among those caring for >10 patients (mean increase=0.17, 95% CI 0.05,0.27, d=0.47). Residents’ demographics (including specialty), fatigue, busyness, stressfulness, and number of shifts were not associated with post-shift IAT or difference scores. In multivariable models, ED overcrowding was associated with greater post-shift bias (coefficient=0.11 per 1 unit of NEDOCS score, SE=0.05, 95% CI 0.00,0.21).

Conclusions

While resident implicit bias remained stable overall pre-shift to post-shift, cognitive stressors (overcrowding and patient load) were associated with increased implicit bias. Physicians in the ED should be aware of how cognitive stressors may exacerbate implicit racial bias.

INTRODUCTION

Characteristics of the Emergency Department (ED) can pose challenges to delivering high quality, equitable care. For example, ED providers face high acuity, diagnostic uncertainty, time pressure, and workflow interruptions.1-5 Night shifts in particular can lead to physician fatigue,6 which is associated with suboptimal patient care7-9 and diminished personal safety.10,11 ED crowding has also been linked with lower quality care for adult and pediatric patients.12-18 In the context of overcrowding, ED physicians must make time-sensitive decisions for multiple patients without established doctor-patient relationships. These aspects of the ED may make physicians more prone to the use of heuristics, or mental shortcuts, which can include racial bias and stereotyping.19,20 However, no one has directly investigated the impact of cognitive stressors in the ED work environment on physician racial bias.

Racial bias can be either implicit (unconscious) or explicit (conscious). Implicit bias refers to unconscious attitudes, positive or negative, towards a person, group, or idea.21 Implicit bias lies below the surface, but may still unintentionally influence behavior, such as perceptions about patients or decisions about patient management.21 Explicit biases, in contrast, are conscious attitudes that are recognized by the individual, and can therefore be measured through self-report.21

While little-to-no explicit bias has been found among healthcare providers, research has consistently documented that healthcare providers across a range of specialties and levels of experience have implicit racial bias, with most exhibiting implicit preference for whites over blacks that differ from explicit, self-reported egalitarian attitudes.22-31 Levels of implicit bias have been linked with racial variation in medical decisions22,23,30 and patient ratings of care.27,32 Although early research suggests that implicit bias is relatively stable across time and context,33 there is a growing body of evidence demonstrating that certain situations can readily induce implicit bias34 and implicit bias is malleability in response to changes in the immediate environment.35,36

Evidence from social psychology also indicates that individuals exposed to mentally fatiguing tasks (vs. control subjects not exposed to mental fatigue) display more implicit racial bias on a reaction time task requiring them to identify as quickly as possible whether or not an object is a weapon or household object following subliminal priming of black or white faces.37 While all health care providers may experience some mental fatigue, the ED setting often poses significant challenges that may exacerbate levels of implicit bias, making it particularly important to examine implicit bias in the context of emergency medicine. Investigating racial bias in the ED is also important given that EDs serve as safety nets for vulnerable populations, providing access to care independent of income, insurance, and race/ethnicity.

The main objectives of this study were to examine potential differences in physicians’ implicit racial bias post-ED shift compared to pre-shift, and determine the association of demographics and cognitive stressors with differences in bias and levels of post-shift bias. We chose to focus our study on residents because the inexperience of physicians in training requires greater cognitive load to make clinical decisions, leaving less capacity to integrate individual (instead of group based) information.20 This may make residents particularly prone to bias and stereotyping in the ED. Given prior evidence that experiencing mental fatigue leads to greater levels of implicit bias,37 we hypothesized that working in the ED would lead to greater implicit racial bias post-shift compared to pre-shift, and cognitive stressors encountered during the shift would be associated with greater post-shift implicit bias.

METHODS

Study Design and Setting

We performed a pre-post repeated measures study using electronic assessments of resident physicians’ implicit and explicit racial bias before and after an ED shift. The study site was an academic pediatric ED with over 70,000 annual visits, of which 61.5% involve patients who are non-Hispanic white, 33.5% non-Hispanic black, 0.9% Asian, and 0.6% Hispanic. This ED serves as the pediatric emergency medicine training site for over 200 pediatric, emergency medicine, and family practice residents. This study was approved by the Institutional Review Board.

Selection of Participants

We recruited residents working in the ED between April and June 2013 to complete electronic assessments pre- and post-ED shift. ED shifts were 8 to 10 hours long, and residents were only enrolled during day and evening shifts. We chose not to enroll residents working an overnight shift (11pm-7am) to help distinguish cognitive fatigue due to their work from physical fatigue due to lack of sleep. Members of the research team involved in recruitment included 2 ED faculty (white males), 2 ED fellows (black female and Asian female), and 2 research nurses (white females). Due to the demographic characteristics and availability of the research staff, we were unable to match recruiters and participants based on gender or race. A standard recruitment script was used to explain the goals of the study and invite residents to participate. The script made it clear that participation was voluntary. Residents interested in participating were taken to a quiet administrative office within the ED to complete a confidential electronic assessment. To account for the practice effect associated with our implicit measure,38,39 residents were randomized to complete their first assessment either before (pre-shift) or after (post-shift) a scheduled shift. Pre-shift assessments were performed when residents arrived for their shift, prior to initiating patient care. Post-shift assessments were done once residents completed their shift. Those who completed their first assessment pre-shift completed their second assessment at the end of the same shift. Those who completed their first assessment post-shift completed their second assessment prior to their next scheduled shift whenever possible. Residents were compensated $5 for each assessment ($10 total if both assessments were completed). Residents completing both surveys were also entered into a raffle each month for one of the investigators to work an ED shift for the raffle winner.

Study Protocol

Pre-shift assessments included an informed consent page, demographic questions (race/ethnicity, age, gender, specialty, training year), and measures of implicit and explicit racial bias. Post-shift assessments included an informed consent page, questions about cognitive stressors encountered during the shift, and the same measures of implicit and explicit racial bias. Study identification numbers were used to link pre- and post-shift assessments to maintain confidentiality in scoring. Each assessment took approximately 10 minutes.

Cognitive Stressors

Residents were asked post-shift to report their current fatigue, average fatigue during the past week, shift busyness, and shift stressfulness using 11-point scales (0=none, 5=moderate, 10=extreme). Residents also reported the number of patients they cared for during the shift and number of shifts worked during the past week. We obtained the highest National ED Overcrowding Scale (NEDOCS) level during the shift, which was recorded as a part of normal ED workflow every four hours by ED staff. NEDOCS is an objective measure of overcrowding calculated based on an algorithm that includes the number of patients in the ED relative to ED beds, number of admitted patients compared to hospital beds, number of patients on ventilators, longest admit time, and waiting room time.40 The NEDOCS algorithm is interpreted using 6 levels, where 1=not busy, 2=busy, 3=extremely busy but not overcrowded, 4=overcrowded, 5=severely overcrowded, and 6=dangerously overcrowded.40

Implicit Racial Bias

Our main outcome was post-shift implicit racial bias and difference in implicit racial bias post-shift compared to pre-shift. We used the race Implicit Association Test (IAT) to measure implicit racial bias.41-45 The IAT is a validated and reliable tool that has been used in hundreds of studies across a range of disciplines,45,46 including healthcare.22-31,47,48 Details about IAT procedures, scoring, and psychometric properties have been published elsewhere.41,43,44 Briefly, participants categorize pictures of black and white faces with words that represent good and bad in randomized blocks of trials. The IAT measures the strength of association between these categories (e.g., black – bad, white – good) using response latency and frequency of errors. Participants who categorize white faces paired with good words more quickly and with fewer errors than white faces paired with bad words have an implicit pro-white bias. Scored using the D algorithm, IAT values ranging from −0.15-0.15 indicate no racial bias; 0.16-0.35, slight pro-white bias; 0.36-0.65, moderate pro-white bias; and >0.65, strong pro-white bias.49 Negative scores of similar magnitudes indicate pro-black bias. We limited our assessment of racial attitudes to white and black races because 95% of the patients served by the study site are white or black.

Explicit Racial Bias

We assessed explicit racial bias using existing scales.22-24,50 Residents were asked their feelings towards black and white people using 11-point temperature scales (0=cold, 5=neutral, and 10=warm). Temperature difference was calculated by subtracting the black temperature score from the white temperature score, with positive values indicating pro-white/anti-black bias.23 Residents also reported their racial preferences (0=strong preference for blacks, 5=neutral, and 10=strong preference for whites).

Data Analysis

Descriptive statistics were used to summarize resident demographic characteristics and cognitive stressors and to report pre- and post-shift implicit and explicit racial bias. We calculated Cohen’s d to compare the magnitude of implicit and explicit racial bias measures that used different scales (implicit: scale −2 to +2; explicit: scale 0 to 10), and to allow for more meaningful interpretation of pre- to post-shift changes in IAT scores.23 Cohen’s d yields a measure of effect size where 0.2=small effect, 0.5=medium effect, and 0.8=large effect.51 For the implicit and explicit bias measures, Cohen’s d represented the magnitude of pro-white/anti-black (or pro-black/anti-white) compared to zero bias for either race. Pre- to post-shift changes in bias scores were calculated by subtracting pre-shift scores from post-shift scores, with Cohen’s d used to determine effect sizes for pre- to post-shift differences.

Simple linear models were used to quantify pre- to post-shift differences in IAT score while adjusting for the practice effect, represented by group assignment (first assessment pre- or post-shift). We performed sub-analyses of IAT difference scores using simple linear models on residents who worked when the ED was extremely busy to dangerously overcrowded (NEDOCS 3-5), and residents who cared for >10 patients during a shift. Simple linear models (adjusted only for group assignment) were also used to determine the association between demographics (including specialty) and cognitive stressors with either post-shift IAT scores or IAT difference scores, each examined separately and hypothesized a priori to have a potential association with implicit bias. We then performed multivariable linear models for post-shift and difference in IAT scores, adjusting for covariates found to have potential associations (20% level of significance) in simple models. These multivariable models passed diagnostic checks for collinearity (variance inflation factor) and well-behaved residuals. The sensitivity of model coefficients to the removal of individual variables was also assessed to confirm reliable estimates.

Wilcoxon signed-ranked tests were used to compare differences in explicit measures, which were not normally distributed. The association of explicit with implicit measures was evaluated using Spearman rho correlation coefficients and bootstrapped 95% confidence intervals.

A priori power calculations indicated that a minimum of 41 residents would be needed to achieve 80% power to detect a conservative mean paired difference in IAT score of 0.1 with a 5% level of significance. Based on mean IAT scores and standard deviations in the published literature, a pre-post difference <0.1 would result in a Cohen’s d <0.2, or no effect of ED shift on IAT scores. We used IBM SPSS 21 (IBM Corp., Armonk, NY) to perform all analyses.

RESULTS

Demographic characteristics of study subjects (Table 1)

Table 1. Participant Demographic Characteristic and Shift Cognitive Stressors (n=91)a,b.

Demographic Characteristics
Race, N (%)
 Non-Hispanic white 67 (73.6)
 Non-Hispanic black 1 (1.1)
 Hispanic 4 (4.4)
 Asian 12 (13.2)
 Other 5 (5.5)
 Unknown c 2 (2.2)
Age, yr, mean (SD) 28.8 (+2.2)
Female sex, N (%) 47 (52.2)
Specialty, N (%)
 Pediatrics 41 (45.1)
 Emergency Medicine 32 (35.2)
 Other 17 (18.7)
 Unknown 1(1.1)
Training year, N (%)
 1 32 (35.2)
 2 34 (37.4)
 3 23 (25.3)
 4 1 (1.1)
 Unknown 1 (1.1)
Shift Cognitive Stressors
Current fatigue, mean (SD) 5.8 (1.5)
Fatigue during past week, mean (SD) 4.7 (2.0)
Busyness of shift, mean (SD) 5.2 (2.0)
Stressfulness of shift, mean (SD) 4.2 (1.9)
Number of shifts worked past week, mean (SD) 3.8 (1.8)
Number of patients cared for, n (%)
 0-10 55 (60.4)
 11+ 36 (39.6)
NEDOCS, highest, n (%)
 Level 1 (not busy) 0 (0)
 Level 2 (busy) 19 (20.9)
 Level 3 (extremely busy,
 not overcrowded)
48 (52.7)
 Level 4 (overcrowded) 18 (19.8)
 Level 5 (severely overcrowded) 2 (2.2)
 Level 6 (dangerously overcrowded) 0 (0)
 Unknown 4 (4.4)

Abbreviations: y, years; SD, standard deviation; NEDOCS, National Emergency Department Overcrowding Scale

a

Data are based on electronic assessments of 91 of the 106 residents working in a Pediatric Emergency Department between April to June 2013.

b

Because of rounding, percentages may not total 100.

c

Participant did not answer the question.

Of 106 residents working in the pediatric ED during the study period, 91 (85.8%) were enrolled and completed both assessments and were included in our analyses. Of the remaining, 4 (3.9%) were enrolled but completed only one assessment and 11 (10.4%) were missed due to no recruiter available or recruiter unable to locate resident. Forty-seven participants (51.6%) completed their first assessment pre-shift, and 44 (48.4%) completed their first assessment post-shift. Most participants were non-Hispanic white (73.6%), specialized in pediatrics (45.1%) or emergency medicine (35.2%), and were in their first or second year of training (72.6%).

Participants’ self-reported cognitive stressors (Table 1)

The mean fatigue post-shift was 5.8 (standard deviation [SD]=1.5), and mean fatigue over the course of the past week was 4.7 (SD=2.0) on a scale of 0 to 10. On average, residents rated the busyness and stressfulness of their shift as 5.2 (SD=2.0) and 4.2 (SD=1.9), respectively, on a scale of 0 to 10, and worked an average of 3.8 (SD=1.8) shifts in the past week. Thirty-six residents (39.6%) cared for more than 10 patients during their shift. The NEDOCS ranged from 2-5, with scores indicating that the ED was extremely busy, overcrowded, or severely overcrowded (NEDOCS 3-5) for 68 (74.7%) of participants.

Main Results for Implicit Bias (Table 2)

Table 2. Pre- and Post-Shift Implicit and Explicit Racial Bias of Resident Physicians Working in a Pediatric Emergency Department.

Pre-Shift Post-Shift Difference
(Post-Shift – Pre-Shift)
Mean SD Cohen’s
d
Mean SD Cohen’s
d
Mean 95% CI a Cohen’s
d
Implicit Bias: Race IAT d
 All Participants 0.50 0.34 1.48 0.55 0.39 1.40 0.05 −0.02,0.14 0.13
 High NEDOCS e 0.48 0.34 1.41 0.57 0.35 1.63 0.09 0.01,0.17* 0.24
 >10 Patients f 0.46 0.33 1.40 0.63 0.34 1.85 0.17 0.05,0.27* 0.47
Explicit Bias
 Temperature Difference b 0.28 0.67 0.42 0.29 0.69 0.42 0.01 −0.07,0.09 0.03
 Racial Preference c 5.40 0.96 0.42 5.43 1.03 0.42 0.03 −0.07,0.14 0.06
Correlation Between Explicit Measures and Race IAT g
r 95% CI r 95% CI r 95% CI
Temperature Difference −0.06 −0.16,0.27 0.24 0.04,0.41 0.25 0.04,0.42
Racial Preference 0.13 −0.08,0.33 0.14 −0.07,0.34 0.02 −0.20,0.23

Abbreviations: SD, standard deviation; IAT, Implicit Association Test; r, Spearman Rho Correlation Coefficient

a

95% CIs are based on Wilcoxon signed-ranked tests (explicit measures) and linear models adjusting for the practice effect of group assignment (implicit bias) to determine if the mean difference in explicit and implicit measures pre- and post-shift are different from 0.

b

Using self-reported temperature scales for feelings towards whites and blacks, the temperature difference was calculated by subtracting the black temperature score from the white temperature score, with a non-zero value indicating explicit bias.

c

Black/White Preference is based on a self-report 0-10 scale; where 0=strong preference for black, 5=neutral (no preference), 10=strong preference for white.

d

IAT interpretation: values ranging from −0.15-0.15 indicates no racial bias; 0.16-0.35, slight pro-white bias; 0.36-0.65, moderate pro-white bias; and >0.65, strong pro-white bias. Negative scores of similar magnitudes indicate implicit pro-black bias.

e

Analyses among 68 of the 91 participants who worked when the ED was extremely busy to dangerously overcrowded (NEDOCS Level 3-5).

f

Analyses among 36 of the 91 participants who cared for >10 patients during their shift.

g

Correlation of each explicit measure with implicit bias using Spearman’s Rho with bootstrapped 95% confidence intervals.

*

p≤0.05

We found moderate implicit pro-white/anti-black bias pre-shift (M=0.50, SD=0.34, d=1.48) and post-shift (M=0.55, SD=0.39, d=1.40; Table 2). As shown in Figure 1, most residents had IAT scores consistent with pro-white racial bias pre-shift (n=77, 84.6%) and post-shift (n=76, 83.5%). There was no significant difference in pre- to post-shift IAT scores in our overall sample (mean increase=0.05, 95% CI −0.02, 0.14, d=0.13).

Figure 1. Percentage of Residents with Categories of Implicit Bias Pre- and Post- Pediatric Emergency Department Shift.

Figure 1

Implicit Racial Bias Pre-Shift and Post-Shift.

Figure shows the percentage of residents with each category of IAT scores pre-shift and post-shift.

Among the 68 residents who worked when the ED was extremely busy to severely overcrowded (NEDOCS 3-5), we found a significant increase in pre- to post-shift IAT scores (mean increase=0.09, 95% CI 0.01, 0.17, d=0.24), indicating greater pro-white implicit bias post-shift. Among the 36 participants who cared for more than 10 patients during their shift, there was also a significant increase in IAT scores pre- to post-shift (mean increase=0.17, 95% CI 0.05, 0.27, d=0.47).

IAT scores did not vary by resident specialty, with moderate pro-white bias among the 41 pediatric residents (pre-shift M=0.54, SD=0.31, d=1.74; post-shift M=0.57, SD=0.37, d=1.54; mean increase=0.03, 95% CI −0.10, 0.16, d=0.07), 32 emergency medicine residents (pre-shift M=0.49, SD=0.33, d=1.48; post-shift M=0.52, SD=0.41, d=1.27; mean increase=0.03, 95% CI −0.11, 0.18, d=0.08), and 17 residents from other specialties (pre-shift M=0.45, SD=0.37, d=1.22; post-shift M=0.58, SD=0.42, d=1.38; mean increase=0.13, 95% CI −0.08, 0.33, d=0.33). In linear models adjusting for the practice effect, none of the resident demographic characteristics (race/ethnicity, age, gender, specialty, training year) were associated with post-shift or difference in IAT scores (data not shown), and were therefore not included in multivariable models.

ED overcrowding as measured by higher NEDOCS was associated with greater post-shift bias (coefficient=0.12 per 1 unit of NEDOCS score, SE=0.05, 95% CI 0.01, 0.22). Number of shifts worked in the previous week (coefficient=0.02, SE=0.02 per shift, 95% CI −0.03, 0.06), shift stressfulness (coefficient=0.04 per 1 unit on a 0-10 scale, SE=0.02, 95% CI −0.01, 0.08), and caring for more patients (coefficient=0.12 if caring for >10 versus ≤10 patients, SE=0.08, 95% CI −0.04, 0.28) also trended toward significance (p<0.2) as predictors of post-shift bias. These four variables were thus included in the multivariable model examining post-shift bias.

There were also trends towards greater pre- to post-shift difference in bias associated with caring for more patients (coefficient=0.16 if caring for >10 versus ≤10 patients, SE=0.08, 95% CI 0.00, 0.32), current fatigue (coefficient=0.04 per 1 unit on a 0-10 scale, SE=0.02, 95% CI −0.01, 0.09), and number of shifts worked in the previous week (coefficient=0.04 per shift, SE=0.02, 95% CI −0.01, 0.08). These three variables were entered into a multivariable model examining pre- to post-shift difference in IAT scores.

In multivariable analysis adjusting for the practice effect and covariates with associations with post-shift implicit bias in simple models (stressfulness, number of shifts, number of patients, and NEDOCS; Table 3), higher NEDOCS level was associated with greater post-shift implicit bias (coefficient=0.11 per 1 unit of NEDOCS score, SE=0.05, 95% CI 0.00,0.21). No factors were associated with significant pre- to post-shift difference in IAT scores in the multivariable model of that outcome (data not shown).

Table 3. Multivariable Linear Model to Determine Associations Between Resident Cognitive Stressors with Post-Shift Implicit Bias a.

Post-Shift
Coefficient SE 95% CI
Stressfulness (per 1 unit on a 0-10 scale) 0.03 0.02 −0.01,0.07
Number of Shifts (per 1 shift) −0.01 0.02 −0.05,0.04
Number of Patients > 10 (vs. ≤ 10) 0.06 0.08 −0.10,0.21
NEDOCS (per 1 unit increase in NEDOCS) 0.11 0.05 0.00,0.21*

Abbreviations: SE, standard error; CI, confidence interval

*

p≤0.05

a

Data are based on multivariable linear models adjusted for practice effect

Results for Explicit Bias (Table 2)

Although measures of explicit bias showed slight pro-white bias, the effect size (Cohen’s d) for implicit bias was over three times as large as the explicit measures, and there was no correlation between implicit and explicit racial bias at either time point (Table 2). Explicit bias did not vary by resident demographic characteristics (race/ethnicity, age, gender, specialty, training year), nor did it differ post-shift compared to pre-shift (data not shown).

DISCUSSION

To the best of our knowledge this is the first published study to examine how an ED-shift impacts levels of physician implicit racial bias. This is important because the ED is characterized by time-pressure, competing demands, overcrowding, stress, and fatigue, making providers in the ED more prone to the use of heuristics such as racial bias. Our finding that the majority of physicians working in the ED exhibited implicit pro-white bias is consistent with most other studies examining levels of implicit bias among other types of healthcare providers.22-31 In contrast to our primary hypothesis, levels of implicit racial bias remained stable overall pre- to post-shift. However, our secondary hypothesis that cognitive stressors encountered during an ED shift would be associated with higher levels of bias was supported, in that ED overcrowding and caring for more patients during a shift were associated with greater post-shift implicit bias.

Our hypotheses were based on research showing that decisions become more difficult as more decisions must be made and that such decision fatigue increases the brain’s reliance on heuristics.52 They were also based on psychological research showing that experimentally increasing cognitive stress produces increased levels of implicit bias and more sterotyping behavior.37,52-55 In the ED, cognitive stress may come from competing mental tasks (e.g. decision-making for multiple patients, interruptions while writing orders), environmental factors (e.g. overcrowding, inadequate staffing, noise), provider psychological or physical state (e.g. fatigue, stress, hunger), and the level of difficulty associated with clinical tasks. Although it has been hypothesized that characteristics of the healthcare setting increase cognitive load and may increase bias,19,20 we are the first to directly test this hypothesis and show that characteristics of the ED work environment (overcrowding and patient load) increase physician implicit racial bias.

Although our objective measures of cognitive stress (NEDOCS and number of patients) were associated with implicit bias, we did not find associations between implicit bias and more subjective measures of cognitive stress (fatigue, busyness, or stressfulness of the shift). The lack of association between these self-reported measures and bias could be related to modest levels of fatigue, busyness, and stress reported during the study period. This study was performed between April and June, which is typically a lower volume season in pediatric EDs. We also chose not to recruit physicians before or after overnight shifts so as not to confound sleep deprivation with the effects of a shift in the ED. Results may have differed if the study was conducted during higher volume season or included physicians working overnight shifts. It is also possible that physicians are generally unaware of the effect that the ED work environment has on them, particularly when comparing the fatigue, busyness, and stress of 8 or 10-hour ED shifts to 24-hour inpatient shifts.

Our study focused on whether ED work environment affected overall levels of implicit bias. It was beyond the scope of this study to examine how cognitive load in the ED influences the application of bias through mechanisms such as disparate communication patterns, nonverbal behavior, and variation in care. Prior research has shown that implicit biases predict discriminatory behaviors outside of the healthcare setting.45,56 Within the context of healthcare, physician implicit bias has been linked with poor communication during visits27 and poor ratings of care among black patients.27,32 Other studies examining how implicit bias impacts patient care have had mixed results.22,23,26,29-31,57 Higher implicit pro-white bias was not associated with clinical assessments of patients by medical students26, recommendations for total knee replacement for osteoarthritis 29, or assessment of trauma patients31 using vignettes. However, implicit pro-white bias has been linked with racial differences in thrombolysis treatment recommendations for chest pain22 and prescribing narcotic analgesics for children with postoperative pain23 using vignettes. One study that examined the effect of time pressure on the association between racial bias and clinical decision making among 81 physicians found that racial bias was associated with differences in assessment of case vignettes depicting black and white patients with chest pain only when time pressure was experimentally induced.58

Only two studies have investigated the effect on implicit physician bias on actual care. The first study among 14 physicians caring for 162 patients with spinal cord injury found higher implicit bias was associated with poorer psychosocial health outcomes, including social integration, depression, and life satisfaction.30 The second study among 138 primary care physicians and 4,794 of their patients with hypertension found no link between levels of physician implicit bias with hypertension treatment intensification, patient adherence, or blood pressure control.57

There are no known studies investigating the impact of bias on actual patient care in the ED. The pediatric ED is an important setting to study the impact of bias on disparities because minority children are more likely than white children to utilize the ED,59 and racial disparities have been well documented in ED care.60-63 Furthermore, the ED has unique characteristics that distinguish it from the primary care setting, including the lack of a patient provider relationship, overcrowding, and higher patient acuity, which may yield different results from studies in outpatient settings. Given the unique characteristics of the ED work environment, investigating the link between cognitive stressors, provider bias, and racial/ethnic variation in ED care represents an important next step in disparities research to help determine the clinical significance of our findings.

Our findings suggest that strategies to improve staffing and prevent overcrowding may reduce implicit provider bias and its potential impact on patient care in the ED. This may be especially true for institutions that serve a large number of minority patients. Although not examined in this study, addressing other factors that can contribute to cognitive stress in the ED (e.g. interruptions, time pressure, not eating) and use of evidence-based decision support tools to decrease cognitive load may reduce the impact of bias on clinical decision-making.

Interventions have been proposed to mitigate the effects of implicit provider bias on healthcare disparities at the provider, patient, and system level, although these have not yet been tested in the ED setting.35,64-69 Examples include using positive image priming to counter automatic stereotyping,35,64,65 self-affirmation to reduce the potential impact of stereotype threat on communication,64,66-69 and system-level interventions aimed at improving quality and reducing disparities.64 Until future studies test and validate such interventions in the ED, it is important to make ED providers aware of their implicit racial bias, which is over three times greater than their conscious bias. Residents and the providers who train and educate residents in the ED should also be aware of how cognitive stressors, including ED overcrowding and patient load, may exacerbate their baseline levels of bias.

LIMITATIONS

Study limitations warrant consideration when interpreting our results. First, this study was performed in a sample of residents at a single institution, which may limit the generalizability of our results. Due to the small sample size of black (n=1) and Hispanic participants (n=4), we were unable to perform sub-analyses by physician race. Other research has demonstrated that black physicians on average show no racial bias.24 It is possible that we would find different results if the study was conducted with a more diverse group of participants, or in a setting that serves a more racially and ethnically diverse patient population. However, the race of our participants is similar to those of practicing physicians in the US. Due to the nature of the study design, we can report associations between cognitive stressors and bias, but cannot determine causality. Although we examined several aspects of cognitive load (NEDOCS, patient number, fatigue, business, and stressfulness), there may be other unmeasured factors, such as the level of complexity in diagnostic decision making, that may also impact cognitive load in the ED. Finally, the racial makeup of patients seen during the shift was not measured, and therefore cannot be examined in relation to post-shift bias. Strengths of our study include assessment of cognitive stressors in the ED that influence implicit bias using a repeated measures design with a high participant response rate.

CONCLUSIONS

We did not find greater levels of bias post-shift compared to pre-shift as hypothesized. However, we provide new evidence that cognitive stressors can increase implicit bias, including ED overcrowding and caring for more patients during an ED shift. Efforts should be made to increase awareness of clinicians in the ED about their implicit racial biases, which are in contrast to their explicit egalitarian attitudes. Moderators of bias such as patient volume should be taken into consideration in future studies that seek to understand how bias impacts patient care in the ED, as well as intervention studies that seek to mitigate disparities in care by addressing provider bias.

Acknowledgments

Funding Sources: This research was conducted while Dr. Johnson was a fellow at the Children’s Hospital of Pittsburgh and supported by a grant from the Agency for Healthcare Research and Quality (T32 HS 017587). Data analysis and manuscript preparation was completed while Dr. Johnson was funded by the National Heart Lung and Blood Institute (K12 HL109009). The project was also supported by the University of Pittsburgh Clinical and Translational Science Institute (CTSI) through the National Institutes of Health through Grant Number UL1-TR-000005. The sponsoring agencies had no role in the design and conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript. The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the sponsoring agencies.

Footnotes

Prior Presentations: Findings reported in this manuscript were presented as an oral abstract at the Eastern Society for Pediatric Research Annual Meeting, Philadelphia, Pennsylvania, April 2014; and the Pediatric Academic Societies Meeting, Vancouver, British Columbia, Canada, May 2014. No complete reports or full data in the form of tables or figures have been distributed to conferences attendees or journalists.

There are no financial disclosures or conflicts of interest to report

References

  • 1.Kovacs G, Croskerry P. Clinical decision making: an emergency medicine perspective. Acad Emerg Med. 1999;6(9):947–52. doi: 10.1111/j.1553-2712.1999.tb01246.x. [DOI] [PubMed] [Google Scholar]
  • 2.Croskerry P, Sinclair D. Emergency medicine: a practice prone to error? CJEM. 2001;3(4):271–6. doi: 10.1017/s1481803500005765. [DOI] [PubMed] [Google Scholar]
  • 3.Krug SE, Frush K. Patient safety in the pediatric emergency care setting. Pediatrics. 2007;120(6):1367–75. doi: 10.1542/peds.2007-2902. [DOI] [PubMed] [Google Scholar]
  • 4.Coiera EW, Jayasuriya RA, Hardy J, Bannan A, Thorpe ME. Communication loads on clinical staff in the emergency department. Med J Aust. 2002;176(9):415–8. doi: 10.5694/j.1326-5377.2002.tb04482.x. [DOI] [PubMed] [Google Scholar]
  • 5.Laxmisan A, Hakimzada F, Sayan OR, Green RA, Zhang J, Patel VL. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform. 2007;76(11-12):801–11. doi: 10.1016/j.ijmedinf.2006.09.019. [DOI] [PubMed] [Google Scholar]
  • 6.Joffe M. Emergency department provider fatigue and shift concerns. Clin Pediatr Emerg Med. 2006;7(4):248–54. [Google Scholar]
  • 7.West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;302(12):1294–300. doi: 10.1001/jama.2009.1389. [DOI] [PubMed] [Google Scholar]
  • 8.Firth-Cozens J, Greenhalgh J. Doctors’ perceptions of the links between stress and lowered clinical care. Soc Sci Med. 1997;44(7):1017–22. doi: 10.1016/s0277-9536(96)00227-4. [DOI] [PubMed] [Google Scholar]
  • 9.Williams ES, Manwell LB, Konrad TR, Linzer M. The relationship of organizational culture, stress, satisfaction, and burnout with physician-reported error and suboptimal patient care: results from the MEMO study. Health Care Manage Rev. 2007;32(3):203–12. doi: 10.1097/01.HMR.0000281626.28363.59. [DOI] [PubMed] [Google Scholar]
  • 10.Ayas N, Barger LK, Cade BE, et al. Extended work duration and the risk of self-repoted percutaneous injuris in interns. JAMA. 2006;296(9):1055–62. doi: 10.1001/jama.296.9.1055. [DOI] [PubMed] [Google Scholar]
  • 11.Barger L, Cade BE, Ayas NT, et al. Harvard work hours, health, and safety group Extended work shifts and the risk of motor vehicle crashes among interns. N Engl J Med. 2005;352(2):125–34. doi: 10.1056/NEJMoa041401. [DOI] [PubMed] [Google Scholar]
  • 12.Schull MJ, Vermeulen M, Slaughter G, Morrison L, Daly P. Emergency department crowding and thrombolysis delays in acute myocardial infarction. Ann Emerg Med. 2004;44(6):577–85. doi: 10.1016/j.annemergmed.2004.05.004. [DOI] [PubMed] [Google Scholar]
  • 13.Pines JM, Localio AR, Hollander JE, et al. The impact of emergency department crowding measures on time to antibiotics for patients with community-acquired pneumonia. Ann Emerg Med. 2007;50(5):510–6. doi: 10.1016/j.annemergmed.2007.07.021. [DOI] [PubMed] [Google Scholar]
  • 14.Timm NL, Ho ML, Luria JW. Pediatric emergency department overcrowding and impact on patient flow outcomes. Acad Emerg Med. 2008;15:832–7. doi: 10.1111/j.1553-2712.2008.00224.x. [DOI] [PubMed] [Google Scholar]
  • 15.Pines JM, Prabhu A, Hilton JA, Hollander JE, Datner EM. The effect of emergency department crowding on length of stay and medication treatment times in discharged patients with acute asthma. Acad Emerg Med. 2010;17:834–9. doi: 10.1111/j.1553-2712.2010.00780.x. [DOI] [PubMed] [Google Scholar]
  • 16.Sills MR, Fairclough D, Ranade D, Kahn MG. Emergency department crowding is associated with decreased quality of care for children with acute asthma. Ann Emerg Med. 2011;57(3):191–200. doi: 10.1016/j.annemergmed.2010.08.027. [DOI] [PubMed] [Google Scholar]
  • 17.Kennebeck SS, Timm NL, Kurowski EM, Byczkowski TL, Reeves SD. The association of emergency department crowding and time to antibiotics in febrile neonates. Acad Emerg Med. 2011;18:1380–5. doi: 10.1111/j.1553-2712.2011.01221.x. [DOI] [PubMed] [Google Scholar]
  • 18.Depinet HE, Iyer SB, Hornung R, Timm NL, Byczkowski TL. The effect of emergency department crowding on reassessment of children with critically abnormal vital signs. Acad Emerg Med. 2014;21:1116–20. doi: 10.1111/acem.12478. [DOI] [PubMed] [Google Scholar]
  • 19.Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med. 2002;9:1184–204. doi: 10.1111/j.1553-2712.2002.tb01574.x. [DOI] [PubMed] [Google Scholar]
  • 20.Burgess DJ. Are providers more likely to contribute to healthcare disparities under high levels of cognitive load? How features of the healthcare setting may lead to biases in medical decision making. Med Decis Making. 2010;30(2):246–57. doi: 10.1177/0272989X09341751. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Greenwald A, Banaji MR. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychol Rev. 1995;102(1):4–27. doi: 10.1037/0033-295x.102.1.4. [DOI] [PubMed] [Google Scholar]
  • 22.Green AR, Carney DR, Pallin DJ, et al. Implicit bias among physicians and its prediction of thrombolysis decisions for black and white patients. J Gen Intern Med. 2007;22(9):1231–8. doi: 10.1007/s11606-007-0258-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Sabin JA, Rivara FP, Greenwald AG. Physician implicit attitudes and stereotypes about race and quality of medical care. Med Care. 2008;46(7):678–85. doi: 10.1097/MLR.0b013e3181653d58. [DOI] [PubMed] [Google Scholar]
  • 24.Sabin J, Nosek BA, Greenwald A, Rivara FP. Physicians’ implicit and explicit attitudes about race by MD race, ethnicity, and gender. J Health Care Poor Underserved. 2009;20(3):896–913. doi: 10.1353/hpu.0.0185. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.White-Means S, Zhiyong D, Hufstader M, Brown LT. Cultural competency, race, and skin tone bias among pharmacy, nursing, and medical students: implications for addressing health disparities. Med Care Res Rev. 2009;66(4):436–55. doi: 10.1177/1077558709333995. [DOI] [PubMed] [Google Scholar]
  • 26.Haider AH, Sexton J, Sriram N, et al. Association of unconscious race and social class bias with vignette-based clinical assessments by medical students. JAMA. 2011;306(9):942–51. doi: 10.1001/jama.2011.1248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Cooper LA, Roter DL, Carson KA, et al. The associations of clinicians’ implicit attitudes about race with medical visit communication and patient ratings of interpersonal care. Am J Public Health. 2012;102(5):979–87. doi: 10.2105/AJPH.2011.300558. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Blair IV, Havranek EP, Price DW, et al. Assessment of biases against Latinos and African Americans among primary care providers and community members. Am J Public Health. 2013;103(1):92–8. doi: 10.2105/AJPH.2012.300812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Oliver MN, Wells KM, Joy-Gaba JA, Hawkins CB, Nosek BA. Do physicians’ implicit views of African Americans affect clinical decision making? J Am Board Fam Med. 2014;27(2):177–88. doi: 10.3122/jabfm.2014.02.120314. [DOI] [PubMed] [Google Scholar]
  • 30.Hausmann LR, Myaskovsky L, Niyonkuru C, et al. Examining implicit bias of physicians who care for individuals with spinal cord injury: A pilot study and future directions. J Spinal Cord Med. 2014 Jan 21; doi: 10.1179/2045772313Y.0000000184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Haider AH, Schneider EB, Sriram N, et al. Unconscious race and class bias: its association with decision making by trauma and acute care surgeons. J Trauma Acute Care Surg. 2014;77(3):409–16. doi: 10.1097/TA.0000000000000392. [DOI] [PubMed] [Google Scholar]
  • 32.Blair IV, Steiner JF, Fairclough DL, et al. Clinicians’ implicit ethnic/racial bias and perceptions of care among black and latino patients. Ann Fam Med. 2013;11(1):43–52. doi: 10.1370/afm.1442. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Gawronski B, Strack F. On the propositional nature of cognitive consistency: dissonance changes explicit, but not implicit attitudes. J Exp Soc Psychol. 2003;40:535–42. [Google Scholar]
  • 34.Gregg AP, Seibt B, Banaji MR. Easier done than undone: asymmetry in the malleability of implicit preferences. J Pers Soc Psychol. 2006;90(1):1–20. doi: 10.1037/0022-3514.90.1.1. [DOI] [PubMed] [Google Scholar]
  • 35.Dasgupta N, Greenwald AG. On the malleability of automatic attitudes: combating automatic prejudice with images of admired and disliked individuals. J Pers Soc Psychol. 2001;81(5):800–14. doi: 10.1037//0022-3514.81.5.800. [DOI] [PubMed] [Google Scholar]
  • 36.Dasgupta N. Implicit attitudes and beliefs adapt to situations: a decade of research on the malleability of implicit prejudice, stereotypes, and the self-concept. In: Devine P, Plant A, editors. Advances in experimental scoial psychology. Vol. 47. Academic Press; 2013. pp. 233–79. [Google Scholar]
  • 37.Govorun O, Payne K. Ego-depletion and prejudice: separating automatic and controlled components. Soc Cognition. 2006;24(2):111–36. [Google Scholar]
  • 38.Vaughn ED, Thomas A, Doyle AL. The Multiple Disability Implicit Association Test: Psychometric Analysis of a Multiple Administration IAT Measure. Rehabil Couns Bull. 2011;54(4):223–35. [Google Scholar]
  • 39.Thomas A, Vaughn D, Doyle A. Implementation of a computer based Implicit Association Test as a measure of attitudes toward individuals with disabilities. J Rehabil. 2007;73(2):3–14. [Google Scholar]
  • 40.Weiss SJ, Derlet R, Arndahl J, et al. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS) Acad Emerg Med. 2004;11:38–50. doi: 10.1197/j.aem.2003.07.017. [DOI] [PubMed] [Google Scholar]
  • 41.Greenwald AG, McGhee DE, Schwartz JL. Measuring individual differences in implicit cognition: the implicit association test. J Pers Soc Psychol. 1998;74(6):1464–80. doi: 10.1037//0022-3514.74.6.1464. [DOI] [PubMed] [Google Scholar]
  • 42.Nosek BA, Smyth FL. A multitrait-multimethod validation of the Implicit Association Test: implicit and explicit attitudes are related but distinct constructs. Exp Psychol. 2007;54(1):14–29. doi: 10.1027/1618-3169.54.1.14. [DOI] [PubMed] [Google Scholar]
  • 43.Nosek BA, Greenwald AG, Banaji MR. Understanding and using the Implicit Association Test: II. Method variables and construct validity. Pers Soc Psychol Bull. 2005;31(2):166–80. doi: 10.1177/0146167204271418. [DOI] [PubMed] [Google Scholar]
  • 44.Nosek BA, Greenwald AG, Banaji MR. The Implicit Association Test at age 7: A methodological and conceptual review. Psychology Press; 2007. pp. 265–92. [Google Scholar]
  • 45.Greenwald AG, Poehlman TA, Uhlmann EL, Banaji MR. Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. J Pers Soc Psychol. 2009;97(1):17–41. doi: 10.1037/a0015575. [DOI] [PubMed] [Google Scholar]
  • 46.Wittenbrink B, Schwarz N. Implicit measures of attitudes. Guilford Press; New York, NY: 2007. [Google Scholar]
  • 47.Penner LA, Dovidio JF, West TV, et al. Aversive Racism and Medical Interactions with Black Patients: A Field Study. J Exp Soc Psychol. 2010;46(2):436–40. doi: 10.1016/j.jesp.2009.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Sabin JA, Greenwald AG. The influence of implicit bias on treatment recommendations for 4 common pediatric conditions: pain, urinary tract infection, attention deficit hyperactivity disorder, and asthma. Am J Public Health. 2012;102(5):988–95. doi: 10.2105/AJPH.2011.300621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Greenwald AG, Nosek BA, Banaji MR. Understanding and using the implicit association test: I. An improved scoring algorithm. J Pers Soc Psychol. 2003;85(2):197–216. doi: 10.1037/0022-3514.85.2.197. [DOI] [PubMed] [Google Scholar]
  • 50.Nosek BA. Moderators of the relationship between implicit and explicit evaluation. J Exp Psychol Gen. 2005;134(4):565–84. doi: 10.1037/0096-3445.134.4.565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Cohen J. Statistical power analysis for the behavioral sciences. 2nd L. Erlbaum Associates; Hillsdale, N.J.: 1988. [Google Scholar]
  • 52.Danziger S, Levav J, Avnaim-Pesso L. Extraneous factors in judicial decisions. PNAS. 2011;108(17):6889–92. doi: 10.1073/pnas.1018033108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Dijker AJ, Koomen W. Stereotyping and attitudinal effects under time pressure. Eur J Soc Psychol. 1996;26(1):61–74. [Google Scholar]
  • 54.Wigboldus DHJ, Sherman JW, Franzese HL, van Knippenberg A. Capacity and comprehension: Spontaneous stereotyping under cognitive load. Soc Cognition. 2004;22(3):292–309. [Google Scholar]
  • 55.Muroff JR, Jackson JS, Mowbray CT, Himle JA. The influence of gender, patient volume and time on clinical diagnostic decision making in psychiatric emergency services. Gen Hosp Psychiat. 2007;29(6):481–8. doi: 10.1016/j.genhosppsych.2007.08.008. [DOI] [PubMed] [Google Scholar]
  • 56.McConnell AR, Leibold JM. Relations among the implicit association test, discriminatory behavior, and explicit measures of racial attitudes. J Exp Soc Psychol. 2001;37:435–42. [Google Scholar]
  • 57.Blair IV, Steiner JF, Hanratty R, et al. An Investigation of Associations Between Clinicians’ Ethnic or Racial Bias and Hypertension Treatment, Medication Adherence and Blood Pressure Control. J Gen Intern Med. 2014 Feb 19; doi: 10.1007/s11606-014-2795-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Stepanikova I. Racial-ethnic biases, time pressure, and medical decisions. J Health Soc Behav. 2012;53(3):329–43. doi: 10.1177/0022146512445807. [DOI] [PubMed] [Google Scholar]
  • 59.Flores G, The Committee on Pediatric Research Technical Report-Racial and Ethnic Disparities in the Health and Health Care of Children. Pediatrics. 2010;125(4):e979–1020. doi: 10.1542/peds.2010-0188. [DOI] [PubMed] [Google Scholar]
  • 60.Rand CS, Butz AM, Kolodner K, Huss K, Eggleston P, Malveaux F. Emergency department visits by urban African American children with asthma. J Allergy Clin Immunol. 2000;105(1 Pt 1):83–90. doi: 10.1016/s0091-6749(00)90182-9. [DOI] [PubMed] [Google Scholar]
  • 61.Hambrook JT, Kimball TR, Khoury P, Cnota J. Disparities exist in the emergency department evaluation of pediatric chest pain. Congenit Heart Dis. 2010;5(3):285–91. doi: 10.1111/j.1747-0803.2010.00414.x. [DOI] [PubMed] [Google Scholar]
  • 62.Lane WG, Rubin DM, Monteith R, Christian CW. Racial differences in the evaluation of pediatric fractures for physical abuse. JAMA. 2002;288(13):1603–9. doi: 10.1001/jama.288.13.1603. [DOI] [PubMed] [Google Scholar]
  • 63.Johnson TJ, Weaver MD, Borrero S, et al. Association of Race and Ethnicity With Management of Abdominal Pain in the Emergency Department. Pediatrics. 2013;132(4):e851–8. doi: 10.1542/peds.2012-3127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Blair IV, Steiner JF, Havranek EP. Unconscious (implicit) bias and health disparities: where do we go from here? Perm J. 2011;15(2):71–8. doi: 10.7812/tpp/11.979. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Blair IV, Ma JE, Lenton AP. Imagining stereotypes away: the moderation of implicit stereotypes through mental imagery. J Pers Soc Psychol. 2001;81(5):828–41. doi: 10.1037//0022-3514.81.5.828. [DOI] [PubMed] [Google Scholar]
  • 66.Burgess DJ, Warren J, Phelan S, Dovidio J, van Ryn M. Stereotype threat and health disparities: what medical educators and future physicians need to know. J Gen Intern Med. 2010;25(Suppl 2):S169–77. doi: 10.1007/s11606-009-1221-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Cohen GL, Garcia J, Apfel N, Master A. Reducing the racial achievement gap: a social-psychological intervention. Science. 2006;313(5791):1307–10. doi: 10.1126/science.1128317. [DOI] [PubMed] [Google Scholar]
  • 68.Cohen GL, Garcia J, Purdie-Vaughns V, Apfel N, Brzustoski P. Recursive processes in self-affirmation: intervening to close the minority achievement gap. Science. 2009;324(5925):400–3. doi: 10.1126/science.1170769. [DOI] [PubMed] [Google Scholar]
  • 69.Miyake A, Kost-Smith LE, Finkelstein ND, Pollock SJ, Cohen GL, Ito TA. Reducing the gender achievement gap in college science: a classroom study of values affirmation. Science. 2010;330(6008):1234–7. doi: 10.1126/science.1195996. [DOI] [PubMed] [Google Scholar]

RESOURCES