Adapting the Everyday Discrimination Scale to Medical Settings: Reliability and Validity Testing in a Sample of African American Patients

Abstract

Objective

Despite evidence that discrimination within the health care system may play an important role in perpetuating health disparities, instruments designed to measure discrimination within the health care setting have not been adequately tested or validated. Consequently, we sought to test the psychometric properties of a modified version of the Everyday Discrimination scale, adapted for medical settings.

Design

Cross-sectional study.

Setting

Academic medical center in Chicago.

Participants

Seventy-four African American patients.

Outcome measures

We measured factor analysis, internal consistency, test-retest reliability, convergent validity and discriminant validity.

Results

Seventy-four participants completed the baseline interviews and 66 participants (89%) completed the follow-up interviews. Eighty percent were women. The Discrimination in Medical Settings (DMS) Scale had a single factor solution (eigenvalue of 4.36), a Cronbach’s alpha of 0.89 and test-retest reliability of .58 (P<.0001). The DMS was significantly correlated with an overall measure of societal discrimination (EOD) (r=.51, P<.001) as well as two of its three subscales (unfair: r=−.04, P=.76; discrimination: r=.45, P<0.001; worry: r=−.36, P=.002). The DMS was associated with the overall African American Trust in Health Care Scale (r=.27, P=.02) as well as two key subscales (racism: r=.31, P<.001; disrespect: r=.44, P<.001). The DMS scale was inversely associated with the Social Desirability Scale (r=.18, P=.13). The DMS scale was not correlated with the Center for Epidemiologic Studies Depression Scale (r=.03, P=.80).

Conclusions

The Discrimination in Medical Settings Scale has excellent internal consistency, test-retest reliability, convergent validity and discriminant validity among our sample of African American patients. Further testing is warranted among other racial/ethnic groups.

Introduction

Understanding how racial/ethnic discrimination contributes to health disparities has emerged as a key research area.^1–5 Although an extensive literature links experiences of societal discrimination to negative health outcomes,^6–8 significantly less is known about the impact of perceived discrimination occurring within medical settings on racial/ethnic disparities in health.⁸ Early evidence suggests that perceived racial discrimination in health care is associated with several important health outcomes, including lower satisfaction with care, reduced adherence to care, lower quality patient/provider communication, underutilization of preventive health services, and poor overall self-reported health.^9–12 Among patients with diabetes, self-reported health care discrimination is associated with worse diabetes control and diabetes complications (eg, retinopathy and diabetes-related foot disorders).^13,14

While it is not currently known if reports of health care discrimination vary by clinical setting (eg, outpatient physician offices vs emergency departments), it is known that vulnerable patients (eg, poor, uninsured, less educated) and racial/ethnic minorities are more likely to report such discrimination.^12–15 Discrimination in medical settings may be amenable to intervention because the health care system can address differences in health outcomes through quality improvement efforts that target racial/ethnic minority patients and their health care providers, and address barriers to the equi delivery of healthcare.¹⁶

The Institute of Medicine and the Agency for Healthcare Research and Quality have recommended additional research in racial/ethnic health care discrimination,^1,3 yet currently, there is no standard measurement approach or validated instruments to measure such perceptions.^17,18 Studies of health care discrimination often utilize single-item measures that have not been validated¹² or rely on instruments designed to measure societal discrimination.¹⁰

While several studies have modified measures of societal discrimination to clinical settings,^9,19 the psychometric testing of these adapted measures is generally absent from the published literature. Consequently, we sought to test the psychometric properties of a modified version of Williams’ Everyday Discrimination Scale (EDS) adapted to medical settings.⁹ The EDS^19,20 is one of the most widely utilized measures of self-reported discrimination and has been validated across a variety of populations.^19,21 The Discrimination in Medical Settings (DMS) Scale is a modified version of the EDS that has been previously used to study perceived discrimination in medical settings, among patients with a range of clinical conditions (eg, diabetes, HIV/AIDs, breast cancer screening), but whose psychometric properties (eg, test-retest reliability, construct validity) are largely unknown.^9,22 Our study examined the reliability and construct validity of the DMS scale in an urban sample of African Americans with diabetes.

Methods

Participants

This study was part of a larger study of patient/provider communication among patients with diabetes, for which eligible participants were African-Americans adults (≥21 years) who had an established relationship with an attending primary care physician (defined as ≥3 visits over the preceding 2 years) at an urban academic internal medicine practice. After receiving approval from the institutional review board, a letter was sent to physicians explaining the study and requesting permission to recruit their patients. All physicians gave consent. Study participants were recruited using purposeful sampling,²³ and were identified by searching administrative databases for patient visit information and ICD-9-CM codes for diabetes. All patients in the study had a diagnosis of diabetes in their record.

Three attempts were made to contact participants via telephone. In addition, culturally-appropriate recruitment materials were posted in the clinic. Study participants received a $15 grocery store gift card as an incentive. The study was conducted from November 2007 until August 2008. Patients were interviewed at baseline and at 2 months by a race-concordant interviewer, as this has been shown to facilitate accurate data collection, particularly when discussing race-sensitive topics (eg, discrimination).²⁴

Study Measures

Discrimination in Medical Settings

The DMS Scale was adapted from existing discrimination scales based on prior studies of health care discrimination (Table 1).^9,22 It modified items from the EDS,^19,20 a nine-item instrument that asks about the frequency of experiences with everyday mistreatment. The EDS has shown high levels of internal consistency, convergent validity and divergent validity among African American men and women,^19–21 and has been widely used in studies of discrimination and health.^25,26

Table 1.

Discrimination in Medical Settings Scale

Item	Test sample		Retest sample		Theoretical range	Factor loading
	Mean (SD)	Observed range	Mean (SD)	Observed range
You are treated with less courtesy than other people.	1.77 (.83)	1.0–5.0	1.59 (.61)	1.0–3.0	1.0–5.0	.84
You are treated with less respect than other people.	1.74 (.80)	1.0–5.0	1.58 (.70)	1.0–3.0	1.0–5.0	.89
You receive poorer service than others.	1.66 (.75)	1.0–5.0	1.67 (.81)	1.0–5.0	1.0–5.0	.88
A doctor or nurse acts as if he or she thinks you are not smart.	1.63 (.80)	1.0–5.0	1.44 (.66)	1.0–3.0	1.0–5.0	.82
A doctor or nurse acts as if he or she is afraid of you.	1.33 (.58)	1.0–5.0	1.20 (.44)	1.0–3.0	1.0–5.0	.50
A doctor or nurse acts as if he or she is better than you.	1.81 (.94)	1.0–5.0	1.52 (.75)	1.0–3.0	1.0–5.0	.76
You feel like a doctor or nurse is not listening to what you were saying.	1.99 (.90)	1.0–5.0	1.70 (.76)	1.0–3.0	1.0–5.0	.76
All 7 items	1.71 (.63)	1.0–5.0	1.53 (.50)	1.0–5.0	1.0–5.0

We modified the EDS based on its performance in other health care settings and feedback from cognitive interviews (n=20) in this study. We began with the 9 EDS items, 6 of which had been adapted previously to clinical settings by Bird et al,⁹ and 3 of which were modified by members of the research team. Results from cognitive interviews indicated that patients did not interpret, “people think you are being dishonest” consistently and felt that the items, “being called names/insulted,” and “being threatened/harassed” were extreme and unlikely health care occurrences; these three items were dropped. We also added an item assessing patient/provider communication (provider listening) utilized in prior EDS adaptations.⁹ The resulting 7-item instrument was identical to that used by Bird et al,⁹ and asked about prior experiences of mistreatment while getting health care that persons attributed to race, ancestry or national origin. Examples include, “You are treated with less respect than other people,” “You feel like a doctor or nurse is not listening to what you were saying,” and “A doctor or nurse acts as if he or she thinks you are not smart.” Responses were assessed with a 5-point Likert scale (1-never, 2-rarely, 3-sometimes, 4-most of the time, 5-always).

Racial Discrimination (Societal)

The Krieger Experiences of discrimination (EOD) scale²⁷ is a 17-item measure of racial discrimination that has been validated (Cronbach’s alpha: >.74, convergent and discriminant validity) in a multi-ethnic population and is widely used in population-based health research.^28,29 It measures racial discrimination in 9 different settings (eg, educational, housing, employment) and includes one item on health care discrimination. The EOD is associated with psychological distress and health behaviors (ie, cigarette smoking) among African-Americans (Table 2).

Table 2.

Other measured study variables

Measure	Response format	Sample item from scale	Mean (SD)	Observed range	Theoretical range
African-American trust in health care	1 (never true) to 5 (always true)
Racism		Hospital clerks will often serve Whites before Blacks.	2.33 (.74)	1.0–4.0	1.0–5.0
Hidden agenda		Hospitals cover up their medical mistakes.	2.84 (.59)	1.6–4.4	1.0–5.0
Doctor distrust		Doctors think of themselves before their patient’s well-being.	2.22 (.63)	1.0–3.5	1.0–5.0
Doctor trust		Doctors care about their patients as people.	3.84 (.67)	2.0–5.0	1.0–5.0
Disrespect		Sometimes doctors treat black patients with disrespect.	2.09 (.74)	1.0–4.0	1.0–5.0
General racial discrimination (EOD)
Response to unfair treatment	engaged response vs passive response	If you feel you have been treated unfairly, do you usually: ________.	1.64 (.61)	2.0–4.0	2.0–4.0
Experience of discrimination	yes or no	Have you ever experienced discrimination, been prevented from doing something, or been hassled or made to feel inferior in any of the following situations because of your race, ethnicity, or color?	4.06 (2.82)	9.0–18.0	9.0–18.0
Worry about unfair treatment due to race	1 (rarely/never of the time) to 3 (most of the time)	When you were a child or teenager (up to age 18), how much did you worry about people in your racial/ethnic group experiencing unfair treatment because of their race, ethnicity, or color?	1.21 (.50)	1.0–3.0	1.0–3.0
Global racial discrimination	1 (never) to 4 (often)	How often do you feel that racial/ethnic groups, who are not White, such as African Americans and Latinos, are discriminated against?	2.81 (.74)	1.0–4.0	1.0–4.0
Depression (CES-D)	1 (rarely or none of the time [<1 day]) to 4 (most or all of the time [5–7 days])	During this past week, I was bothered by things that usually don’t bother me.	12.15 (9.99)	0.0–39.0	0.0–60.0
Social desirability (MC-Form C)	true or false	I never hesitate to go out of my way to help someone in trouble.	19.25 (4.23)	14.0–24.0	13.0–26.0

Trust in Health Care

The African American Trust in Health Care scale is a 24-item instrument that was validated (Cronbach’s alpha: >.66, convergent validity) in a population of urban African-Americans³⁰ and correlates with prior negative experiences in health care.

Depression

The Center for Epidemiologic Studies Depression Scale (CES-D)³¹ is a 20-item validated instrument (Cronbach’s alpha: >.84, concurrent, convergent and discriminant validity) developed by the National Institute of Mental Health as a screening tool for clinical depression. Societal discrimination has been associated with depression and other measures of psychological distress.¹⁹ The CES-D has been widely used across clinical and community samples,³² and has been validated in several large cohort studies of African-Americans.³³

Social Desirability

The Modified Marlowe-Crowne Social Desirability Scale (MC-Form C)³⁴ is a 13-item modified instrument that has been validated (Kuder-Richardson formula reliability estimate: .76, concurrent validity). It measures the need to have approval from others by responding in socially accepted ways; social desirability is commonly used in research on self-reported measures of affect, personality, attitudes and perceptions.³⁴

Data Analysis

We examined the factor structure of the DMS using an exploratory principal components factor analysis with vari-max rotation. Exploratory (rather than confirmatory) factor analysis was used because the DMS scale is an adapted version of the EDS, which has demonstrated both a one-factor²⁷ and two-factor²¹ solution across samples. Consequently, it was difficult to generate an a priori hypothesis about the factor structure of the DMS in the current sample.

Reliability was assessed through internal consistency and test-retest reliability. Internal consistency was assessed through Cronbach’s alpha in both the original sample and the retest sample. Due to the skewed nature of the data, nonparametric methods were utilized. Test-retest reliability was assessed by Spearman rank correlations between baseline scores and at two month follow-up.

We also assessed construct validity of the DMS. Convergent validity was assessed through Spearman correlations between the DMS and the EOD Scale, CES-D, and African American Trust in Health Care Scale. The Spearman correlation between the DMS and the Modified Marlowe-Crowne Social Desirability Scale (MC-Form C) was used to test discriminant validity. Large, significant correlations were considered supportive of convergent validity and small, non-significant correlations supportive of discriminant validity. Correlations used to assess discriminant validity were judged to be small or not small through comparison to correlations supporting convergent validity. Of note, we elected to use Spearman’s correlations because of the distributional nature of the data; our results were similar, however, to the same analyses using Pearson’s correlations (data not shown).

Differences in DMS scores by patient demographic characteristics and self-rated health status were tested using the Wilcoxon rank sum test (because of the skewed data) and the Kruskal-Wallis analysis of variance by ranks. We did not assess for the effects of race concordance because of the low number of race concordant patient/provider pairs (<5%). STATA 10.0 was used for all analyses. Statistical significance was defined as a two-tailed P<.05.

Results

Seventy-four participants completed the baseline interviews and 66 participants (89%) completed the follow-up interviews at 2 months. Study participants were aged an average of 66 years and the majority were female (80%) (Table 3). The mean duration of diabetes was 13.5 years and approximately 40% of participants described their health as fair or poor. The majority of patient/provider dyads (96%) were racially discordant.

Table 3.

Participant demographics (n=74)

	%*
Female	80
Age (mean, yrs)	66
<55	12
55–64	31
65–74	35
≥75	22
Education
Some high school	15
High school graduate	25
Some college	36
College graduate	24
Insurance
Uninsured	0
Medicare	64
Medicaid	27
Medicare + Medicaid	73
Private Insurance	20
Medicare + private	42
Self reported health status
Excellent	4
Very good	12
Good	42
Fair	34
Poor	7
Refused/no answer	1
Co-morbid conditions
None	19
Stroke	9
Coronary artery disease	19
Hypertension	76
Hyperlipidemia	47
Peripheral vascular disease	32
Duration of Diabetes, mean	13.5 yrs

^*Values in the table reflect percentages with the exception of the mean age and the mean duration of diabetes which are reported in years.

The DMS had an overall mean of 1.71 (SD .63) in the original sample and 1.53 (.50) in the retest sample (Table 1). The DMS item with the highest mean value was “You feel like a doctor or nurse is not listening to what you were saying” (1.99 [SD .90] in the original sample and 1.70 [.76] in the retest sample), and the item with the lowest mean value was “A doctor or nurse acts as if he or she is afraid of you” (1.33 [.58] in the original sample and 1.20 [.44] in the retest sample).

The factor analysis yielded a single factor solution. The eigenvalue of the single factor was 4.36, and accounted for 62% of the variance. No other factors had eigenvalues greater than 1. All items loaded on the first factor, with 6 of the items on the DMS scale having loadings greater than .70 (Table 1). Only one item “A doctor or nurse acts as if he or she is afraid of you” had a loading less than .7 at .5.

The DMS Scale had a Cronbach’s alpha of .89 in the original sample and .85 in the retest sample. The test-retest reliability was .58 (P<.001). The DMS was significantly correlated with the overall measure of societal discrimination (EOD) (r=.51, P<.001) as well as two of its three subscales (unfair: r=.02, P=.85; discrimination: r=.45, P<.001; worry: r=−.43, P=.001) (Table 4). The DMS was associated with the overall African American Trust in Health Care scale (r=.27, P=.02) as well as two key subscales (racism: r=.31, P<.001; disrespect: r=.44, P<.001). The DMS scale was inversely associated with the Social Desirability scale (r=.18, P=.13). The DMS scale was not correlated with the CES-D (r=.03, P=.80).

Table 4.

DMS Scale Spearman rank correlations

	rho	P
Experiences of Discrimination (EOD)
Unfair	.02	.85
Discrimination	.45	<.001
Worry	−.43	<.001
Global	.55	<.001
African American trust in health care
Overall	.27	.02
Racism	.31	<.001
Hidden agenda	.1	.40
Distrust	.21	.06
Trust	−.09	.43
Disrespect	.44	<.001
Depression (CES-D)	.03	.83
Social desirability (MC-Form C)	.18	.13

There were no statistically different differences in DMS scores by sex, age, or self-reported health status (Table 5). DMS scores did vary by education. For example, college graduates had higher median DMS scores than those who did not complete high school (2.14 vs 1.14 [P=.003] in the original sample and 1.86 vs 1.36 [P=.02] in the retest sample).

Table 5.

DMS scores by sociodemographics and health status

All participants	Test sample			Retest sample
	Mean (SD)	Median	P	Mean (SD)	Median	P
	1.71 (.63)	1.71	–	1.53 (.50)	1.5	–
Sex
Female	1.68 (.64)	1.71	.30	1.51 (.50)	1.36	.67
Age (mean, yrs)			.76			.78
<55	1.63 (.60)	1.86		1.54 (.56)	1.43
55–64	1.80 (.81)	1.71		1.51 (.44)	1.57
65–74	1.60 (.51)	1.43		1.63 (.62)	1.71
≥75	1.78 (.53)	1.86		1.40 (.39)	1.29
Education			.003			.02
Some high school	1.44 (.52)	1.14		1.51 (.58)	1.36
High school graduate	1.46 (.56)	1.14		1.28 (.31)	1.21
Some college	1.69 (.48)	1.79		1.48 (.48)	1.43
College graduate	2.16 (.74)	2.14		1.82 (.52)	1.86
Self reported health status			.36			.75
Excellent	1.64 (.71)	1.64		1.86 (1.01)	1.86
Very Good	2.00 (.52)	2		1.73 (.74)	1.86
Good	1.63 (.50)	1.71		1.47 (.43)	1.36
Fair	1.66 (.80)	1.36		1.46 (.49)	1.29
Poor	2.00 (.60)	1.86		1.63 (.22)	1.71

Discussion

This study examined the psychometric properties of the DMS Scale in a sample of African American patients with diabetes. Based on the Williams Everyday Discrimination Scale,¹⁹ the DMS Scale was designed to measure routine experiences of discrimination with health care providers and staff. Consistent with current recommendations, this discrimination scale is multi-item, and includes a range of possible experiences.^27,35

Findings indicate that the DMS Scale has a single factor solution, and good internal consistency and test-retest reliability. It was significantly correlated with the African American Trust in Health Care Scale,³⁰ a measure of overall mistrust of the health care system, including key subscales such as disrespect and racism. The DMS also had significant positive correlations with the Experiences of Discrimination Scale,²⁷ a measure of societal experiences of discrimination across multiple domains. Thus, the scale has demonstrated good convergent validity. The DMS Scale was not significantly associated with a measure of social desirability, providing evidence of good discriminant validity as well.

The DMS Scale was not associated with depressive symptoms as measured by the CES-D. This contrasts prior studies which reported positive correlations between discrimination and depressive symptoms.³⁶ It is possible that discrimination in medical settings operates differently than other forms of societal discrimination. That is, while societal discrimination may induce or exacerbate depressive symptoms in people, health care discrimination may result in physician-specific behavioral responses, such as nonadherence to treatment recommendations for depression. Wagner et al found that a global measure of discrimination predicted depression among African-Americans with diabetes while a health care discrimination measure generally did not, although it did predict nonadherence to antidepressant medication.³⁶

Scores on the DMS differed by education, with college-educated African Americans reporting significantly more discrimination in medical settings than less educated African Americans. This is consistent with some reports of societal discrimination, where African Americans from higher socioeconomic status (SES) backgrounds reported higher levels of discrimination than their lower-SES counterparts.³⁷ This may be partly due to their higher concentration in integrated settings, and more frequent interactions with Whites.³⁷ Consequently, higher-SES African Americans may have more prior exposure to discriminatory experiences and may therefore be more apt to recognize it in medical settings.

African American men had slightly, but not significantly, higher scores on the DMS than African American women. This is consistent with prior research with societal discrimination.³⁸ However, this finding is inconsistent with at least one prior report on medical discrimination, where African American women reported more discrimination than men.³⁹ The small proportion of men in our sample makes it difficult to draw substantive conclusions about our pattern of results. Thus, additional research on sex differences in exposure to health care discrimination is warranted.

Actual DMS responses ranged from never experiencing racial discrimination within health care to always experiencing such discrimination for each item. The mean score was 1.71 in the original sample and 1.53 in the retest sample, indicating that the majority of participants reported never or rarely experiencing racial discrimination in health care. This pattern is consistent with studies of societal discrimination that used the Everyday Discrimination Scale, upon which the DMS is based.⁷ What is particularly noteworthy is that reports of such experiences, despite being relatively uncommon, are associated with negative health outcomes. The DMS item that explored provider listening had the highest mean score, underscoring the importance of patient/provider communication to self-reported experiences of health care discrimination.¹¹

Participants in this study were African American patients with a medical provider – individuals who are engaged in the health care system. Because much of the literature about health care disparities has focused on differences within the health care system (eg, diabetes care, kidney transplants),³ it is important to understand, and have valid instruments to measure, perceptions of discrimination among patients actively involved in this system. In addition, all patients in the study had a diagnosis of diabetes. Epidemiological studies have shown associations between perceived discrimination and chronic diseases, and there is evidence that health care discrimination may be associated with worse diabetes outcomes.^13,14 However, findings from the current study may not generalize to the general population.

Limitations of the current study include the relatively small sample size potential selection bias. Patients with particularly strong personal encounters with the health care system may have been more likely to participate in the study. Third, all study participants had an established physician relationship and the majority (96%) of these relationships were racially discordant; these factors may have biased their responses. However, the DMS instrument does not ask exclusively about experiences within the current patient/physician relationship; experiences with other health care staff (eg, front desk staff, nurses) and prior physicians are also reflected in patient survey responses.

Nonetheless, our study has several strengths. The DMS Scale was validated in a sample of African Americans – a group that is both disproportionately burdened by disease⁴⁰ and disproportionately mistrustful of the health care system.⁴¹ Further, because African Americans are generally monolingual (and were in the current sample), validating the scale in this population allows for some distinction between perceptions of discrimination due to race versus language. Nonetheless, additional research is needed to determine whether the DMS Scale functions similarly across racial/ethnic groups.

To our knowledge, this study is the first to validate a measure of discrimination for use in medical settings. Findings suggest that the DMS is a valid and reliable instrument for assessing patient perceptions of discrimination by healthcare providers and staff. Future research should focus on: 1) examining associations between scores on the DMS and important health outcomes; 2) exploring whether reports of health care discrimination vary by medical setting (eg, emergency department vs outpatient office); and 3) identifying ways to reduce exposure to health care experiences that are perceived as discriminatory.