Ethnicity Effects on Clinical Diagnoses in Patients with Psychosis: Comparisons to Best Estimate Research Diagnoses

Abstract

Objective

Ethnicity effects on diagnoses are frequently reported and have variably been attributed to diagnostic biases verses ethnic differences in exposures to stress, substance use and other factors.

Method

We compared best estimate gold standard research diagnoses to clinical diagnoses among 129 Caucasian, 57 African American, and 50 Hispanic patients with psychosis admitted to an inpatient research unit.

Results

Clinical and research diagnoses showed significantly greater agreement in Hispanic than in African American patients (Caucasian patients were intermediate). Diagnostic agreement for paranoid schizophrenia was likewise the best in Hispanic patients. But while paranoid schizophrenia tended to be over-diagnosed in African American patients, it was under-diagnosed in the Caucasian patients. Patterns of diagnostic agreement for schizoaffective and “other” diagnoses were similar among the three ethnic groups.

Conclusions

Diagnostic unreliability may explain the excess of paranoid schizophrenia reported for African Americans. Further research is needed to elucidate the influence of ethnicity on clinical diagnosis before other theories to explain group differences can be reasonably proposed and reliably tested.

Treated prevalence studies consistently find African Americans are more likely than Caucasians to be diagnosed with schizophrenia^1–3 and less likely to receive psychotic affective and bipolar diagnoses.⁴ For example, although African American patients were less likely to self-report psychotic symptoms, they were more likely to be diagnosed with schizophrenia in a large sample of 19,219 inpatients and outpatients from a behavioral health system in New Jersey. Conversely, Latinos in this sample were less likely to be diagnosed with schizophrenia and more likely to receive affective disorder diagnoses, despite reporting more psychotic symptoms.⁵ Explanations for such discrepancies include biases and lack of cultural awareness by clinicians or the differential reporting of symptoms by ethnic subgroups, if there are no actual group differences, or racial/ethnic differences in genetic or environmental factors that influence the risks for psychiatric disorders, if such reports are valid.

Diagnostic practices could conceivably explain the group differences. Indeed, the findings of Trierweiler and colleagues’^6–8 suggest that even when using standardized diagnostic criteria, clinical judgment and clinicians’ characteristics play a differential role in how symptoms are attributed to African American and Caucasian patients. In their analyses, Strakowski and colleagues, find higher frequencies of schizophrenia among African American patients compared to Caucasians, despite similar rates of affective symptoms,⁹ and specifically psychotic mania.¹⁰

In comparison to the above described clinical diagnoses that varied with ethnicity, epidemiologic community research studies generally find no differences in schizophrenia diagnoses by ethnicity or that schizophrenia is less common among African Americans in controlled analyses.^11–12 Thus, various studies suggest racial biases exist in the diagnostic process.¹³ Questions concerning diagnostic accuracy must be resolved before genetic and environmental theories for these effects can be examined, since testing these hypotheses will rest on diagnostic validity and reliability.

One approach to investigating the possibility of clinician diagnostic errors is to compare diagnoses generated by treating clinicians with best-estimate research diagnoses that are generated during the same admission. Best-estimate research diagnostic procedures have greater diagnostic accuracy than clinical diagnoses¹⁴ and are appropriately used as “gold standard” assessments for research purposes.^15–16 Discordance between clinical diagnoses and best-estimate research diagnoses may indicate areas of inaccuracy for clinician-generated diagnoses. Previous studies find lower diagnostic agreement between the hospital diagnosis and the research diagnosis for African American patients compared to Caucasians.^{17, 18} Furthermore, the pattern of disagreement suggest clinicians often failed to identify affective symptoms in the African American patients.⁹ Sohler et al.¹⁹ also found African American patients had increased odds of receiving a discordant diagnosis than Caucasian patients, however, the magnitude of the effect was reduced in controlled analyses. To our knowledge, few findings have explicitly examined concordance between clinical and research diagnoses among both Hispanic patients and African American patients (but see Lawson et al.²⁰).

We examined the agreement between clinical and best-estimate research diagnoses by race/ethnicity and further probed the sensitivity and specificity of psychotic diagnoses within racial/ethnic groups. We hypothesized that African American patients would receive the most discordant diagnoses, and Caucasian patients the least. We hypothesized that Hispanic patients will also have lower concordance than Caucasian patients.

Method

The study involves a retrospective medical chart review of inpatients admitted to the Schizophrenia Research Unit (SRU) at The New York State Psychiatric Institute (NYSPI) from 1990 to 2003. The SRU is a 12-bed inpatient unit that is part of a research program that provides inpatient evaluation and clinical treatment for patients with severe mental illness. The average length of stay on the unit is 3 to 6 months and during this time, patients may participate in any number of research studies designed to study diagnostic, neurological, biochemical, physiological, and psychosocial aspects of severe mental illness. Patients who met the study criteria and provided written informed consent for the study were evaluated by the research team (~64% of the admitted cases). They participated in research studies under an NIMH Clinical Research Center Grant.²¹ Typically, the patients screened into the SRU were English speaking or Bilingual (Spanish), did not have primary active substance abuse problems or significant histories of violence, and had a psychotic condition. The present study is focused on the comparison of clinical hospital diagnoses made by attending psychiatrists or by psychiatry residents under their supervision, with the best-estimate consensus diagnoses made using research assessment data and clinical data.

Procedure

A comprehensive list of all patients evaluated on the SRU receiving a best-estimate consensus diagnosis from 1990 to 2003 was obtained from the computerized database. The medical charts available at NYSPI were matched to this list using unique patient ID numbers and pulled. The total number of matched patients was N=267. The clinical hospital diagnoses obtained from the medical chart, are made using typical clinical data obtained in hospital settings, (i.e., patient report of current symptoms, past psychiatric records obtained from previous hospitalizations at other hospitals in NYC, and any accompanying family or friends’ report of patients’ functioning). Patients who participate in the research protocols also participate in best-estimate consensus diagnostic procedures. These diagnoses are largely based upon information from face to face structured psychiatric interviews with the Diagnostic Interview for Genetic Studies (DIGS).²² The DIGS is used to assess lifetime and current psychiatric diagnoses. Its interrater reliability is kappa =.95 for DSM-IV diagnosis and kappa’s =.80 for individual symptoms.²² The consensus diagnosis is typically ascertained using these data as well as: admission information on age, gender, education, ethnicity, the age of onset of positive symptoms, the patient’s age at first treatment, global assessments of functioning for the worst period in the current episode and for the last month, past psychiatric records, and family interviews when possible. Typically, the best-estimate consensus diagnosis was made in a meeting that included the unit chief from the clinical team and the diagnostic experts of the research team. With a few exceptions, most of the diagnosticians on the SRU have been Caucasian, but since 1998, Hispanic representation among the team has increased.

This secondary analysis study was approved by the human subjects committee and all patients provide written informed consent to participate in research studies. The treating clinician’s primary discharge diagnosis was obtained from the discharge summary note of each chart located in medical records, and the best-estimate research diagnosis was obtained from the research database maintained on all patients admitted to the SRU. Patients’ race/ethnicity was obtained from this research database. Medical records were linked with the research database using unique patient ID numbers given to all SRU patients.

Data Analyses

The kappa statistic was used to determine diagnostic agreement between the clinical hospital diagnosis and best-estimate research diagnosis. Chi square tests were used to test whether the degree of agreement differed significantly between African American, Hispanic, and Caucasian patients. Sensitivity and specificity of the clinical diagnosis of psychotic disorders were estimated using the best-estimate consensus research diagnosis as the gold-standard, and compared in each racial/ethnic group. Finally, we conducted logistic regression analyses to determine whether relevant demographic and clinical variables significantly influenced the relationship between race/ethnicity and diagnostic discordance.

Results

The original study sample consisted of 267 ethnically diverse patients: 51% (n= 135) were Caucasian; 22% (n=59) African American; 20% (n=54) Hispanic; 6% (n=17) Asian/Pacific Islander; and 1% (n=2) had “other” ethnicities. The present study focused on the Caucasian, African American, and Hispanic patients (n=248). While there are seven diagnostic categories represented among the consensus research diagnoses, 3 were excluded in the present study due to too small numbers of cases represented in each category (i.e., schizophrenia-catatonic, schizophrenia-residual, and substance abuse). The remaining diagnostic groupings are shown in Table 2. The schizoaffective disorder category includes depressed and manic subtypes. The schizophrenia-other category includes both undifferentiated and disorganized subtypes. The most frequent diagnosis in the “other” psychotic disorder category was Psychosis NOS, but this category also includes a variety of other diagnoses including major depression with psychotic features, depressive disorder NOS, and personality disorder. The total number of participants represented within the 4 final diagnostic categories is presented in Table 2.

Table 2.

Comparison of the clinical and best-estimate research diagnoses in a sample of 238 inpatients with severe mental illness by racial/ethnic group

	Frequency N Clinical Diagnoses	Frequency N Research Diagnoses	Sensitivity	Specificity	Overall Kappa
Caucasian	129	129			.488
Schizoaffective	38	35	.743	.872
Schizophrenia-Paranoid	30	29	.517	.850
Schizophrenia-Other	50	55	.673	.824
Other	11	10	.500	.950
African American	57	57			.362
Schizoaffective	6	5	.800	.962
Schizophrenia-Paranoid	27	24	.625	.636
Schizophrenia-Other	17	23	.522	.853
Other	7	5	.400	.904
Hispanic	50	50			.599
Schizoaffective	17	17	.824	.909
Schizophrenia-Paranoid	17	17	.706	.849
Schizophrenia-Other	12	14	.643	.917
Other	4	2	.500	.938

Table 1 shows the demographic and clinical data for each racial/ethnic group. As indicated, Caucasians were significantly older and more educated than Hispanics and African Americans. The gender distribution of the patient sample was similar across racial/ethnic groups. Clinically, there was no significant racial/ethnic difference on mean age of onset of symptoms; however, Caucasians on average entered treatment at significantly younger ages than Hispanics. There was no significant difference in the global assessment of functioning (GAF) upon admission between the racial/ethnic groups; however, on average, Hispanics were discharged with a significantly higher GAF score than Caucasians.

Table 1.

Demographic and Clinical Characteristics of a Sample of 238 Caucasian, African American, and Hispanic Inpatients with Psychosis

	Caucasian n=130	African American n=58	Hispanic n=50	Analysis F, df, p-value
Mean (SD)
Mean Years of Education	13.6 (2.7)	12.3 (2.3)	12.1 (3.2)	7.13, 2, .001
Mean Age in years	34.5 (10.2)	29.0 (9.1)	31.9 (10.0)	6.40, 2, .002
Mean Age of Onset	20.7 (5.9)	21.3 (5.5)	21.7 (8.2)	.416, 2, .660
Mean Age at First Treatment	19.3 (7.4)	21.6 (7.3)	22.9 (7.8)	4.78, 2, .009
Mean GAF at Admission	37.3 (11.3)	35.1 (9.0)	36.1 (10.1)	.943, 2, .391
Mean GAF at Discharge	48.7 (13.7)	52.1 (11.6)	54.2 (12.0)	3.67, 2, .027
	Frequency			X2, df, p-value
Gender
Male (%)	83 (63.8)	35 (60.3)	28 (56.0)	.97, 2, .616

Table 2 shows the distribution of the four clinical hospital discharge and research diagnostic categories for Caucasian, African American, and Hispanic patients. The distribution was significantly different for both clinical hospital diagnoses (X²=17.18, df=6, p<.01) and best-estimate research diagnoses (X²=16.44, df=6, p<.05). With regard to clinical hospital diagnoses, while 34.0% (n=17) of Hispanics, and 28.5% (n=38) of Caucasians received a diagnosis of schizoaffective disorder, only 10.3% (n=6) of African Americans received this diagnosis, (X²=9.58, df=2, p<.01). The frequency of paranoid schizophrenia was significantly different by group (X²=9.76, df=2, p<.01). While only a quarter of Caucasians (23.8%: n=30) and a third of Hispanics (n=17) were considered to have paranoid schizophrenia, almost half of African Americans 46.6% (n=27) were diagnosed with paranoid schizophrenia. Likewise, with regard to best-estimate research diagnoses, the racial/ethnic groups differed in their frequency of schizoaffective disorder (X²=10.5, df=2, p<.01) and paranoid schizophrenia (X²=7.67, df=2, p<.05). (Table 2).

Overall agreement between the research and clinical diagnoses was moderate across the four diagnostic groups (i.e., schizoaffective disorder, schizophrenia-paranoid, schizophrenia-other, and “other” disorders), kappa= .49 (95% Confidence Interval: 0.41,-.57). Overall diagnostic agreement was higher among Hispanic patients than African American patients (X²=3.13, df=2, p=.06). While 24 African Americans (42.1%) had discordant clinical hospital and best-estimate research diagnoses, only 14 Hispanics (28%) had discrepant diagnoses. Forty-six (35.9%) Caucasians had discordant diagnoses.

Using the best-estimate research diagnosis as the gold standard, sensitivity and specificity analyses were conducted and used as indicators of under and over-diagnosis (See Table 1 for details). Percent of diagnostic agreement for the schizoaffective diagnosis and “other” diagnostic categories were similar across all three racial/ethnic groups. The majority of the discrepancies in clinical and research diagnoses for African American and Caucasian patients were due to the paranoid schizophrenia and schizophrenia-other categories. For example, out of 29 Caucasian patients who received a research diagnosis of paranoid schizophrenia, only 51.7% (n=15) also received this clinical diagnosis; indicating that Caucasians were under-diagnosed with paranoid schizophrenia. In contrast, out of 33 African American patients who did not receive a research diagnosis of paranoid schizophrenia, 36.4% (n=12) received this clinical diagnosis; African Americans were over-diagnosed with paranoid schizophrenia. In terms of “schizophrenia-other,” African Americans tended to be under-diagnosed; of 23 African American patients who received these research diagnoses, only 52.2% (n=12) also received the same clinical diagnosis.

We used logistic regression to compute the odds ratio of receiving a discordant diagnosis in adjusted models. Given the significantly different kappa values for Hispanic and African American groups, we focused on effects for these two groups. The odds ratio (OR) in the unadjusted model for African Americans vs. Hispanics was 1.87 (95% Confidence Interval: 0.83,-4.21), and in adjusted models, (i.e. controlling for mean age, level of education, and age at first treatment) was 2.18 (95% Confidence Interval: 0.93,- 5.14), indicating a marginally significant effect.

Discussion

The purpose of the present study was to compare diagnostic reliability in Caucasian, African American, and Hispanic patients with psychosis. We found diagnostic concordance was unexpectedly remarkable among Hispanic patients. Clinicians were most accurate in diagnosing psychotic conditions in this group, which had a higher accuracy than the groups of Caucasian and African American patients. The diagnostic agreement for the African American cases was below the accepted threshold of adequate reliability.²³ The kinds of diagnostic errors were related to the race and ethnicity of the patient. Clinicians over-diagnosed paranoid schizophrenia in African American cases, while they under-diagnosed paranoid schizophrenia in Caucasian cases. They also under-diagnosed undifferentiated or disorganized schizophrenia in African American patients, identifying such cases as having paranoid subtypes.

Our findings are consistent with previous studies that demonstrate a greater potential for clinician biases for African American patients.²⁴ It is not clear why clinicians diagnosed Hispanic patients more reliably given the greater potential for cultural and language differences between diagnosticians and Hispanic patients. One possibility is that the location of the research unit in a predominantly Latino community has sensitized the clinical staff to Latino diagnostic issues, which are emphasized in the clinical training. The Hispanic patients on this unit tended to be discharged with a higher GAF score than the African American and Caucasian patients, which was not evidenced upon admission. It is possible that the Hispanic patients improved the most because they had the correct diagnosis. Another possibility is the greater comfort of the Latino population disclosing mental health issues. There may be less cultural mistrust among the Hispanics in this patient population than among the African American patients. This might be especially true given that after 1998, Hispanic diagnosticians and staff were better represented on the unit.

Limitations

One major limitation is that the base rate of diagnoses was not equally prevalent across ethnic groups. Thus, the relatively high percent of diagnostic agreement among African Americans of the schizoaffective diagnosis could be misleading because very few African American patients were given that diagnosis by either the clinicians or the gold standard procedures. The small sample size also limits our ability to conduct time-sensitive analyses to assess the impact of the long event horizon under which the data was gathered. This finding needs to be replicated on a larger sample with more representation within each racial/ethnic group and among the diagnostic groupings. Notwithstanding, the present study highlights the importance of moving towards a truly multicultural paradigm in psychiatric research, and away from making just Black/Caucasian comparisons. The increased reliability found among Hispanic patients needs to be replicated on a larger more representative sample. It is unclear whether our findings would generalize to other hospital populations given our patient sample was drawn from a specialized research unit. The fact that African American clinicians are not adequately represented on either the clinical or research diagnostic teams may have played a role in the low diagnostic reliability of African American patients. Indeed, Trierweiler et al.^6–7, find that the race of the clinician influences how symptoms are attributed to different racial/ethnic groups. This should be empirically tested in future studies and compared to how well having Hispanic clinicians improves diagnostic reliability. After 1998, the SRU incorporated Hispanic Spanish-speaking diagnosticians and staff, thus it is possible that diverse representation among diagnosticians and staff improves diagnostic accuracy.

Conclusions and Implications

In sum, our results suggest the diagnostic process is particularly reliable for Hispanic psychotic patients in structured clinical settings. This reliability seems to be driven by the ability of clinicians to detect schizoaffective diagnoses consistently. For African American patients, the relatively lower diagnostic agreement seems to be driven by difficulties distinguishing paranoid schizophrenia from other psychotic disorders. Our results suggest that more empirical attention needs to be placed examining the diagnostic process including provider variables, as misclassification of disease by specific racial/ethnic groups can mask or bias relationships between race/ethnicity and psychiatric illness. A thorough analysis that identifies whether differential symptoms predict diagnostic discordance differently in Hispanic and African American patients will help us to understand the meaning behind the high and low reliability, respectively. Psychiatry, while a field of medicine, may be greatly impacted by cultural differences among patients. In our ever changing multicultural society, it will be very important to tease apart diagnostic inconsistencies that are more prevalent among groups of color.