Exploring Racial Disparities in The Brief Jail Mental Health Screen

Abstract

The authors analyzed validation data from the Brief Jail Mental Health Screen (BJMHS) to determine whether race predicted screening results and if such a prediction was driven by particular screen items. A total of 22,000 individuals entering five jails over two 8-month periods were screened. The authors constructed binary logistic regression models to assess the impact of race on screening positive and endorsing particular items. Blacks and Latinos had lower odds than Whites of screening positive. Blacks and Latinos had somewhat lower odds than Whites of endorsing two or more symptom items but had appreciably lower odds of endorsing items regarding prior mental health service utilization. Racial differences in BJMHS screening results likely reflect the reproduction of racial disparities rather than valid differences because results were driven by items reflecting known barriers in access to mental health services. Nonetheless, the instrument is likely to remain an improvement over typical jail screening procedures.

The United States surpasses all other countries in the proportion of individuals it incarcerates, and it is close behind China in the total number of people detained or imprisoned (Walmsley, 2009). More than 2.3 million individuals are incarcerated in state and federal prisons and local jails in the United States (Sabol, West, & Cooper, 2009), or 1 out of every 100 adults (Pew Center on the States, Public Safety Performance Project, 2008). These figures, although striking in their own right, belie gross racial and gender disparities in incarceration rates. Men make up roughly 90% of incarcerated populations (Sabol & Minton, 2008; Sabol, Minton, & Harrison, 2007; Sabol et al., 2009). One in 15 Black men older than 18 is incarcerated, compared to 1 in 106 White men in the same age group; for Latino men, the figure is 1 in 36 (Pew Center on the States, Public Safety Performance Project, 2008).¹ In jails (the focus of this report), there were roughly 13 million admissions nationally in 2007 (Sabol & Minton, 2008), and Blacks made up roughly 38% of the population, a share 3 times higher than community samples (Sabol et al., 2007; U.S. Census Bureau, 2008). Within this context, people with serious mental illnesses (SMI) are also overrepresented. Recent estimates suggest that 14.5% of men and 31.0% of women entering jails have SMI, or roughly 2 million individuals in 2007 (Steadman, Osher, Robbins, Case, & Samuels, 2009).

A virtual policy consensus exists around the need for jail diversion programs (Council of State Governments, 2002), which, when appropriate, link people with SMI to community-based treatment in lieu of incarceration. Such programs have proliferated over the past decade (Steadman & Naples, 2005), and there has been an attendant focus on accurately identifying people with SMI during jail intake procedures and developing effective mental health screening instruments toward that end—for example, the Brief Jail Mental Health Screen (Steadman, Scott, Osher, Agnes, & Robbins, 2005) and the Correctional Mental Health Screens for men and women (Ford, Trestman, Wiesbrock, & Zhang, 2009; National Institute of Justice, 2007). The effectiveness of these recently validated instruments has been judged not only on statistical performance measures but also in the unique policy, procedural, and demographic (viz., gender) context of jail settings. Less attention has been paid, however, to race and racial disparities in the instruments used to identify people for further mental health assessment in jails—and ultimately treatment and jail diversion resources. Our use of the term disparity is informed by the National Institute of Medicine's typology and refers to differences that do not result from actual variations in clinical needs or preferences; rather, disparities reflect differences created by structural barriers (e.g., poverty, low access to health insurance) and individual-level factors (e.g., biases, prejudices, stereotyping, and clinical uncertainty) that result in differential treatment of racial minority groups (Smedley, Stith, & Nelson, 2002; Swanson et al., 2009).

There are a number of reasons to be concerned about racial disparities with regard to mental health screening. According to the Surgeon General's report “Mental Health: Culture, Race and Ethnicity” (U.S. Department of Health and Human Services, 2001), “Cultural factors affect how individuals define, evaluate, seek help for, and present their health problems to family members, friends, and service providers” (p. 18). As such, measurement error can occur in a number of ways. Ramirez, Ford, Stewart, and Teresi (2005) highlight two such sources of error: cross-cultural differences in interpreting both the meaning of concepts related to mental illnesses and the items used to measure those constructs. They argue that it is unrealistic to assume that these concepts and experiences can be measured the same way among diverse groups of people. The most obvious example is when concepts lack direct equivalents in other languages (Snowden, 2003). Furthermore, members of different cultural and ethnic groups may express symptoms of mental disorders differently, and these differences may be prone to misinterpretation by clinicians from different backgrounds (Peters, Bartoi, & Sherman, 2008).

Race may also affect the way that screening instruments are administered. Racial disparities may be produced during the screening process, when unwarranted judgments or erroneous assumptions are made about people on the basis of their race or ethnicity. For example, screen administrators may ignore familiar symptoms of mental illness (Lopez, 1989) or prejudicially believe that certain groups are resistant to treatment (Shim, Compton, Rust, Druss, & Kaslow, 2009). Furthermore, these assumptions and misinterpretations may explain why some individuals are less likely to disclose symptoms to service providers, especially if they have experienced prior discrimination in the health system (Peters et al., 2008). These sorts of effects can operate even when minority group members have favorable attitudes toward mental health treatment (Shim et al., 2009).

Further refining the initial validation of the Brief Jail Mental Health Screen (BJMHS; Steadman, Scott, et al., 2005) and subsequent revalidation for women (Steadman, Robbins, Islam, & Osher, 2007), the primary aim of this report is to assess the performance of the BJMHS across race categories. Building on a number of the authors' prior works developing and validating the BJMHS, and given its widespread dissemination to jails across the country, the authors conducted secondary analyses of data from prior validation studies to determine (a) whether race predicts screening result and, if so, (b) whether particular items in the instrument are generating such a difference.

Method

The present analysis relies on the design used in the two prior studies of the BJMHS (Steadman et al., 2007; Steadman, Scott, et al., 2005). The original design represented a cross-sectional survey of individuals in five jails in two states over two 8-month phases. The sampling frame was all adults entering five jails: Montgomery County, Maryland; Prince George's County, Maryland; Rensselaer County, New York; Albany County, New York; and Monroe County, New York. The Monroe County Jail was chosen to replace the Albany County Jail in the gender revalidation study, as the latter was unable to participate. For the aforementioned validation (Phase 1) and subsequent revalidation of the BJMHS for women (Phase 2), all individuals entering the jails were screened for SMI with the BJMHS on intake, and within 72 hours a subsample was selected to receive a validation interview using the Structured Clinical Interview for DSM-IV Diagnoses (SCID; First, Spitzer, Gibbon, & Williams, 2002). The subset was identified through stratified random and systematic sampling to (a) ensure an adequate number of positive screens and (b) ensure an adequate sample of women to enable separate gender analyses, which was an aim of the revalidation study. Researchers compiled lists of detainees who met these criteria, and clinical interviewers, blind to screening results, approached detainees on the lists to participate in SCID validation interviews.

Participants

Participation in both prior studies was voluntary. Informed consent forms were obtained for all SCID interview participants. Participants were informed that their stay in jail would not be affected. All human participant procedures were approved by the institutional review boards associated with each data collection site.

Both samples were predominantly Black and male, which is consistent with national jail figures. The Phase 1 sample included 11,438 individuals (9,926 males, 1,478 females, and 34 missing), and the Phase 2 sample included 10,562 individuals (8,864 males, 1,658 females, and 40 missing). In Phase 1, the sample was 25.7% White, 57.6% Black, 10.0% Latino, 2.5% “other,” and 4.1% “no race specified.” The Phase 2 sample was 18.6% White, 55.5% Black, 12.5% Latino, 3.3% “other,” and 10.1% “no race specified.” The mean age of individuals was 31.6 in Phase 1 and 31.9 in Phase 2. In Phase 1, of the 11,168 individuals with valid screening data, 1,194 screened positive, or 10.7%. In Phase 2, of the 10,277 individuals with valid screening data, 1,663 screened positive based on the original screen configuration, or 16.2%.

Measure

The BJMHS is an eight-item questionnaire designed specifically for use in jails. Details on the length of time to administer, sensitivity, specificity, false positive rate, and false negative rate for men and women can be found in the prior reports cited above. The instrument includes six symptom items and two service utilization items (“Are you currently taking any medication prescribed for you by a physician for any emotional or mental health problems?” and “Have you ever been in a hospital for emotional or mental health problems?”). Endorsement of any two symptom items or any one service utilization item constitutes a positive screening result.

Procedure

We examined data from the initial validation and subsequent revalidation for women. In Phase 2, items were added to the screening instrument, and no data on criminal charges were recorded. To assess the performance of the screening instrument across race categories, we constructed binary logistic regression models using screening result as the outcome variable and race as the primary predictor of interest, controlling for clinically relevant confounders including but not limited to age and gender (Nagin & Land, 1993; U.S. Department of Health and Human Services, 1999) and criminal charge (James & Glaze, 2006). We also controlled for jail site to rule out effects of jail location or screen administration that may have occurred across the five jails. We used forward selection to determine whether gender modified the effect of race on screening outcome but found no such interaction and thus did not include it in our final model. To explore whether race predicted responses to certain items in the instrument, we constructed analogous models with each item as the outcome variable. All models were constructed with forward-style selection procedures. Cases with no recorded race were excluded, and cases with missing values on potential confounding variables were excluded listwise (all variables were missing less than 5.0% of values, except for criminal charge in Phase 1, which was missing 8.4% of values, and age in Phase 2, which was missing 6.3% of values). All analyses were conducted in SPSS Version 16 and/or SAS Version 9.2.

Results: Racial Differences in Screening Results

Race was a significant predictor of screening result in both phases: χ²_{Phase 1}(3) = 127.562, p < .001; χ²_{Phase 2}(3) = 60.705, p < .001. As shown in Table 1, Blacks had less than half the odds as Whites of screening positive on the BJMHS in Phase 1 and slightly more than half the odds as Whites in Phase 2. Latinos had roughly one third the odds as Whites of screening positive on the BJMHS in both phases.

Table 1. Phase 1 and Phase 2 Binary Logistic Regression Models Predicting Likelihood of Screening Positive on the Brief Jail Mental Health Screen.

	Phase 1				Phase 2
	Waldb	p	OR	95% CI	Waldb	p	OR	95% CI
Race
Whitea			1				1
Black	105.641	.000	0.440	0.377–0.515	32.093	.000	0.640	0.548–0.747
Latino	54.859	.000	0.317	0.234–0.430	49.036	.000	0.395	0.304–0.512
Other	9.695	.002	0.473	0.295–0.758	1.056	.304	0.852	0.627–1.157
Jail
Montgomerya			1				1
Prince	27.363	.000	1.587	1.335–1.886	195.939	.000	2.939	2.527–3.417
Rensselaer	19.235	.000	0.583	0.458–0.742	9.137	.003	1.477	1.147–1.901
Albany/Monroe	13.177	.000	0.713	0.594–0.856	112.266	.000	2.840	2.341–3.444
Age
Younger than 25a			1
25–34	2.351	.125	1.156	0.960–1.392	5.117	.024	1.202	1.025–1.410
35–44	32.077	.000	1.661	1.393–1.979	17.563	.000	1.410	1.201–1.656
45–54	52.899	.000	2.221	1.792–2.755	35.549	.000	1.791	1.479–2.169
55 or older	16.617	.000	2.269	1.530–3.364	24.485	.000	2.434	1.711–3.463
Female	145.421	.000	2.628	2.246–3.075	233.186	.000	2.934	2.555–3.369
High or intox. during screen	17.161	.000	2.760	1.707–4.461	3.349	.067	1.880	0.956–3.698
Charge					—	—	—	—
Othera			1		—	—	—	—
Violent	1.233	.267	1.269	0.833–1.932	—	—	—	—
Property	0.929	.335	1.223	0.812–1.844	—	—	—	—
Sex	16.063	.000	3.134	1.793–5.478	—	—	—	—
DUI	4.031	.045	0.590	0.353–0.988	—	—	—	—
Drug	0.240	.624	0.896	0.578–1.390	—	—	—	—
Public	1.438	.230	1.319	0.839–2.073	—	—	—	—
Technical	0.682	.409	1.200	0.779–1.848	—	—	—	—
Traffic	2.178	.140	0.685	0.415–1.132	—	—	—	—
Financial	2.903	.088	1.762	0.918–3.383	—	—	—	—
Weapon or nonviolent	0.045	.832	1.070	0.575–1.990	—	—	—	—
Constant	93.880	.000	0.118		586.470	.000	0.091

Note. N_{Phase 1} = 9,821; N_{Phase 2} = 8,727. Wald = Wald χ²; p = p value; OR = odds ratio; CI = confidence interval.

^aReference category.

^bDegrees of freedom = 1.

Item analysis

Given that Blacks and Latinos had lower odds than Whites of screening positive on the BJMHS, we next examined the individual items in the instrument to determine whether race predicted responses to certain questions. Table 2 presents odds Prins et al. / RACIAL DISPARITIES IN BJMHS 639 ratios for screen items that showed any significant racial differences. Race significantly predicted endorsement of service utilization items in both phases. In Phase 1, Blacks and Latinos had less than half the odds as Whites of having ever been hospitalized or currently taking medication for mental illnesses. In Phase 2, Blacks had roughly two thirds the odds as Whites and Latinos had roughly one third the odds as Whites of endorsing these service utilization items. Results were mixed for symptom items. Latinos had lower odds than Whites of having lost or gained weight in Phases 1 and 2. In Phase 1, Blacks and Latinos had somewhat lower odds than Whites of having felt useless or sinful. In Phase 2, Blacks and Latinos had appreciably lower odds than Whites of having felt useless of sinful.

Table 2. Odds Ratios for Endorsing Brief Jail Mental Health Screen Items by Race.

	Q3. Lost or gained weight		Q6. Felt useless or sinful		Q7. Taking medication		Q8. Prior hospitalization
Race	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI
Phase 1
White	1		1		1		1
Black	0.733	0.495–1.086	0.689*	0.481–0.986	0.394**	0.326–0.476	0.483**	0.401–0.579
Latino	0.420*	0.180–0.980	0.414*	0.192–0.891	0.376**	0.259–0.547	0.385**	0.268–0.554
Other	1.586	0.693–3.629	1.15	0.481–2.751	0.344**	0.178–0.664	0.460*	0.257–0.824
Phase 2
White	1		1		1		1
Black	0.822	0.665–1.017	0.649**	0.526–0.800	0.632**	0.524–0.763	0.614**	0.508–0.744
Latino	0.631*	0.440–0.905	0.468**	0.323–0.678	0.378**	0.266–2.536	0.357**	0.249–0.512
Other	1.105	0.753–1.622	0.773	0.512–1.169	0.778	0.525–1.154	0.731	0.492–1.088

Note. OR = odds ratio; CI = confidence interval. Only items with any significant racial differences are displayed.

Q3 = “Have you currently lost or gained as much as two pounds a week for several weeks without even trying?” Q6 = “Have there currently been a few weeks when you felt like you were useless or sinful?” Q7 = “Are you currently taking any medication prescribed for you by a physician for any emotional or mental health problems?” Q8 = “Have you ever been in a hospital for emotional or mental health problems?”

^*p < .05.

^**p < .001.

Symptom versus service items

We next explored the first six symptom items of the BJMHS in greater detail, in light of the disjunction between prior service utilization and symptom endorsement. After confirming that the BJMHS had reasonable internal consistency across race categories, we computed a new scale variable with the six symptom items, wherein individuals who answered yes to two or more items (a positive screen) were coded as 1 and individuals who answered yes to one or no items were coded as 0. Race was not a significant predictor of answering yes to two or more symptom items in our Phase 1 model at α = .01, Wald χ²(3) = 9.328, p = .025. Despite the apparent trend toward significance in Phase 1, none of the individual race category odds ratios had p values less than .05. For Phase 2, race was a significant predictor of answering yes to two or more symptom items, Wald χ²(3) = 23.207, p < .001. Blacks had lower odds than Whites, OR = 0.758 (95% CI = 0.634–0.907) p = .002, of endorsing two or more symptom items, as did Latinos, OR = 0.535 (95% CI = 0.394–0.727) p = .001. These findings suggest either that the symptom items that differ across racial categories are not sensitive predictors of SMI in Blacks and Latinos or that the BJMHS is detecting small but valid differences in symptom occurrence.

Discussion

Our preliminary finding—that the BJMHS's uneven performance across racial groups appears to be driven primarily by disparities in prior service utilization and not clear differences in symptom occurrence—warrants serious consideration and further examination. In particular, future research should test the strength of service utilization items in predicting SMI for Blacks and Latinos who have accessed such services and explore whether including more sensitive symptom items may allow for the removal of one or both of the service utilization items from the BJMHS entirely.

The strengths of this analysis include large sample sizes, virtually complete coverage of the sampling frame, and replication across multiple jail sites and time periods. Nevertheless, our findings are constrained by a number of limitations. Because the original research focused on an initial validation of the BJMHS in Phase 1 and a more specific validation for women in Phase 2, data were not collected on a number of variables that may have strengthened our findings and enabled more detailed analyses. In particular, no data were collected on detainees' socioeconomic status, insurance status, or prior outpatient treatment, which may have helped explain the role of access to, and utilization of, mental health services in predicting screening outcome. In addition, data were not collected on the race of correctional intake officers who administered the BJMHS, which might have enabled examination of any racial bias in the implementation of screening procedures.

The most significant limitation of our secondary analysis is that the data set did not allow us to directly test the cross-racial validity of the BJMHS by comparing race, screening result, and actual diagnoses. In conjunction with the primary research aims, individuals who screened positive and women were oversampled for SCID validation interviews; the subsample was not representative of the racial distribution of all detainees or detainees with SMI.

It is possible that the BJMHS is capturing valid racial differences in lifetime occurrence of SMI. If Whites are more likely to have a history of SMI and receive treatment and that treatment is presumably effective, then this might explain both Whites' more frequent endorsement of service utilization items and the minimal differences in symptoms among Whites, Blacks, and Latinos. Although our data set, as noted, does not allow us to test this hypothesis directly, there are three reasons why we believe our finding is still relevant: First, using lifetime symptom occurrence in screening instruments may overestimate the immediate need for mental health treatment in resource-poor jail environments (Steadman, Scott, et al., 2005). Even if the alternative hypothesis is granted, racial disparities in access to mental health services—qua disparities—exist independently of any valid racial differences in the lifetime or current prevalence of mental illnesses, as will be discussed in detail below. Second, major psychiatric epidemiological surveys have found similar rates of mental illness among Blacks, Whites, and Latinos, although there are documented disparities in the accuracy of diagnoses, rates of specific disorders, and rates of disorders among Latinos born in the United States versus born in Latin America (U.S. Department of Health and Human Services, 2001). Third, the BJMHS contains only six symptom items and thus does not represent an exhaustive symptom inventory from which broad conclusions should be drawn about the racial distribution of symptoms or symptom clusters. Examined alongside the finding that Whites had consistently and appreciably higher odds of having ever been hospitalized or currently taking medications for mental illnesses, the absence of major racial differences in symptom endorsement suggests that the overall performance of the BJMHS constitutes a disparity that may be driven by a mechanism other than the cross-racial validity of symptom items, namely, items measuring prior mental health service utilization, an imperfect proxy for current mental illness for which there are widely documented racial disparities (Alegría et al., 2002; Cook, McGuire, & Miranda, 2007; Kessler et al., 2005; Mojtabai, 2005; Olfson, 2009; Shim et al., 2009).

Racial Disparities In Service Access and Utilization

Endorsing either of the two mental health service utilization items on the BJMHS results in an “automatic positive,” whereas two symptom items are required to produce a positive screen. This places considerable weight on prior service utilization in predicting the likelihood of SMI, a potentially dubious calculus given the large body of literature on racial disparities in service access and utilization. Blacks and Latinos are half as likely as Whites to receive any treatment for any mental disorders experienced in a prior 12-month period (Kessler et al., 2005). They are also less likely than Whites to have any contact with a mental health professional, are less able to afford mental health care, and, Latinos only, are less able to afford medications (Mojtabai, 2005). When racial minority groups do access mental health treatment, it is more likely to be of poor quality and from a primary care physician rather than a psychiatrist or mental health professional (Alegría et al., 2002; Shim et al., 2009). Whites are twice as likely as Blacks and Latinos to be treated with antidepressant medications, for a variety of ailments (Olfson, 2009). There is also evidence of widening disparities in racial minority group members' expenditures on mental health services, with Whites spending roughly 60% more than Blacks and 117% more than Latinos in 2003–2004 (Cook et al., 2007). Primm, Osher, and Gomez (2005) suggest that a lack of access to mental health services actually contributes to the high numbers of Blacks and Latinos with mental illnesses in the criminal justice system.

Some of the issues around treatment-seeking behavior and screening bias mentioned above may also be operating in these findings. In terms of treatment-seeking behavior, Shim et al. (2009) suggest that Blacks and Latinos may be less likely to utilize mental health services because of stigmatizing beliefs about mental illnesses that originate from families and social or religious networks, “double stigma” associated with being in a minority group that experiences racial discrimination in addition to having a mental illness, or, as discussed above, cultural and linguistic barriers that can undermine the therapeutic alliance or result in underreferral to treatment. Because of the gatekeeping role that screening and referral processes often play in accessing treatment and services, these phenomena can have significant material consequences (Snowden, 2003).

The Role of Jail Screening Instruments in Reproducing Racial Disparities

Given that two disproportionately weighted items in the BJMHS represent multifaceted racial barriers in service access and utilization, their impact on screening results may be to reproduce prior disparities in the mental health system. In other words, the BJMHS's service utilization items may create additional disparities in the present by bringing into the foreground those that occurred in the past. New disparities are likely to include limited access to jail mental health treatment resources and exclusion from diversion programs that target detainees with SMI for linkage to community-based treatment. These, in turn, may lead to more severe impairment and deeper penetration into the criminal justice system, respectively. Indeed, there is evidence to suggest that participants in jail diversion programs are disproportionately White (Case, Steadman, Dupuis, & Morris, 2009; Naples, Morris, & Steadman, 2007; Steadman, Redlich, Griffin, Petrila, & Monahan, 2005), and inadequate identification of Blacks and Latinos during the screening process may be a contributing factor. That such disparities might emerge from an instrument with content that is prima facie benign demonstrates how even the most banal questions can become operative in the reproduction of institutional and structural inequality.

Broadly speaking, this phenomenon may be more widespread than the above analysis initially suggests. In our experience providing technical assistance to jails across the country, we have found that most have not implemented validated screening instruments and instead use homegrown tools that tend to rely heavily on prior service utilization as a proxy for current mental illness.

Conclusion

Despite troublesome—albeit inconclusive—evidence for racial differences in the performance of the BJMHS, it would be imprudent for jail staff to discontinue using the BJMHS altogether or modify it of their own accord, as it is still likely to represent an important improvement over typical jail screening procedures. Unfortunately, the answer is not as simple as removing the service-related items from the BJMHS, as this may trade a reduction in racial disparities for a reduction in sensitivity. Although a comparable instrument such as the Correctional Mental Health Screen has been tested for cross-racial symptom validity (Ford, Trestman, Wiesbrock, & Zhang, 2007), it too contains an item on prior service utilization that was not examined in a manner similar to the analysis above. We know of no other comparable jail mental health screening instruments that have ruled out racial disparities related to prior service utilization. Jail staff should acknowledge that Blacks and Latinos with SMI may be underreferred by the BJMHS and thus remain undetected in the jail's general population. As such, jail staff should be receptive to conducting subsequent mental health assessments of members of these groups when appropriate, and further refinement of the BJMHS should be undertaken to determine its cross-racial validity.

Biographies

Seth J. Prins, MPH, is a predoctoral fellow in the Psychiatric Epidemiology Training Program, Department of Epidemiology, Columbia University, Mailman School of Public Health. He was a senior policy analyst on mental health and criminal justice projects at the Council of State Governments Justice Center and received an MPH in sociomedical sciences from Columbia University, Mailman School of Public Health.

Fred C. Osher, MD, is the director of health systems and health services policy at the Council of State Governments Justice Center and oversees the health components of the Justice Center's initiatives. He is a community psychiatrist with clinical, research, and policy interests in effective services for persons with serious mental illnesses and co-occurring substance use disorders.

Henry J. Steadman, PhD, has been president of Policy Research Associates, Inc. since he founded it in 1987. His work has resulted in eight books, more than 130 journal articles in a wide range of professional journals, 20 chapters, and numerous reports. He received his BA and MA in sociology from Boston College and his PhD in sociology from the University of North Carolina at Chapel Hill.

Pamela Clark Robbins is vice president for operations and has directed research projects within Policy Research Associates, Inc. since 1987. She is an expert in the areas of research design, project management, data collection, and analysis. Her primary research interests lie in the areas of mental illness and violence.

Brian Case, MA, is a project associate at Policy Research Associates, Inc. He received his Master's in criminal justice from the School of Criminal Justice at the State University of New York in Albany.