Results of the Brief Jail Mental Health Screen Across Repeated Jail Bookings

Abstract

Objectives:

The Brief Jail Mental Health Screen (BJMHS) is widely used at intake in county jails to identify detainees who may have serious mental illness and who should be referred for further mental health evaluation. The BJMHS may be administered multiple times across repeated jail bookings; however, the extent to which results may change over time is unclear. To that end, we examined the odds of screening positive on the BJMHS across repeated jail bookings.

Methods:

Data were drawn from the administrative and medical records of a large, urban county jail that used the BJMHS at jail booking. The study sample comprised BJMHS results for the 12,531 jail detainees who were booked at least twice during the 3.5-year period (total N=41,965 bookings). Multilevel logistic modeling was used to examine changes over time overall and within the four decision rules (current psychiatric medication; prior hospitalization; two or more current symptoms; referral for any other reason).

Results:

Results show that the odds of a positive screen overall increase with each jail booking, as do the odds of referral for any other reason. In contrast, the odds of screening positive for two or more current symptoms and prior hospitalization decrease. There was no change in the odds of screening positive for current psychiatric medication across bookings.

Conclusion:

Findings show that BJMHS results change across bookings. Further research is needed to determine whether changes reflect true changes in mental health status, issues with fidelity, the repeated nature of the screening process, or other factors.

Introduction

The prevalence of serious mental illnesses, including mood disorders, schizophrenia, and other psychotic disorders, is three to four times greater among jail detainees than in the general population (1), and estimates suggest that over two million people with serious mental illness are booked into U.S. jails yearly (2). In an effort to improve clinical outcomes, and, ultimately, reduce overrepresentation of serious mental illness in jails, the American Psychiatric Association (APA) and the National Commission on Correctional Health Care (NCCHC) issued standards for mental health screening, referral, and evaluation for inmates in jails and prisons (3, 4). Specifically, the NCCHC requires a mental health screen within 14 days of booking, followed by an in-depth evaluation if the screen is positive (5). Accordingly, mental health screening has become commonplace in county jails across the U.S. (6) with the goal of identifying detainees at intake who may have serious mental illness and referring them for further, in-depth mental health evaluation. In this way, mental health screening may facilitate diagnosis and aid jail staff in making decisions regarding classification, treatment, and management (7).

Given the high volume of admissions, relatively low levels of staffing, and typically short periods of stay in U.S. county jails, mental health screening tools must be very short and easily administered. The Brief Jail Mental Health Screen (BJMHS) is one such instrument. Administered verbally by intake officers in about five minutes or less and requiring minimal training, the BJMHS queries the presence of current psychiatric symptoms, current use of psychiatric medication, and prior psychiatric hospitalization via eight yes/no questions. Detainees are referred for further mental health evaluation if they meet one or more of four decision rules (8). The BJMHS has been recommended by SAMHSA for use as a jail mental health screening tool for nearly 20 years (9; 10) and is used in jails across the US (6).

Despite its widespread use, there has been relatively limited empirical investigation of the BJMHS. Of the handful of studies that have investigated the tool, the majority focus on its performance compared to “gold standard” mental health assessment tools, such as the Structured Clinical Interview for DSM-IV (SCID-IV) (8, 11) or the Mini International Neuropsychiatric Interview assessments (12), or compared to other jail mental health screening tools, such as the Jail Screening Assessment Tool (13, 14). Taken together, evidence regarding its performance is mixed; some studies show good concordance with the criterion (11) and others, less so (12, 13, 14). Beyond criterion validity, there remain questions regarding the functioning of the tool in practice. In particular, we are not aware of research that has examined BJMHS results over time, even though detainees may be assessed multiple times across repeated jail bookings. This is the focus of the current investigation.

Mental health screening results may change over time for reasons related to changes in mental health status. For example, following release from jail, individuals with serious mental illness may face significant challenges, including, stress (15), housing instability, negative peer influences (16), employment difficulties (17), and limited access to treatment (18), that contribute to worsened illness trajectories. However, for many with serious mental illness, jail booking may represent the first point at which their symptoms are identified, and they are referred to treatment (19). As a result, they may experience improved illness trajectories across repeated bookings (20–23). Alternatively, BJMHS results may change over time for reasons other than true changes in illness trajectories. For instance, impaired insight or cognitive functioning could impact self-report at one screening but not at another (24). Further, because the BJMHS is administered as a face-to-face interview, social desirability, impression management, and rapport also may impact results (25; 26; 27). A detainee may be more comfortable with certain staff or may become more comfortable with the process (and more likely to disclose information) over time. Staff may become familiar with an individual detainee over repeated admissions (28) and refer them for further evaluation, even in the absence of current symptoms. Or, staff beliefs, attitudes, and experiences may influence the way in which they approach detainees (29), which may affect detainee self-report and staff interpretation of that self-report. Finally, there are no training guidelines for the BJMHS (8), which may impact staff knowledge or understanding of the decision rules and, consequently, the fidelity with which the BJMHS is administered.

The Present Study

While there are many reasons mental health screening results may change across repeated bookings, empirical evidence is lacking with respect to the BJMHS or any jail mental health screening tool, for that matter. Yet, mental health screening results have important implications for justice-involved persons with serious mental illness. As such, understanding factors that may affect these screening results, including repeated administration, is critical. Our goal was to determine the odds of screening positive on the BJMHS across multiple jail bookings, after accounting for relevant sociodemographic (i.e., age, sex, race, housing status) and other clinical (i.e., substance use) characteristics that may impact results. We used multilevel logistic modeling to examine changes in BJMHS results overall and for each of the four decision rules across repeated bookings in a population of detainees admitted to a large, metropolitan jail.

Methods

All study procedures were approved by the university’s Institutional Review Board. The sample comprised 12,531 detainees across 41,965 bookings to a large, urban county jail in the southeastern U.S. This sample was drawn from the population of detainees (N=37,988) booked into the study jail between January 1, 2013 and June 30, 2016. Only those who were readmitted to the study jail at least one time (N=12,531) were included in this study, since our interest was in examining change in screening results over time. Data were drawn from administrative and medical records.

The BJMHS was administered to all detainees in the context of routine practice by jail nurses at intake. As briefly described in the introduction, the BJMHS consists of eight, self-report dichotomous items (no=0, yes=1). These eight items are divided into two sections. The first section includes six items that query the presence of psychiatric symptoms in the past six months. The second section queries treatment history, including one item that addresses current use of psychotropic medication (item seven) and one item that addresses lifetime psychiatric hospitalization (item eight) (8). A positive screen occurs when a detainee meets at least one of the following decision rules: 1) yes to at least two of items one through six; 2) yes to item seven; 3) yes to item eight; or 4) if the screener feels the detainee should be referred for any other reason (even in the absence of any yeses). We examined odds of meeting each of these four decision rules, as well as an overall positive screen (i.e., meeting at least one of the decision rules), as outcome variables.

An optional third section includes the screener’s comments or impressions regarding several extenuating circumstances (e.g., language barrier, inmate was under the influence of drugs/alcohol). This section does not bear on screening results, and thus, was not included as an outcome variable.

The predictor variable, booking number during the study period, allowed us to ascertain change in BJMHS across repeated jail bookings. Covariates included: age, sex, race, homeless status, and substance use problems. Age (in years at the time of each booking), sex (0=female, 1=male), and race (0=non-Caucasian, 1=Caucasian) were obtained from the administrative records. Homelessness (0=no, 1=yes) was self-reported at intake. Substance use problems (0=no, 1=yes) was a composite measure comprised of the optional third section of the BJMHS and four items from the jail intake medical screen. A person was coded ‘yes’ for substance use problems if they met any of the following criteria: under the influence, signs of withdrawal, self-reported drug use, or self-reported problematic alcohol use (30).

We performed several multilevel logistic regression models using SAS 9.4 PROC GLIMMIX, with bookings (Level 1) nested within individuals (Level 2). This technique allowed for examination of the rate of change in outcomes, incorporated all available data from each person, and controlled for the differences in timing of each repeated measure (i.e., time lapse between bookings). We ran one model examining odds of an overall positive screen followed by four models in which we examined each of the BJMHS decision rules separately. These five models were run once without covariates and once with covariates.

Results

Sample Characteristics

The average number of bookings per person in our analytic sample was 3.35±2.53 (Range=2–65) during the study period, the average length of stay was 21.42±50.59 days (Range=0–1049), and the average time between bookings was 846.15±288.37 days (Range=0–1276). The sample was 34.8% (N=4362) Caucasian and 65.2% (N=8162) Non-Caucasian (65.1% (N=8151) African American, 0.1% (N=11) Asian). Average age at first booking during the study period was 31.50±11.57 years (Range=16–79). The vast majority (81.7%, N=10232) of the sample were men. About 8.0% (N=1002) of detainees reported homelessness and 40.0% (N=4994) screened positive for problematic substance use. At their initial booking, 62.9% (N=6600) of the sample were charged with more serious offenses (typically felonies punishable by imprisonment for a year or more; e.g., assault, burglary, sexual offenses, etc.), while 37.1% (N=3886) were charged with less serious offenses (typically misdemeanors punishable by imprisonment for less than a year; e.g., stolen property, disorderly conduct, etc.). The most common charges at initial booking included assault (21.2%, N=2654), drug/narcotic (15.3%, N=1916), and larceny/theft (11.5%, N=1440).

Descriptive Statistics

In total, 23.3% (N=2900) of detainees screened positive overall on the BJMHS at initial booking during the study period. Rates of positive screens for each decision rule at initial booking ranged from 2.7% (N=340) for current psychiatric symptoms to 20.6% (N=2579) for referral for any other reason (i.e., at the assessor’s discretion). Visual exploration of the descriptive statistics suggested the frequency of positive screens may be changing over time; however, rates and directions of change seem to differ across items and decision rules (see Figure 1). A breakdown of positive responses to individual BJMHS items and to BJMHS decision rules can be seen in Table 1.

Figure 1.

Percent of Jail Detainees who Screened Positive Overall and by Decision Rule across Bookings

Table 1.

Affirmative Responses to Brief Jail Mental Health Screen Items and Decision Rules Overall and Across First Three Bookings

Construct	Brief Jail Mental Health Screen Item	All Bookings N=41965		Booking Number
				1 N=12531		2 N=12531		3 N=6199
		n	%	n	%	n	%	n	%
Thought control	1. Do you currently believe that someone can control your mind by putting thoughts into your head or taking thoughts out of your head?	543	1.3	194	1.6	166	1.3	63	1.0
Paranoia	2. Do you currently feel that other people know your thoughts and can read you mind?	484	1.2	167	1.3	146	1.2	64	1.0
Weight loss/gain	3. Have you currently lost or gained as much as two pounds a week for several weeks without even trying?	579	1.4	202	1.6	162	1.6	82	1.3
Activity level	4. Have you or your family or friends noticed that you are currently much more active than you usually are?	531	1.3	178	1.4	151	1.4	74	1.2
Lethargy	5. Do you currently feel like you have to talk or move more slowly than you usually do?	492	1.2	184	1.5	135	1.5	65	1.1
Feeling useless/sinful	6. Have there currently been a few weeks when you felt like you were useless or sinful?	1941	4.7	701	5.7	578	4.6	278	4.5
Current medication	7. Are you currently taking any medication prescribed for you by a physician for any emotional or mental health problems?	5284	12.7	1446	11.6	1608	12.9	776	12.6
Lifetime hospitalization	8. Have you ever been in a hospital for emotional or mental health problems?	3903	9.4	1198	9.6	1137	9.1	594	9.7
Decision Rules	Criteria
Current psychiatric medication	Yes to item 7.	5284	12.7	1446	11.6	1608	12.9	776	12.6
Lifetime hospitalization	Yes to item 8.	3903	9.4	1198	9.6	1137	9.1	594	9.6
Current symptoms	Yes to 2 of items 1 through 6.	969	2.3	340	2.7	275	2.2	132	2.1
Referral for any other reason	Referral for reasons other than above criteria.	3048	7.3	830	6.6	891	7.1	428	6.9

Notes. % = valid percentage of affirmative responses. n = number of affirmative responses.

Multilevel Logistic Models

Table 2 presents the estimates for each of the five multilevel logistic models. Results of the first multilevel logistic model confirmed that the odds of a positive screen overall (i.e., meeting at least one of the decision rules) increased with each additional booking into jail. Specifically, each additional booking was associated with a 2% increase in the odds of a positive screen (OR=1.02, 95%CI=1.01–1.04). Results of the model including covariates showed the same the pattern. (Full results available in online supplement.)

Table 2.

Multilevel Logistic Models of Booking Number Predicting Brief Jail Mental Health Screening Results

Model	Model Estimates
	−2LL	Estimate	SE	t	df	OR	95% CI
1. Met criteria for a positive screen overall	194522.6	0.025	0.006	4.34	41686	1.02	1.01–1.04
2. Currently taking psychiatric medication	209701.6	0.010	0.007	1.49	41833	1.01	1.00–1.02
3. Experienced any prior hospitalization	216054.0	−0.037	0.008	−4.48	41807	0.96	0.95–0.98
4. 2 or more current psychiatric symptoms	257560.7	−0.049	0.014	−3.55	33298	0.95	0.93–0.98
5. Referral for any other reason	220830.3	0.030	0.006	5.12	29143	1.03	1.02–1.04

Notes. −2LL = negative two log likelihood; SE = standard error for the estimate; t = t value; df = degrees of freedom; OR = odds ratio; 95% CI = 95% confidence interval for the odds ratio.

Results of the multilevel logistic models examining each of the decision rules showed that the odds changed across repeated bookings for three of the four rules (see Table 2). The odds of screening positive for prior hospitalization and for reporting two or more current psychiatric symptoms decreased over time. With each additional booking, the odds of screening positive for prior hospitalization decreased by about 4% (OR=0.96, 95%CI=0.95–0.98) and the odds of screening positive for current symptoms decreased by about 5% (OR=0.95, 95%CI=0.93–0.98). In contrast, detainees were more likely to be referred for any other reason (i.e., at the assessor’s discretion) with each additional booking into jail. For this decision rule, each additional booking was associated with a 3% increase in the odds of referral (OR=1.03, 95%CI=1.02–1.04). Finally, the odds of screening positive for current use of psychiatric medication did not change across bookings (OR=1.01, 95%CI=1–1.02). As before, results of the models including covariates showed the same the patterns of results. (Full results available in online supplement).

Discussion

Across the United States, the BJMHS is administered multiple times to detainees who cycle in and out of county jails; yet, little is known about how these mental health screening results may change across repeated jail bookings. Drawing data from bookings into a large, metropolitan county jail, we examined whether BJMHS results changed overall and by decision rule. Our findings demonstrate that BJMHS results do change across repeated jail bookings. In particular, we found that the odds of screening positive for prior hospitalization and current psychiatric symptoms decreased across bookings while the odds of screening positive for current psychiatric medication remained unchanged. Odds of referral for any other reason and of screening positive overall increased across bookings. These patterns of results remained even after controlling for demographic variables, homelessness, and substance use. In the sections that follow, we discuss these findings in more detail.

Summary of Findings

Detainees were less likely to screen positive for current psychiatric symptoms across repeated bookings. On the one hand, it is possible that a decrease in the reporting of symptoms reflects improvements in illness trajectories. Indeed, as discussed earlier, jails may represent a critical point of identification, referral, and initiation of psychiatric treatment for many adults with serious mental illness. On the other hand, changes in responses to the symptom items across bookings may not necessarily represent improved illness trajectories. Instead, different responses may reflect natural fluctuation - but not necessarily marked improvement - in clinical presentation (31). Further, symptoms not queried by the BJMHS, such as symptoms associated with anxiety disorders, may have worsened over time and gone undetected by the BJMHS (32).

Our findings also may be attributable to problems associated with the self-report nature of the items, such as differences in insight or willingness to disclose information. Indeed, the question about hospitalization on the BJMHS queries lifetime hospitalization. The fact that detainees were less likely to screen positive for lifetime hospitalization across repeated bookings cannot indicate that lifetime hospitalization rates decreased. Rather, this indicates that reporting of hospitalization decreased among detainees who had previously reported being hospitalized; in other words, a small proportion of detainees (0.5%) recanted their prior hospitalization. This calls into question the reliability of this item across repeated administrations. Alternatively, it is possible that jail staff skipped this question at some bookings or forgot to record detainees’ responses. It is also possible that changes in this item are due to data entry errors. These possibilities are important areas for further investigation.

To further illustrate the susceptibility of BJMHS items to reporting issues, research demonstrates that some people find repeated screening or assessment to be therapeutic, possibly increasing the likelihood they will self-report symptoms, while others find it frustrating and suspicious (33). The jail mental health screening process is unlikely to confer positive therapeutic effects. Instead, it seems more realistic that having to answer the same questions across multiple bookings led detainees to feel that staff are not paying attention to their answers or that staff are trying to catch them giving conflicting answers. Detainees may grow tired of repeating the same responses across multiple screens and become less likely to report current symptoms or prior hospitalization. Further, detainees may be fearful that their disclosing symptoms will be used to deny them pretrial release, lead to involuntary hospitalization, or impact their case in some way (33). Finally, detainees may be especially less likely to disclose mental health information if they feel that the disclosure of this information is not being used in a meaningful way (33), a strong possibility given the limited availability of treatment in jails generally (34) and the study jail, specifically.

In spite of the decreases in self-reported symptoms and hospitalization, the odds of a positive screen overall increased across bookings. This increase appears to be driven by increases in discretionary referral for “any other reason”. This may indicate that jail staff do not rely on the results of the other three decision rules when making referral decisions. In this study, the BJMHS only detected current symptoms in about 2% of detainees while other studies place estimates of serious mental illness at 14.5% for men and 31.0% for women (2). Staff may not trust the screening results due to incredibly low detection rate which may explain why staff referral rates increase over time even though positive screens for the other three decision rules decrease. At the same time, however, this discretion is built into the BJMHS itself as one of the decision rules dictates that detainees should be referred “if [staff] feel it is necessary for any other reason” (8). Thus, it may not be that staff do not trust the screening results but rather that they do not trust the detainee self-report. Such a “professional override” is included in many instruments designed for use in correctional settings (35). In this way, assessors are using the BJMHS as designed. Whether this rate of professional override is acceptable is questionable; no guidance nor research exists to support the acceptable rate of professional override on this or any other instrument. This is an important avenue of investigation in relation to the BJMHS and more generally.

It is also possible that symptoms not included on the BJMHS increased across repeated bookings and were identified by jail staff, resulting in referral. Or, staff may become familiar with a detainee across repeated bookings and refer based on past rather than current clinical presentation. The familiarity hypothesis suggests that an evaluator will gain additional information with each subsequent administration of a test which leads to improved assessment accuracy (28). However, this hypothesis has not always held when tested within the context of risk assessments (36; 28) and thus, warrants further examination in the context of jail mental health screening. Indeed, confirmation bias suggests that such prior knowledge – or familiarity - may result in decreased screening accuracy over time. It is also possible that changes in screening results across bookings reflect the limited training jail staff receive prior to implementation of the BJMHS. There is no formal training required for the BJMHS, which may impact the fidelity with which the screen is administered and scored. Establishing a standardized training protocol, including booster training, should improve adherence to items and decision rules. Finally, changes in screening results may be due to poor reliability of the BJMHS items. A review of BJMHS studies demonstrated highly variable findings across studies (37) calling into question the tool’s accuracy and consistency in detecting mental health problems. Jails may be better served by other screening measures such as the Correctional Mental Health Screen or the Jail Screening Assessment Tool, which have been demonstrated in other studies to outperform the BJMHS in terms of detecting mental health problems among jail inmates (14, 38).

Limitations

A few limitations of the study design should be noted. First, we do not have information on the number of times a person was booked into this jail prior to our study period. As such, our first BJMHS results may not be the first time they were screened. Second, because the BJMHS is the only mental health screen administered in this jail and follow-up with an in-depth mental health evaluation was inconsistent, we were unable to compare results to another screening tool or against a criterion measure, such as the SCID or clinical diagnosis. For these reasons, we cannot determine whether the observed changes reflect actual changes in mental health status, issues with fidelity of administration, issues of disclosure and reporting of symptoms, characteristics of the BJMHS itself, or otherwise. Third, we did not have access to treatment data and cannot speak to the treatment-related or other mechanisms through which BJMHS results changed. Fourth, we investigated changes in mental health screening results across repeated jail bookings using one tool in one jurisdiction; generalizability of findings must be examined in future research. Despite these limitations, the study’s strength lies in our longitudinal analysis of results of a widely used mental health screening tool in a large jail population. This is the first study of its kind.

Conclusion

Mental health screening tools may be administered repeatedly to individuals who cycle in and out of jails (6). Yet, there has been limited investigation of whether results of these jail mental health screens change over time and the regulatory bodies that issue standards for screening and assessment of jail detainees have not provided guidance regarding screening protocols for repeat detainees (3; 4). Mental health screening results can have serious implications for jail detainees, as they are administered at a critical intercept for identification, referral, and intervention (19). Given that jails have the highest volume of justice-involved individuals in the U.S., mental health screening tools used in jails have the potential to affect millions of people. For these reasons, it is imperative that research examines how mental health screening results change over time, as well as the mechanisms of change. Indeed, examinations of test-retest reliability and measurement of mental health symptoms over time are recognized as crucial steps in the development of psychiatric measures (39). Jail mental health screening tools must be examined in this way as well. Furthermore, decision rule number four, referral for any other reason (i.e., the professional override), was the only decision rule to increase over repeated bookings. While it is positive that jail staff are attentive to the mental health needs of detainees beyond those captured by the BJMHS, it is critical to uncover the factors that jail staff are considering in making these override decisions. On the one hand, such research may elucidate factors that could improve the accuracy and utility of the BJMHS or other jail-based, mental health screening tools. On the other hand, such research may identify jail staff’s reliance on factors irrelevant to mental health but instead reflective of misconceptions or bias. The latter finding would represent issues of concern for implementation that may require training. Professional overrides are an understudied component of jail assessment tools. Our findings provide evidence of an increased use of professional overrides across repeated assessments and emphasize the need for research to understand the use of professional overrides in jail assessment tools.

Highlights.

• In a population of detainees booked into a large urban jail, we found that the odds of screening positive overall on the Brief Jail Mental Health Screen (BJMHS) increased across repeated jail bookings.

• Examination of decision rules showed that the odds of screening positive for two or more current symptoms and prior hospitalization decreased across repeated jail bookings, while the odds of referral for any other reason increased and the odds of screening positive for current psychiatric medication did not change.

• Further research is needed to identify mechanisms for change in jail mental health screening results and the generalizability of findings.