Abstract
Objectives
To assess: (1) the prevalence of mental health and substance use in patients presenting to the emergency department (ED) through use of a computer adaptive test (CAT‐MH), (2) the correlation among CAT‐MH scores and self‐ and clinician‐reported assessments, and (3) the association between CAT‐MH scores and ED utilization in the year prior and 30 days after enrollment.
Methods
This was a single‐center observational study of adult patients presenting to the ED for somatic complaints (97%) from May 2019 to March 2020. The main outcomes were computer‐adaptive‐assessed domains of suicidality, depression, anxiety, post‐traumatic stress disorder (PTSD), and substance use. We conducted Pearson correlations and logistic regression for objectives 2 and 3, respectively.
Results
From a sample of 794 patients, the proportion of those at moderate/severe risk was: 24.1% (suicidality), 8.3% (depression), 16.5% (anxiety), 12.3% (PTSD), and 20.4% (substance use). CAT‐MH domains were highly correlated with self‐report assessments (r = 0.49–0.79). Individuals who had 2 or more ED visits in the prior year had 62% increased odds of being in the intermediate‐high suicide risk category (odds ratio [OR], 1.62; 95% confidence interval [CI], 1.07–2.44) compared to those with zero prior ED visits. Individuals who scored in the intermediate‐high‐suicide risk group had 63% greater odds of an ED visit within 30 days after enrollment compared to those who scored as low risk (OR, 1.63; 95% CI, 1.09, 2.44).
Conclusion
The CAT‐MH documented that a considerable proportion of ED patients presenting for somatic problems had mental health conditions, even if mild. Mental health problems were also associated with ED utilization.
Keywords: anxiety, depression, mental health, screening, suicide, utilization
1. INTRODUCTION
1.1. Background
According to the 2019 National Hospital Ambulatory Medical Care Survey, there are approximately 150 million visits to US emergency departments (ED) annually. 1 Visits for psychiatric or substance abuse account for approximately 10% of these visits as of 2016. 2 Importantly, this represents a dramatic rise (approximately 30%) in the number of visits primarily for mental health complaints over the past 2 decades, which has outpaced the rise in overall ED visitation. 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ED service delivery costs associated with the most prevalent mental health and substance use diagnoses are in excess of $3.5 billion, with anxiety and depression accounting for $1.7 billion annually. 10 The severity of mental health symptomology is positively correlated with rate of ED use. 11 , 12 However, mental health problems may be missed during ED encounters. In fact, 45% of patients who are seen in the ED for non‐psychiatric primary complaints have comorbid psychiatric disorders, including major depression, general anxiety, substance abuse, and suicidal ideation. 13 , 14 , 15 The ED offers the opportunity to screen for mental health problems.
1.2. Importance
The barriers to standard and truly universal mental health screening in the ED are many and include the competing priorities of ED physicians delivering care to critically ill patients in busy and crowded ED environments, limited access to adequate mental health resources for consultation and outpatient care, as well as the training and attitudes of ED clinicians. 16 , 17 These same barriers have limited the approach to widespread mental health screening in the ED and represent a major missed opportunity to provide and/or triage to appropriate care as a significant proportion of ED patients presenting with non‐psychiatric complaints have comorbid mental health issues as noted above. It is noteworthy that many patients who die by suicide often visit the ED or access the healthcare system shortly before their attempt for a non‐psychiatric reason. 18 , 19 Psychiatric screening and subsequent mental health care triage in the ED may have downstream alleviation on the healthcare system.
1.3. Goals of this investigation
In this study, we build and expand on the work of Beiser et al who demonstrated the feasibility of using the brief computer adaptive diagnostic screen (the CAD‐MDD 12 ) and depression severity measurement test (CAT‐DI 20 ) for depression assessment in a broad sample of ED patients with non‐psychiatric primary complaints. Whereas the previous study evaluated 1 domain, our study evaluates 5 domains. Our objectives in this study were 3‐fold. First, we aimed to document the prevalence of suicidality, depression, anxiety, post‐traumatic stress disorder (PTSD), and substance abuse in an ED population presenting primarily for somatic complaints. Second, we compared the results of the computer adaptive test‐mental health (CAT‐MH) standardized screening to standard practice ED assessments for suicide, clinician‐reported rating of suspected suicide attempt in the 30 days post‐enrollment, and other validated self‐report screening tools for anxiety, depression, somatic symptoms, and pain. Finally, we examined the association between ED utilization and mental health problems among ED patients.
2. METHODS
2.1. Study design and setting
This was a single‐center prospective observational sample of randomly selected non‐consecutive patients presenting to a 90,000 annual visit Midwestern ED over the course of 10 consecutive months. This study was approved by the Indiana University School of Medicine institutional review board (protocol no. 1904527368), and all participants gave informed consent. Patient enrollment began on May 24, 2019, and ceased on March 10, 2020 with the rapid increase of coronavirus disease 2019 (COVID‐19) cases and resulting stay‐at‐home orders.
2.2. Selection of participants
Trained research personnel used the ED tracking system within the electronic medical record (EMR) system (Cerner Firstnet) to screen for potentially eligible subjects presenting to the ED primarily during weekday daytime hours. To reduce sampling bias, research personnel began each 4‐ to 6‐hour screening shift by selecting a random digit between 0 and 9 using an online random number generator. All subjects regardless of triage status, method of arrival, or ED location (except dedicated psychiatric observation rooms) who were on the ED census at that time with the last digit of their age matching the selected number were eligible for screening. Each potentially eligible patient was screened for enrollment sequentially provided they remained on the ED census. This random selection process was repeated each time the list of potentially eligible subjects was exhausted. The treating clinicianss for screened subjects were approached to confirm eligibility criteria prior to patient approach. Exclusion criteria for the study included: (1) age younger than 18 years; (2) being under involuntary detention; (3) lack of or impaired decisional capacity (eg, active psychosis, dementia, developmental delay, intoxication); (4) hemodynamic instability according to the treating clinician; (5) non‐English speaking; and (6) those that may encounter issues with utilization review (eg, prisoner, living out of the state). Therefore, the cross‐section of included patients were ED subjects without overt barriers to providing consent, the majority of whom were presenting with chief somatic complaints. Subjects meeting all enrollment criteria were approached in their private or semi‐private care area for informed consent unless it would interfere with active patient care, or they were no longer on the ED census. The informed consent stated the purpose of the study was to use a “tool called the CAT‐MH to check for a variety of psychological conditions (including depression, anxiety, post‐traumatic stress disorder [PTSD], etc) in patients who need the services of the emergency department…to develop and allot appropriate resources for our patients.” Participants were not compensated for participation.
2.3. Additional data sources
ED utilization data was obtained through the EMR and the Indiana Network for Patient Care managed by the Indiana Health Information Exchange, the nation's largest inter‐organizational clinical data repository, 21 to collect data regarding disposition (admit or discharge) for the index visit, discharge diagnoses, and ED utilization in the 12 months before and after enrollment. Note that, first, time‐to‐event analyses were not possible given that ED utilization data were in the form of total number of visits, rather than specific dates. Second, although we intended to examine ED utilization 12 months before and after enrollment, we limited our analyses to 12 months prior and 30 days after enrollment. Research has demonstrated that the COVID‐19 pandemic greatly affected health care use, particularly in EDs, 22 which would have systematically biased our long‐term follow‐up data.
The Bottom Line
Mental health and substance abuse issues are of significant concern in emergency departments (ED), but current screening practices are imperfect. This single‐center observational study tested the use of a computer‐adaptive test to assess for mental health and substance abuse problems in patients presenting for somatic complaints. In this study, the test was able to identify patients that were at risk for suicidality, depression, anxiety, and substance abuse. Patients who were identified as at risk were more likely to have had past visits to the ED (odds ratio [OR], 1.62) and were more likely to have subsequent visits to the ED (OR, 1.63).
2.4. Variables
2.4.1. Computer adaptive testing with the CAT‐MH
The CAT‐MH 20 , 23 is a suite of mental health computerized adaptive tests (CATs), that can be self‐administered via any internet capable device (computer, tablet, smartphone), either in the clinic, ED, or remotely via a secure patient portal. The CAT‐MH is unique in that unlike traditional CATs developed for educational assessments that are based on unidimensional item response theory (IRT) and are only appropriate for simple constructs such as mathematical ability, the CAT‐MH is based on multidimensional item response theory and is suitable for the measurement of complex constructs such as depression, anxiety, and suicidality. For a given construct (eg, depression) the CAT‐MH draws symptom items from a large bank of potential items (eg, 389 items for depression) that completely covers the latent variable of interest from the lowest to the highest levels of severity. The CAT begins by selecting an item from the middle of the severity continuum. Based on the item response, typically on a 5‐point Likert scale, a provisional severity score and its uncertainty are computed. Based on that score, the next maximally informative item is selected and administered. Based on the response to the second item, the severity score and uncertainty are recomputed. The process continues until the uncertainty in the estimated score falls below 5 points on a 100‐point scale.
All of the CAT‐MH scales have been validated against structured clinical interview and we included CATs for suicidality (CAT‐SS 24 ), depression (CAT‐DI 20 ), anxiety (CAT‐ANX 25 ), PTSD (CAT‐PTSD—an adaptive version of the 20‐item PCL‐5 26 ), and substance use disorder (CAT‐SUD 27 ) in our study. See Appendix S1 for additional details. The CAT‐SUD also determines frequency of use of alcohol, sedatives/hypnotics, opioid analgesics, heroin/methadone, and cocaine/amphetamines. The CAT‐SS has been prospectively validated in terms of its ability to predict future suicidal behavior. 28 , 29 , 30 Approximately 10 items are administered for the suicide subscale, 12 for depression, 12 for anxiety, 6 for PTSD, and 11 for substance use. The CAT‐SS, 24 CAT‐DI, 12 and CAT‐SUD 31 have been previously implemented in EDs.
2.4.2. Standard ED suicide screening
During the study period, standard universal suicide screening took place at the enrollment site ED for individuals not presenting with a chief complaint of suicidal ideation. Please see Appendix S2 for the standard screening administered by nursing personnel through an electronic medical record workflow in a private care area, consisting of questions closely aligned with the Columbia‐Suicide Severity Rating Scale (C‐SSRS). Per ED protocol, treating clinicians were informed of anyone who screened positive for suicidal ideation using this tool. For the purposes of this study, these questions will be referred to as the “standard ED suicide screen.”
Additionally, before patient enrollment and for the purposes of this study, the clinician (either attending physician [17%], resident [65%], or advanced practice provider [18%]) was asked to quantify their concern for a future suicide attempt by marking a 100‐mm visual analog scale (VAS) in response to the question: “What is your level of suspicion that this patient will attempt suicide in the next 30 days?” Additionally, the CAT‐MH, as described abo, also screened for suicide risk.
2.4.3. Non‐adaptive self‐administered screening tools
Socio‐demographic data were collected through patient self‐report. Self‐reported comorbidity/past medical history was assessed by asking the participant to indicate if a doctor or another health care worker had diagnosed or treated the participant for 13 medical items in the past 3 years (see Table 1). Chief complaint was transcribed directly from that location in the EMR.
TABLE 1.
Frequency distribution of intake variables for entire sample and sample screening intermediate‐high suicide risk on CAT‐MH
| Individuals consenting for CAT‐MH (N = 794) | Individuals screened intermediate‐high CAT‐MH suicide risk (N = 191) | |
|---|---|---|
| Demographic information | No. (%) a | No. (%) b |
| Gender | ||
| Female | 492 (62.0) | 125 (65.5) |
| Male | 301 (37.9) | 66 (34.6) |
| Other | 1 (0.1) | 0 (0) |
| Ethnicity | ||
| Hispanic or Latino | 42 (5.3) | 11 (5.8) |
| Not Hispanic or Latino | 725 (91.3) | 174 (91.1) |
| Unknown or not reported | 27 (3.4) | 6 (3.1) |
| Race | ||
| White | 413 (52.0) | 105 (55.0) |
| Black or African American | 316 (39.8) | 65 (34.0) |
| Asian | 3 (0.4) | 0 (0) |
| American Indian/Alaska Native | 6 (0.8) | 2 (1.1) |
| Native Hawaiian or other Pacific Islander | 0 (0) | 0 (0) |
| More than 1 race | 36 (4.5) | 15 (7.9) |
| Other | 20 (2.5) | 4 (2.1) |
| Marital status | ||
| Single | 422 (53.2) | 101 (52.9) |
| Married | 224 (28.2) | 38 (19.9) |
| Separated | 21 (2.6) | 9 (4.7) |
| Divorced | 91 (11.5) | 31 (16.2) |
| Widowed | 36 (4.5) | 12 (6.3) |
| Education | ||
| Some high school | 126 (15.9) | 43 (22.5) |
| High school | 230 (29.0) | 39 (20.4) |
| GED | 42 (5.3) | 12 (6.3) |
| Some college | 210 (26.5) | 51 (26.7) |
| Graduate college | 149 (18.8) | 38 (19.9) |
| Graduate/professional school | 37 (4.7) | 8 (4.2) |
| Employment | ||
| Employed full‐time | 318 (40.1) | 63 (33.0) |
| Employed part‐time | 86 (10.8) | 26 (13.6) |
| Student and working | 19 (2.4) | 3 (1.6) |
| Student and not working | 19 (2.4) | 4 (2.1) |
| Homemaker | 19 (2.4) | 7 (3.7) |
| Unemployed | 99 (12.5) | 28 (14.7) |
| Retired | 80 (10.1) | 10 (5.2) |
| Disabled | 154 (19.4) | 50 (26.2) |
| Chief complaint | ||
| Abdominal pain | 167 (21.0) | 45 (23.6) |
| Abnormal lab/test results | 3 (0.4) | 2 (1.1) |
| Abnormal vital signs (heart rate, blood pressure, etc) | 14 (1.8) | 1 (0.5) |
| Allergic reaction | 3 (0.4) | 1 (0.5) |
| Anxiety | 3 (0.4) | 2 (1.1) |
| Chest pain | 91 (11.5) | 17 (8.9) |
| Ear, nose, and throat complaint | 28 (3.5) | 6 (3.1) |
| Eye complaint | 7 (0.9) | 0 (0) |
| Facial pain | 8 (1.0) | 2 (1.1) |
| Fever/chills/sepsis | 12 (1.5) | 0 (0) |
| Other musculoskeletal pain (back, extremity, etc) | 105 (13.2) | 25 (13.1) |
| Gastrointestinal problem (vomiting, diarrhea, etc) | 51 (6.4) | 14 (7.3) |
| Genitourinary problem (kidney complaints, etc) | 14 (1.8) | 4 (2.1) |
| Headache | 26 (3.3) | 5 (2.6) |
| Medical screen/evaluation | 10 (1.3) | 3 (1.6) |
| Neurological complaint (seizure, focal weakness, etc) | 18 (2.3) | 4 (2.1) |
| Obstetric‐gynecological complaint | 21 (2.6) | 7 (3.7) |
| Post‐operative complication | 4 (0.5) | 2 (1.1) |
| Respiratory problem (cough, asthma, shortness of breath) | 78 (9.8) | 15 (7.9) |
| Sickle cell crisis | 11 (1.4) | 1 (0.5) |
| Skin/soft tissue infection or complaint | 22 (2.8) | 6 (3.1) |
| Substance use | 1 (0.1) | 1 (0.5) |
| Trauma (motor vehicle accident, fall, assault, etc) | 34 (4.3) | 5 (2.6) |
| Weakness/fatigue and dizziness | 44 (5.5) | 14 (7.3) |
| Miscellaneous | 16 (2.0) | 8 (4.2) |
| Invalid | 2 (0.3) | 1 (0.5) |
| Missing | 1 (0.1) | 0 (0) |
| Health history information | ||
| Currently have a primary care physician | 580 (73.0) | 136 (71.2) |
| History of: | ||
| Asthma, emphysema, chronic bronchitis | 251 (31.6) | 75 (39.3) |
| High blood pressure or hypertension | 341 (42.9) | 82 (42.9) |
| High blood sugar or diabetes | 175 (22.0) | 46 (24.1) |
| Arthritis or rheumatism | 226 (28.3) | 63 (33.3) |
| Heart attack, angina, heart failure, or other types of heart disease | 140 (17.6) | 36 (18.9) |
| Stroke, seizures, Parkinson's disease, or another neurological condition | 102 (12.8) | 40 (20.9) |
| Liver disease | 40 (5.0) | 12 (6.3) |
| Kidney or renal disease | 82 (10.3) | 27 (14.4) |
| Cancer diagnosed or treated in the last 3 years | 58 (7.3) | 12 (6.3) |
| Anxiety | 323 (40.7) | 136 (71.2) |
| Depression | 307 (38.7) | 141 (73.8) |
| PTSD | 129 (16.2) | 67 (35.1) |
| Suicidal thoughts or attempt | 58 (7.3) | 45 (23.6) |
| Current use of: | ||
| Tobacco | 269 (33.9) | 88 (46.1) |
| Alcohol ≥4 times/week | 99 (12.5) | 28 (14.7) |
| Marijuana | 163 (20.5) | 73 (38.2) |
| Cocaine | 13 (1.6) | 5 (2.6) |
| Heroin | 3 (0.4) | 1 (0.5) |
| Other | 13 (1.6) | 7 (3.7) |
| Self‐ or clinician‐reported information | No. (%) | No. (%) b |
|---|---|---|
| GAD‐7 (generalized anxiety disorder) | ||
| Minimal (0–4) | 454 (57.2) | 22 (11.5) |
| Mild (5–9) | 184 (23.2) | 60 (31.4) |
| Moderate (10–14) | 73 (9.2) | 40 (20.9) |
| Severe (≥15) | 83 (10.6) | 69 (36.1) |
| PHQ‐8 (depression) | ||
| Minimal (0–4) | 398 (50.1) | 15 (7.9) |
| Mild (5–9) | 188 (23.7) | 33 (17.3) |
| Moderate (10–14) | 118 (14.9) | 61 (31.9) |
| Moderately severe (15–19) | 58 (7.3) | 51 (26.7) |
| Severe (≥20) | 32 (4.0) | 31 (16.2) |
| PCS‐13 (pain catastrophizing) | ||
| None/mild (0–29) | 705 (88.8) | 124 (64.9) |
| Clinically significant (≥30) | 89 (11.2) | 67 (35.1) |
| PHQ‐15 (somatic symptoms) | ||
| Very low (0–4) | 135 (19.3) c | 10 (6.0) d |
| Low (5–9) | 236 (33.8) c | 35 (21.1) d |
| Medium (10–14) | 195 (27.9) c | 51 (30.7) d |
| High (≥15) | 133 (19.0) c | 70 (42.1) d |
| M (SD); median; range | M (SD); median; range | |
| Clinician‐suspected suicide attempt within 30 days (per VAS) | 5.0 (8.9); 0; 0–74 | 6.3 (11.3); 0; 0–74 |
| Admission information | ||
| Number of ED visits within 12 months before enrollment date e | 2.1 (4.7); 1; 0–76 | 3.3 (7.6); 1; 0–76 |
| Number of repeated ED visits in 30 days since enrollment date f | 0.3 (0.8); 0; 0–9 | 0.5 (1.1); 0; 0–7 |
Abbreviation: VAS, visual analog scale.
Based on 794 unique individuals. Rounded to nearest tenth; may not equal 100%.
Based on 191 individuals.
Based on 699 unique individuals.
Based on 166 individuals.
Proportion of individuals in the consenting sample (n = 794) who were rated as “0” suspected suicide attempt risk was 43.45%. Proportion among intermediate‐high risk suicide individuals (n = 191) was 37.70%.
Proportion of individuals in the consenting sample (n = 794) who were rated as “0” suspected suicide attempt risk was 78.46%. Proportion among intermediate‐high risk suicide individuals (n = 191) was 70.16%.
Subsequently, each participant underwent self‐administered screening in 4 domains via a secure tablet. (1) The Generalized Anxiety Disorder 7‐item scale (GAD‐7) is a rapid screening tool for the presence of clinically significant anxiety. 32 , 33 , 34 , 35 The scores range from 0 to 21, with cut points of 5, 10, and 15 corresponding to mild, moderate, and severe anxiety, respectively. (2) The Patient Health Questionnaire 8 item scale (PHQ‐8) is a rapid screening tool for the presence of clinically significant depression. 35 , 36 , 37 , 38 , 39 , 40 , 41 The scores range from 0 to 24, with cut points of 5, 10, 15, and 20 corresponding to mild, moderate, moderately severe, and severe depression, respectively. We used the PHQ‐8 instead of the PHQ‐9 due to suicidality assessment via the C‐SSRS items and little difference in sensitivity and specificity between the PHQ‐8 and 9. 42 (3) The Patient Health Questionnaire‐15 (PHQ‐15) is used to detect participants at risk for somatoform disorders; 43 , 44 scores range from 0 to 30 it has cutoffs of 5, 10, and 15, corresponding to low, medium, and high somatic symptom severity. (4) The Pain Catastrophizing Score (PCS) is a 13‐item self‐administered tool used to quantify a person's thoughts and feelings when they have experienced pain. 45 , 46 The maximum total score is 52 and total scores ≥30 (corresponding to the 75th percentile) are considered clinically significant. In a sample of chronic pain patients who scored ≥30, 70% identified as occupationally disabled, and 66% had moderate depression. 45 The PCS measures rumination and helplessness, which are related to mental health disorders. 47 , 48
2.5. Statistical methods
For objective 1, we documented the percentage of patients endorsing different categories (eg, normal, mild, moderate or intermediate, severe, or high) of the 5 domains assessed by the CAT‐MH. The categories were pre‐established in the original articles based on thresholds producing high sensitivity for the underlying diagnosis (eg, SCID for DSM‐5) for mild, moderate sensitivity and specificity for moderate, and high specificity for severe). We present this for all the patients, as well as a subgroup of patients who scored in the intermediate to high categories of suicidal risk.
For objective 2, we correlated the CAT‐MH scores with clinician‐reported suspected risk of suicide attempt and self‐reported measures (GAD‐7, PHQ‐8, PHQ‐15, and PCS). We then compared the identification of patients at risk for suicide using the CAT‐MH with the standard ED suicide screen procedure.
For objective 3, we conducted logistic regression analyses between the number of ED visits in the year prior to enrollment and CAT‐MH subscales. The predictors, number of ED visits within the prior year, were dummy coded into 3 categories: (1) 0 visits in the year prior, which served as the reference category; (2) 1 visit prior; and (3) 2 or more visits prior. We analyzed each CAT‐MH subscale separately as a binary outcome, dichotomized suicide and substance use into low risk (0) and intermediate‐high risk (1), depression and anxiety into normal/mild (0) and moderate/severe (1), and PTSD into no evidence (0) and possible/highly likely (1). For depression and anxiety, we included mild symptomatology into the reference (0) group as research has utilized scores ≥10 (moderate) as cut‐off points for a diagnosis of generalized anxiety disorder or major depressive episode. 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 We included the following demographic covariates: age, gender, ethnicity, race, marital status, and employment.
To examine the association between the CAT‐MH subscales and subsequent ED visits within 30 days of ED enrollment date, we conducted logistic regression analyses adjusting for demographics. The predictors were each of the categorical risk levels for the CAT‐MH subscales, which we dummy coded such that the lowest level of risk served as the reference category. Given the reduced sample size in the high‐risk categories, we combined all the categories above low risk. For depression and anxiety subscales, mild was also included in the reference category. We dichotomized the outcome into not present (0) or any ED visit (1) within 30 days. All analyses were 2‐tailed at an alpha‐level of 0.01.
2.6. Sensitivity analyses
We conducted 3 sensitivity analyses to examine the potential impact of analytical decisions (ie, categorization of subscales, dichotomization of ED visit recidivism) on the results and interpretation. First, we predicted the continuous CAT‐MH subscales from ED visits in the year prior to enrollment. We included an unstandardized and standardized outcome to examine the magnitude of the effects in terms of a single point increase on the 100‐point scale and in terms of standard deviation units, respectively. Second, when assessing the associations with mental health problems with subsequent ED visits, we utilized continuous, standardized CAT‐MH subscales rather than utilizing categorical CAT‐MH predictors. Third, we conducted ordinal logistic regression to predict ED visits within 30 days after enrollment to examine the association with each CAT‐MH category.
3. RESULTS
3.1. Patient enrollment
In the study, 1854 potentially eligible patients were screened and approached for enrollment. Of those subjects who were approached for enrollment in the study, 828 (44.7%) underwent informed consent and were enrolled. Of those who consented, 97.3% (n = 806) completed the CAT‐MH. Reasons for lack of completion in the remaining 22 subjects included study withdrawal, and excessive interruptions due to clinical care. To conduct a complete case analysis, we dropped an additional 12 individuals with the following missing indicators: demographic variables, use of ED care within 30 days post‐discharge, and the PTSD subscale on the CAT‐MH. We also dropped 1 individual with an extreme outlier of ED care (ie, 75 visits) within 30 days post‐discharge and 2 individuals with invalid ages. The final analytic sample consisted of 794 individuals. See Appendix S3 for the data flowchart. The median time for CAT‐MH completion for the 5 domains was 9.7 minutes (interquartile range [IQR], 5.6 minutes) and mean was 10.8 minutes (SD, 5.5 minutes).
3.2. Sample demographics
Table 1 presents the overall breakdown of our population including frequency distribution of demographic, chief complaint (97% somatic), medical history, suicide, admission, and self‐questionnaire (past medical history, GAD‐7, PHQ‐8, PHQ‐15, PCS) of the subjects included in the sample. Additionally, this information is also provided for the subgroup of patients who screened intermediate‐high‐risk on the CAT‐MH suicide severity rating (n = 191) and is presented in the right‐hand column in Table 1. The frequencies of demographic, chief compliant, and medical information were similarly distributed for individuals who screened intermediate‐high suicide risk as for the entire sample. Individuals who screened intermediate‐high suicide risk had a higher proportion of those who endorsed history of PTSD or suicidal ideation, as well as a higher mean of prior ED visits within the past year, as compared to the entire sample.
3.3. Objective 1
Table 2 presents the frequency distribution of the CAT‐MH subscales defined both categorically and continuously. The continuous subscales demonstrated acceptable skewness and kurtosis. The proportion of those who scored above the threshold of low risk/no evidence group for the following subscales were: (1) 24.1% for suicide, (2) 12.3% for PTSD, and (3) 20.4% for substance use. The proportion of those who scored above the threshold of normal/mild risk group for the following subscales were: (1) 8.3% for depression, and (2) 16.5% for anxiety. Across all categories, approximately 31.9% (n = 253) of the sample screened above the threshold for at least one CAT‐MH subscale and 20.7% (n = 164) for 2 or more CAT‐MH subscales. Approximately one‐third of the sample population (31.9%) had at least 1 high CAT‐MH subscale and 20% of the sample had at least 2 high subscales scores. For example, 16.5% and 8.3% screened positive for moderate/severe anxiety and depression, respectively. The prevalence of moderate or severe depression in this population (8.3%) by CAT‐MH screening is similar to the prevalence (7%) reported in another ED sample using the CAT‐MH. 49 See Appendix S4 for the frequency distribution of use and abuse of specific substances.
TABLE 2.
Frequency distribution of CAT‐MHTM variables for entire sample and those who screened intermediate‐high suicide risk on CAT‐MH
| CAT‐MH categorical subscales | Individuals consenting for CAT‐MH a | Individuals screened intermediate‐high CAT‐MH suicide risk b |
|---|---|---|
| No. (%) | No. (%) | |
| Depression subscale | ||
| Normal | 433 (54.5) | 9 (4.7) |
| Mild | 295 (37.2) | 120 (62.8) |
| Moderate | 35 (4.4) | 32 (16.8) |
| Severe | 31 (3.9) | 30 (15.7) |
| Anxiety subscale | ||
| Normal | 537 (67.6) | 28 (14.7) |
| Mild | 126 (15.9) | 52 (27.2) |
| Moderate | 69 (8.7) | 54 (28.3) |
| Severe | 62 (7.8) | 57 (29.8) |
| PTSD subscale | ||
| No evidence | 696 (87.7) | 104 (54.5) |
| Possible | 74 (9.3) | 63 (33.0) |
| Highly likely | 24 (3.0) | 24 (12.6) |
| Substance use subscale | ||
| Low risk | 632 (79.6) | 73 (38.2) |
| Intermediate risk | 143 (18.0) | 106 (55.5) |
| High risk | 19 (2.4) | 12 (6.3) |
| CAT‐MH continuous subscales | M (SD) skewness; kurtosis | M (SD) skewness; kurtosis |
|---|---|---|
| Depression subscale | 34.1 (21.0); 0.5; −0.2 | 60.2 (15.3); 0.3; −0.1 |
| Anxiety subscale | 26.0 (23.6); 0.9; <0.1 | 54.5 (20.2); −0.1; 0.1 |
| PTSD subscale | 29.8 (20.8); 0.3; −0.4 | 54.1 (14.4) 0.3; 0.6 |
| Substance use subscale | 34.6 (18.6); −0.1; −0.5 | 52.9 (10.6) 0.5; 0.6 |
Note: Rounded to the nearest tenth, may not equal 100%.
Based on 794 unique individuals who completed the CAT‐MH study protocol.
Based on 191 unique individuals who screened in the intermediate or high‐risk group on suicide severity subscale on the CAT‐MH.
3.4. Objective 2
To examine the overlap between suicide screens, Table 3 presents the cross‐tabulation between the CAT‐MH positive suicide screen (ie, those in the high‐risk suicide severity category) and the standard ED's suicide screen. Ten individuals screened positive on the CAT‐MH suicide screen but only 3 of these screened positive by the standard ED suicide screen. Thus, 7 individuals were not identified via the standard screening. All 7 identified participants were in the high‐risk category on the CAT‐MH suicide subscale. The kappa between the suicide screens is 0.46 (95% confidence interval [CI], 0.13–0.79). Table 4 presents the Pearson correlations between the CAT‐MH subscales and the clinician‐rated suspected risk of suicide attempt rating and self‐reported questionnaires. Each CAT‐MH subscale, including suicidality, was weakly correlated with the clinician rating of suspected suicide attempt, and was strongly correlated with the self‐reported questionnaires indexing of anxiety (GAD‐7), depression (PHQ‐8), pain catastrophizing (PCS), and physical symptoms (PHQ‐15).
TABLE 3.
Cross‐tabulation between suicide screen in ED and CAT‐MH
| CAT‐MH suicide screen (No. %) | ||
|---|---|---|
| Negative | Positive a | |
| ED standard care suicide screening | ||
| Positive | 0 (0) | 3 (0.38) |
| Negative | 760 (95.72) | 7 (1.01) |
| Not recorded | 23 (2.90) | 0 (0) |
Note: Derived from 794 unique individuals. CAT‐MH suicide screen based on suicide severity.
Individuals who screened positive were in the high‐risk suicide severity group.
TABLE 4.
Pearson correlation between CAT‐MH subscales, clinician‐rated concern of suicide attempt, and other self‐report measures
| CAT‐MH Subscale | Clinician rating of suspected suicide attempt in next 30 days a | Sum of anxiety (GAD‐7) r a | Sum of depression (PHQ‐8) r a | Sum of somatic symptoms (PHQ‐15) r b | Sum of pain catastrophizing (PCS) r a |
|---|---|---|---|---|---|
| Suicide subscale | 0.11 | 0.73 | 0.79 | 0.50 | 0.63 |
| Depression subscale | 0.14 | 0.72 | 0.78 | 0.53 | 0.63 |
| Anxiety subscale | 0.12 | 0.76 | 0.74 | 0.53 | 0.62 |
| PTSD subscale | 0.12 | 0.73 | 0.70 | 0.49 | 0.57 |
| Substance use subscale | 0.09 | 0.62 | 0.66 | 0.49 | 0.54 |
All P values <0.01.
Based on 794 unique individuals.
Based on 699 unique individuals.
3.5. Objective 3
Table 5 presents the prediction of dichotomous CAT‐MH subscales from the number of ED visits in the year prior to enrollment. Individuals who had 2 or more ED visits in the prior year had a 62% increase in the odds of being in the intermediate‐high suicide risk category (OR, 1.62; 95% CI, 1.07–2.44) compared to those with 0 prior ED visits. Odds ratios for the other CAT‐MH subscales were the following: depression (OR, 1.55 [0.81–2.94]), anxiety (OR, 1.56 [0.96–2.52]), PTSD (OR, 2.86 [1.63–5.02]), and substance use disorder (OR, 1.49 [0.96–2.31]). Individuals with 1 visit to the ED in the prior year were not more likely to score higher on any of the CAT‐MH subscales compared to those with 0 visits. Appendix S5 presents the point estimates for the covariates whereas Appendix S6 provides the data using log‐linked general linear regression with combined categorical predictors.
TABLE 5.
Prediction of CAT‐MH from prior year ED visits using logistic regression
| Visits ED year prior | Suicide subscale, OR (95% CI; P value) | Depression subscale, OR (95% CI; P value) | Anxiety subscale, OR (95% CI; P value) | PTSD subscale, OR (95% CI; P value) | Substance use subscale, OR (95% CI; P value) |
|---|---|---|---|---|---|
| 0 | REF | REF | REF | REF | REF |
| 1 | 0.86 (0.53–1.41; P = 0.56) | 1.02 (0.47–2.21; P = 0.96) | 1.06 (0.60–1.86; P = 0.84) | 1.64 (0.85–3.14; P = 0.14) | 1.11 (0.67–1.85; P = 0.69) |
| ≥2 | 1.62 (1.07–2.44; P = 0.03) | 1.55 (0.81–2.94; P = 0.18) | 1.56 (0.96–2.52; P = 0.07) | 2.86 (1.63–5.02; P < 0.001) | 1.49 (0.96–2.31; P = 0.08) |
Note: Derived from 794 unique individuals. Demographic variables included are age, gender, ethnicity, race, marital status, and employment. Each CAT‐MH subscale was dichotomized: suicide subscale 0 = low risk, 1 = intermediate‐high risk; depression subscale 0 = normal‐mild; 1 = moderate‐severe risk; anxiety subscale 0 = normal‐mild; 1 = moderate‐severe; PTSD subscale 0 = no evidence, 1 = possible‐highly likely; substance use subscale 0 = low risk, 1 = intermediate‐high risk.
Table 6 presents the prediction of dichotomous ED visits (not present/any) within 30 days from CAT‐MH subscales. Individuals who scored in the intermediate‐high‐suicide risk group had 63% greater odds of an ED visit within 30 days after their enrollment compared to those who scored as low risk (OR, 1.63; 95% CI, 1.09, 2.44). The associations with the other domains assessed by the CAT‐MH were not statistically significant and the confidence intervals around the estimates were quite wide. Appendix S7 presents the point estimates for the covariates whereas Appendix S8 provides the data using log‐linked general linear regression with combined categorical predictors.
TABLE 6.
Prediction of ED visits within 30 days using logistic regression with combined categorical predictors
| OR (95% CI; P value) | |
|---|---|
| Suicide | |
| Low risk | REF |
| Intermediate‐high risk | 1.63 (1.09–2.44; P = 0.02) |
| Depression | |
| Normal‐mild | REF |
| Moderate‐severe | 0.97 (0.52–1.81; P = 0.93) |
| Anxiety | |
| Normal‐mild | REF |
| Moderate‐severe | 1.12 (0.70–1.78; P = 0.65) |
| PTSD | |
| No evidence | REF |
| Possible‐highly likely | 1.07 (0.63–1.82; P = 0.80) |
| Substance use | |
| Low risk | REF |
| Intermediate‐high risk | 0.85 (0.55–1.33; P = 0.48) |
Note: Derived from 794 unique individuals. ED visits dichotomized into not present/any (0/1). Demographic variables included are age, gender, ethnicity, race, marital status, and employment.
3.6. Sensitivity analyses
First, we predicted continuous CAT‐MH subscales from visits to the ED within the 12 months prior to enrollment. The pattern of results was similar to the main analyses. For example, patients who visited the ED 2 or more times in the previous year scored slightly higher (4.6 points, 0.25 SD difference) on the continuous suicide subscale (Appendix S9 Second, we predicted ED admissions 30 days after enrollment from standardized continuous CAT‐MH subscales. The trend of results was similar to the main analyses (Appendix S10). For suicide, each SD increase in the suicide subscale was associated with a 19% increased odds of ED admission. Third, when conducting ordinal logistic regression to predict ED visits within 30 days after enrollment, the results followed a similar pattern as the main analyses, although certain estimates were statistically significant (Appendix S11).
4. LIMITATIONS
There are several important limitations to mention. Given that this study depended on voluntary participation, it is unclear if those with underlying physical or psychological conditions were more or less likely to volunteer to participate given factors such as a willingness to disclose and seek help or concern regarding persistent mental health stigmas. Although we randomized the possibility of study inclusion, we relied on a convenience sample of individuals presenting to the ED, and approximately 45% of individuals did not consent to be included in the study. The CAT‐MH tool is offered in different languages, but we excluded those who were non‐English speaking as our enrollment staff would not be able to have an informed consent conversation in another language. Additionally, those who were felt to have issues affecting utilization review (eg, out of state residence) were also excluded. In any case, these all represent a form of selection bias, potentially influencing our results. If individuals with mental health conditions are more inclined to decline participation, it is possible that our results underestimate the magnitude of impairing mental health and/or substance use conditions. The opposite would be true if those with mental health conditions were overrepresented in the sample. Although our data cannot determine the direction of bias, the prevalence of mental health symptomatology is comparable to prior research. 13 Our overall enrollment goals (1000 patients) and ED recidivism targets (12 months) were curtailed by the institution of COVID‐19 precautions and stay‐at‐home orders. Increasing enrollment may have aided our ability to examine separate levels of the CAT‐MH subscales. Finally, although we examined ED utilization, we cannot infer a causal relation between mental health/substance use problems and ED use.
5. DISCUSSION
This prospective observational study represents the most complete use of the validated computer adaptive tool (CAT‐MH) in the ED environment for multidomain mental health screening in a broad random sample of patients presenting to a large urban emergency department primarily for non‐psychiatric complaints. Because a significant proportion of ED patients who are evaluated for non‐psychiatric complaints (eg, chest pain and abdominal pain) have comorbid mental health problems, which often remain undiagnosed during ED encounters, 13 , 14 , 15 , 50 mental health screening is a critical first step in identifying complex symptomatology, triaging care, and potentially preventing ED recidivism.
Estimates suggest that approximately 1% of all US ED visits are related to suicidal ideation. 9 In our enrolled sample, 3 participants screened positive for suicidal ideation according to standard ED suicide screening procedure as described above. Although the CAT‐MH correctly identified each of these individuals and categorized them as high risk, this tool detected 7 additional high‐risk individuals not flagged by standard screening. We cannot determine whether these individuals were false‐positives per other suicide screens, although these individuals were elevated on an underlying continuum of suicidality and likely required clinical attention. Future research is needed to compare the CAT‐MH to other standard suicide practices and predict suicidality to further clarify this finding's implications. Approximately, 22% of individuals scored in the intermediate suicide risk group. Notably, clinician report of suspected risk of future suicide attempt was only very weakly correlated (r = 0.11) with the CAT‐MH index of suicidality. It is possible that patients felt more at ease disclosing their feelings via the tablet administered CAT‐MH rather than directly to the nurse at check in or during the evaluation by the doctor.
One of the aims of our study was to examine the relationship between positive screens for impaired mental health and ED recidivism because mental illness has been shown to be a strong predictor of frequent ED use. 11 , 12 , 51 In our study, compared to those with no visits in the 12 months prior to enrollment, those with 2 or more ED visits were significantly more likely to score in the high ranges for suicide risk, PTSD, and substance use on the CAT‐MH. With regard to subsequent ED use in the 30 days post‐enrollment, suicide risk (intermediate‐high on the CAT‐MH) was associated with an increased likelihood of repeat ED utilization. Taken together, these results are consistent with prior literature showing that approximately 30%–50% of these patients with frequent ED utilization have a mental health or substance use disorder. 11 , 52 , 53 , 54 , 55
5.1. Clinical implications
Although this study did not directly investigate the feasibility of CAT‐MH implementation in an ED setting nor was the intention to inform decision‐making practices, results from this study have important clinical implications. The median response time was 9.69 minutes—roughly 2 minutes per each of the 5 CAT‐MH domains, which is similar to fixed‐item mental health screening tools such as the GAD‐7 and PHQ‐8 depression. Additionally, of those that consented, the extremely high completion rate (97%) is also quite encouraging in that, despite the busy nature of an ED, less than 3% of participants encountered circumstances which prevented timely completion of the tool. Thus, our results suggest that future studies should explore the implementation of the tool in EDs.
The impact of an adaptive screening tool on ED workflow depends on the resources available, as increased screening will need to be met with increased patient follow‐up. In settings with integrated mental health professionals, individuals who screen in the intermediate/high risk categories can meet with clinicians who can conduct further assessments, brief interventions, and coordinate with further care. In settings with limited mental health resources, clinicians may prioritize learning and implementing brief interventions (eg, safety planning, counseling on access to lethal means) for positive suicide risk and developing an up‐to‐date mental health referral list for patients. If an individual screens as positive suicide risk, receives further assessment, and is held in the ED waiting for a psychiatric bed, an adaptive test could be administered multiple times to determine longitudinal changes in suicide risk. Those who screen in low‐risk categories could receive access to internet‐based interventions or follow‐up adaptive screens to facilitate future referrals.
In the study, we were able to demonstrate the use of the validated multidimensional CAT‐MH to document the high rate of impairing mental health conditions and their associated ED utilization in a population of ED patients presenting largely for somatic complaints. There is a strong association between mental health issues and ED recidivism, which may benefit from screening tools such as the CAT‐MH. Future work should focus on assessing the feasibility of universal screening implementation via the multidomain CAT‐MH, how the CAT‐MH can support clinical decision making in an ED setting, and the association between the CAT‐MH suicide subscale specifically with suicide‐related ED visits.
CONFLICTS OF INTEREST
Robert D. Gibbons is a founder of Adaptive Testing Technologies, which distributes the Computerized Adaptive Test—Mental Health suite of computerized adaptive tests (CAT‐MH is a trademark of Adaptive Testing Technologies); the terms of this arrangement have been reviewed and approved by the University of Chicago in accordance with its conflicts of interest policies. The authors report no additional financial or other relationship relevant to the subject of this article.
AUTHOR CONTRIBUTIONS
Brian M. D'Onofrio obtained research funding. Brian M. D'Onofrio, Paul I. Musey, and David G. Beiser conceived and designed the study. Paul I. Musey and Kelli Peterson oversaw patient recruitment and managed the data. Azhar I. Dalal, Serena Maag, Matthew T. Perry, Alex Card, Max B. Bohrer, Jackson Hamersly, and Setarah Mohammad Nader undertook the recruitment of patients. Lauren M. O'Reilly analyzed the data. Paul I. Musey, Lauren M. O'Reilly, and Brian M. D'Onofrio drafted the manuscript, and all authors contributed substantially to the revision.
Supporting information
Supporting Information
ACKNOWLEDGMENT
The study was supported by the Responding to the Additions Crisis Grand Challenge from Indiana University to Brian D'Onofrio. Lauren O'Reilly was supported by the National Institute of Mental Health of the National Institutes of Health (1F31MH121039‐01). Paul Musey was supported by the National Heart, Lung, and Blood Institute as a Trans NIH K12 (K12HL133310) Scholar. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. The supporters had no role in the design, analysis, interpretation, or publication of this study.
Biography
Lauren O'Reilly works in the Department of Psychology and Brain Sciences at Indiana University in Bloomington, Indiana. She is a clinical psychology researcher focusing on the origins of suicidal behavior and how the healthcare system treats it.
O'Reilly LM, Dalal AI, Maag S, et al. Computer adaptive testing to assess impairing behavioral health problems in emergency department patients with somatic complaints. JACEP Open. 2022;3:e12804. 10.1002/emp2.12804
Supervising Editor: Elizabeth Donnelly, PhD, MPH.
Previous Presentation: Lauren O'Reilly presented a selection of the current analyses as a poster at the IASR/AFSP International Summit on Suicide Research from October 24 to October 27, 2021 (virtual conference). Alex Card presented a selection of the current analyses as a poster at the Society for Academic Emergency Medicine National Meeting from May 12 to May 15, 2020 (virtual conference). A version of this manuscript was posted as a non‐peer reviewed pre‐print on medRxiv on January 10, 2022: https://medrxiv.org/cgi/content/short/2022.01.08.22268937v1
REFERENCES
- 1. Cairns C, Kang K. National Hospital Ambulatory Medical Care Survey: 2019 Emergency Department Summary Tables. Journal article. doi: 10.15620/cdc:115748. https://stacks.cdc.gov/view/cdc/115748 [DOI]
- 2. Theriault KM, Rosenheck RA, Rhee TG. Increasing emergency department visits for mental health conditions in the United States. J Clin Psychiatry. 2020;81(5):20m13241. doi: 10.4088/JCP.20m13241 [DOI] [PubMed] [Google Scholar]
- 3. Dark T, Flynn HA, Rust G, Kinsell H, Harman JS. Epidemiology of emergency department visits for anxiety in the United States: 2009–2011. Psychiatr Serv. 2017;68(3):238‐244. doi: 10.1176/appi.ps.201600148 [DOI] [PubMed] [Google Scholar]
- 4. Ballou S, Mitsuhashi S, Sankin LS, et al. Emergency department visits for depression in the United States from 2006 to 2014. Gen Hosp Psychiatry. 2019;59:14‐19. doi: 10.1016/j.genhosppsych.2019.04.015 [DOI] [PubMed] [Google Scholar]
- 5. Capp R, Hardy R, Lindrooth R, Wiler J. National trends in emergency department visits by adults with mental health disorders. J Emerg Med. 2016;51(2):131‐135.e1. doi: 10.1016/j.jemermed.2016.05.002 [DOI] [PubMed] [Google Scholar]
- 6. Hill T, Jiang Y, Friese CR, et al. Analysis of emergency department visits for all reasons by adults with depression in the United States. BMC Emerg Med.. 2020;20(1):51. doi: 10.1186/s12873-020-00347-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Boudreaux ED, Camargo CA Jr., Arias SA, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med. 2016;50(4):445‐453. doi: 10.1016/j.amepre.2015.09.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Weiss AJ, Barrett ML, Heslin KC, Stocks C. Trends in Emergency Department Visits Involving Mental and Substance Use Disorders, 2006–2013: Statistical Brief #216. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Agency for Healthcare Research and Quality (US); 2006. [PubMed] [Google Scholar]
- 9. Owens PL, Fingar KR, Heslin KC, Mutter R, Booth CL. Emergency Department Visits Related to Suicidal Ideation, 2006–2013: Statistical Brief #220. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Agency for Healthcare Research and Quality (US); 2006. [PubMed] [Google Scholar]
- 10. Karaca Z, Moore BJ. Costs of Emergency Department Visits for Mental and Substance Use Disorders in the United States, 2017: statistical Brief #257. Healthcare Cost and Utilization Project (HCUP) Statistical Briefs. Agency for Healthcare Research and Quality (US); 2006. [PubMed] [Google Scholar]
- 11. Niedzwiecki MJ, Sharma PJ, Kanzaria HK, McConville S, Hsia RY. Factors associated with emergency department use by patients with and without mental health diagnoses. JAMA Netw Open. 2018;1(6):e183528. doi: 10.1001/jamanetworkopen.2018.3528 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Beiser DG, Ward CE, Vu M, Laiteerapong N, Gibbons RD. Depression in emergency department patients and association with healthcare utilization. Acad Emerg Med. 2019;28(8):878‐888. doi: 10.1111/acem.13726 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Downey LV, Zun LS, Burke T. Undiagnosed mental illness in the emergency department. J Emerg Med. 2012;43(5):876‐882. doi: 10.1016/j.jemermed.2011.06.055 [DOI] [PubMed] [Google Scholar]
- 14. Claassen CA, Larkin GL. Occult suicidality in an emergency department population. Br J Psychiatry. 2005;186:352‐353. doi: 10.1192/bjp.186.4.352 [DOI] [PubMed] [Google Scholar]
- 15. Ilgen MA, Walton MA, Cunningham RM, et al. Recent suicidal ideation among patients in an inner city emergency department. Suicide Life Threat Behav. 2009;39(5):508‐517. doi: 10.1521/suli.2009.39.5.508 [DOI] [PubMed] [Google Scholar]
- 16. Betz ME, Wintersteen M, Boudreaux ED, et al. Reducing suicide risk: challenges and opportunities in the emergency department. Ann Emerg Med. 2016;68(6):758‐765. doi: 10.1016/j.annemergmed.2016.05.030 [DOI] [PubMed] [Google Scholar]
- 17. Heyland M, Delaney KR, Shattell M. Steps to achieve universal suicide screening in emergency departments: a call to action. J Psychosoc Nurs Ment Health Serv. 2018;56(10):21‐26. doi: 10.3928/02793695-20180503-03 [DOI] [PubMed] [Google Scholar]
- 18. Ahmedani BK, Simon GE, Stewart C, et al. Health care contacts in the year before suicide death. J Gen Intern Med. 2014;29(6):870‐877. doi: 10.1007/s11606-014-2767-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Gairin I, House A, Owens D. Attendance at the accident and emergency department in the year before suicide: retrospective study. Br J Psychiatry. 2003;183:28‐33. doi: 10.1192/bjp.183.1.28 [DOI] [PubMed] [Google Scholar]
- 20. Gibbons RD, Weiss DJ, Pilkonis PA, et al. Development of a computerized adaptive test for depression. Arch Gen Psychiatry. 2012;69(11):1104‐1112. doi: 10.1001/archgenpsychiatry.2012.14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Regenstrief Institute Data Services: Indiana Network for Patient Care. Accessed May 3, 2021, https://www.regenstrief.org/rds/data/
- 22. Ziedan E, Simon KI, Wing C, Effects of state COVID‐19 closure policy on NON‐COVID‐19 health care utilization. NBER Working Paper. 2020;(w27621).
- 23. Gibbons RD, Weiss DJ, Frank E, Kupfer D. Computerized adaptive diagnosis and testing of mental health disorders. Annu Rev Clin Psychol. 2016;12:83‐104. doi: 10.1146/annurev-clinpsy-021815-093634 [DOI] [PubMed] [Google Scholar]
- 24. Gibbons RD, Kupfer D, Frank E, Moore T, Beiser DG, Boudreaux ED. Development of a computerized adaptive test suicide scale‐the CAT‐SS. J Clin Psychiatry. 2017;78(9):1376‐1382. doi: 10.4088/JCP.16m10922 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Gibbons RD, Weiss DJ, Pilkonis PA, et al. Development of the CAT‐ANX: a computerized adaptive test for anxiety. Am J Psychiatry. 2014;171(2):187‐194. doi: 10.1176/appi.ajp.2013.13020178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder checklist for DSM‐5 (PCL‐5): development and initial psychometric evaluation. J Trauma Stress. 2015;28(6):489‐498. doi: 10.1002/jts.22059 [DOI] [PubMed] [Google Scholar]
- 27. Gibbons RD, Alegria M, Markle S, et al. Development of a computerized adaptive substance use disorder scale for screening and measurement: the CAT‐SUD. Addiction. 2020;115(7):1382‐1394. doi: 10.1111/add.14938 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Berona J, Whitton S, Newcomb ME, Mustanski B, Gibbons R. Predicting the transition from suicidal ideation to suicide attempt among sexual and gender minority youths. Psychiatric Services (Washington, DC). 2021;72(11):1261‐1267. doi: 10.1176/appi.ps.202000497 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Brenner LA, Betthauser LM, Penzenik M, Bahraini N, Gibbons RD. Validation of a computerized adaptive test suicide scale (CAT‐SS) among United States military veterans. PloS one. 2022;17(1):e0261920. doi: 10.1371/journal.pone.0261920 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Mustanski B, Whitton SW, Newcomb ME, Clifford A, Ryan DT, Gibbons RD. Predicting suicidality using a computer adaptive test: two longitudinal studies of sexual and gender minority youth. J Consult Clin Psychol. 2021;89(3):166‐175. doi: 10.1037/ccp0000531 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Gibbons RD, Alegria M, Markle S, et al. Development of a computerized adaptive substance use disorder scale for screening and measurement: the CAT‐SUD. Addiction. 2020;115(7):1382‐1394. doi: 10.1111/add.14938 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD‐7. Arch Intern Med. 2006;166(10):1092‐1097. doi: 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
- 33. Plummer F, Manea L, Trepel D, McMillan D. Screening for anxiety disorders with the GAD‐7 and GAD‐2: a systematic review and diagnostic metaanalysis. Gen Hosp Psychiatry. 2016;39:24‐31. doi: 10.1016/j.genhosppsych.2015.11.005 [DOI] [PubMed] [Google Scholar]
- 34. Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med. 2007;146(5):317‐325. [DOI] [PubMed] [Google Scholar]
- 35. Ruiz MA, Zamorano E, Garcia‐Campayo J, Pardo A, Freire O, Rejas J. Validity of the GAD‐7 scale as an outcome measure of disability in patients with generalized anxiety disorders in primary care. J Affect Disord. 2011;128(3):277‐286. doi: 10.1016/j.jad.2010.07.010 [DOI] [PubMed] [Google Scholar]
- 36. Kroenke K, Spitzer RL. The PHQ‐9: a new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):509‐515. [Google Scholar]
- 37. Kroenke K, Spitzer RL, Williams JB. The PHQ‐9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606‐613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Gilbody S, Richards D, Brealey S, Hewitt C. Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta‐analysis. J Gen Intern Med. 2007;22(11):1596‐1602. doi: 10.1007/s11606-007-0333-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Hyphantis T, Kotsis K, Kroenke K, et al. Lower PHQ‐9 cutpoint accurately diagnosed depression in people with long‐term conditions attending the accident and emergency department. J Affect Disord. 2015;176:155‐163. doi: 10.1016/j.jad.2015.01.062 [DOI] [PubMed] [Google Scholar]
- 40. Meltzer AC, Bregman B, Blanchard J. Depression is associated with repeat emergency department visits in patients with non‐specific abdominal pain. West J Emerg Med. 2014;15(3):325‐328. doi: 10.5811/westjem.2013.7.15635 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Al‐Ani M, Winchester DE. Prevalence and overlap of noncardiac conditions in the evaluation of low‐risk acute chest pain patients. Crit Pathw Cardiol. 2015;14(3):97‐102. doi: 10.1097/hpc.0000000000000050 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Wu Y, Levis B, Riehm KE, et al. Equivalency of the diagnostic accuracy of the PHQ‐8 and PHQ‐9: a systematic review and individual participant data meta‐analysis. Psychol Med. 2019;50(8):1368‐1380. doi: 10.1017/S0033291719001314 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Kocalevent RD, Hinz A, Brähler E. Standardization of a screening instrument (PHQ‐15) for somatization syndromes in the general population. BMC Psychiatry. 2013;13:91. doi: 10.1186/1471-244x-13-91 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Kroenke K, Spitzer RL, Williams JB. The PHQ‐15: validity of a new measure for evaluating the severity of somatic symptoms. Psychosom Med. 2002;64(2):258‐266. doi: 10.1097/00006842-200203000-00008 [DOI] [PubMed] [Google Scholar]
- 45. Sullivan MJL, Bishop SR, Pivik J. The pain catastrophizing scale: development and validation. Psychol Assess. 1995;7(4):524‐532. doi: 10.1037/1040-3590.7.4.524 [DOI] [Google Scholar]
- 46. Quartana PJ, Campbell CM, Edwards RR. Pain catastrophizing: a critical review. Expert Rev Neurother. 2009;9(5):745‐758. doi: 10.1586/ern.09.34 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Pryce CR, Azzinnari D, Spinelli S, Seifritz E, Tegethoff M, Meinlschmidt G. Helplessness: a systematic translational review of theory and evidence for its relevance to understanding and treating depression. Pharmacol Ther. 2011;132(3):242‐267. [DOI] [PubMed] [Google Scholar]
- 48. Kirkegaard Thomsen D. The association between rumination and negative affect: a review. Cogn Emot. 2006;20(8):1216‐1235. [Google Scholar]
- 49. Beiser DG, Ward CE, Vu M, Laiteerapong N, Gibbons RD. Screening for depression in emergency department patients and relationship to health care utilization. 2016. [DOI] [PMC free article] [PubMed]
- 50. Abar B, Holub A, Lee J, DeRienzo V, Nobay F. Depression and anxiety among emergency department patients: utilization and barriers to care. Acad Emerg Med. 2017;24(10):1286‐1289. doi: 10.1111/acem.13261 [DOI] [PubMed] [Google Scholar]
- 51. Colligan EM, Pines JM, Colantuoni E, Howell B, Wolff JL. Risk factors for persistent frequent emergency department use in medicare beneficiaries. Ann Emerg Med. 2016;67(6):721‐729. doi: 10.1016/j.annemergmed.2016.01.033 [DOI] [PubMed] [Google Scholar]
- 52. Hunt KA, Weber EJ, Showstack JA, Colby DC, Callaham ML. Characteristics of frequent users of emergency departments. Ann Emerg Med. 2006;48(1):1‐8. doi: 10.1016/j.annemergmed.2005.12.030 [DOI] [PubMed] [Google Scholar]
- 53. Hardy M, Cho A, Stavig A, et al. Understanding frequent emergency department use among primary care patients. Popul Health Manag. 2018;21(1):24‐31. doi: 10.1089/pop.2017.0030 [DOI] [PubMed] [Google Scholar]
- 54. Ondler C, Hegde GG, Carlson JN. Resource utilization and health care charges associated with the most frequent ED users. Am J Emerg Med. 2014;32(10):1215‐1219. doi: 10.1016/j.ajem.2014.07.013 [DOI] [PubMed] [Google Scholar]
- 55. Schmidt EM, Behar S, Barrera A, Cordova M, Beckum L. Potentially preventable medical hospitalizations and emergency department visits by the behavioral health population. J Behav Health Serv Res. 2018;45(3):370‐388. doi: 10.1007/s11414-017-9570-y [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting Information