Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 May 1.
Published in final edited form as: Gen Hosp Psychiatry. 2010 Feb 4;32(3):300–309. doi: 10.1016/j.genhosppsych.2010.01.007

Social and Clinical Factors Associated with Psychiatric Emergency Service (PES) Use and Civil Commitment among African American Youth

Michael A Lindsey 1,2, Sean Joe 3, Jordana Muroff 5, Briggett E Ford 6
PMCID: PMC2862230  NIHMSID: NIHMS169283  PMID: 20430234

Abstract

Purpose

We examined the social and clinical factors associated with arrival status (e.g. involuntary versus voluntary) and civil commitment decisions in psychiatric emergency services (PES) to assess African American youths' help-seeking patterns and entrée into care.

Methods

Patient records were reviewed for 1621 African American youth from an inner-city PES between October 2001 and September 2002. Multivariate logistic regression was used to examine the social and clinical factors associated with arrival status (e.g. involuntary vs. voluntary admission) and case disposition among youth who were involuntarily and voluntarily admitted (e.g. disposition upheld vs. dismissed).

Results

Low income youth with behavior disorders were less likely to arrive voluntarily to PES. Medical insurance, suicidality, DSM diagnosis, substance involvement, GAF scores, and time of day the youth arrived to PES were predictors of voluntary arrival. Older age and GAF scores significantly predicted the decision to uphold an involuntary commitment. Age (younger age less likely), higher GAF scores, insurance status, substance abuse involvement, and arrival time (evening shift) significantly predicted the decision to uphold a voluntary decision.

Implications

Our findings suggest that psychiatric and non-psychiatric factors influence both how African American youth arrive to PES and the decisions made regarding their voluntary/involuntary commitment.

Keywords: Psychiatric emergency services, arrival status, civil commitment, African American youth


According to estimates, 12%–17% (7.5–14 million) of youth in the United States suffer from an emotional or behavioral disorder that impairs their functioning, while only about 20% of all youth with mental health needs receive care. Given their disproportionate residence in resource-poor, urban environments that negatively influence their mental health, urban, African American youth represent a particular risk group with high mental health need and limited service use. Psychiatric emergency services (PES) are typically the first contact or entry point into the mental health service system for adolescents. As problematic symptoms go unrecognized or unaddressed by families and other community-based systems in which youth interact (e.g. schools, child welfare, and juvenile justice), the increased intensity of unaddressed mental health need often leads to more frequent contact with psychiatric and general emergency departments as an entrée point into the mental health system.- Extant research indicates that African American youth are more likely to access PES than other high risk adolescents, and may exhibit a crisis-oriented pattern of care when interfacing with emergency services. Similar to African American adults, the elevated use of PES among African American youth may reflect disparities between African Americans and other ethnic groups regarding the experience of severe mental illness and/or access to care. Given evidence that suggests that the emotional/behavioral symptoms of African American youth are likely to come to the attention of other sectors (e.g. juvenile justice, child welfare) or unlikely to be met by community-based services,- the purpose of this study was to examine the underlying social/clinical factors associated with PES use for this population.

Social/Clinical Factors Associated with PES Use

Very few studies have examined the social/clinical factors related to PES presentations among urban youth, particularly factors influencing their voluntary or involuntary arrival to PES. Most of the available information is in the adult PES literature. For example, studies have found that male gender, African-American ethnicity,- high social support levels, and time of arrival to PES have each been identified as demographic and social factors associated with PES use. Among clinical factors, higher assessed psychiatric need, evidence of self-harm behaviors (or ideation), and behaviorally-linked disorders (or interpersonal violence) have been associated with PES use among adults.

In a study of urgent presentations to PES among ethnic minority youth, age (older), arrival status (involuntary), diagnosis (ADHD), social service involvement, and violent behavior predicted arrival to PES for urgent reasons. Elsewhere, youth referred to PES by police were more likely than youth referred by other sources (e.g. self/family, schools, outpatient mental health programs, and hospitals) to have chaotic family backgrounds (e.g. domestic violence) and more severe psychiatric symptoms, as well as being more likely to have higher clinician perceptions of dangerousness and less likely to be referred for outpatient mental health services. Additionally, adjustment disorder followed by conduct disorder has been identified as the most common diagnosis among youth presenting to PES. Finally, in a recent study emanating from the same data source as the current study, African American and Latino youth were more likely to receive psychotic and behavioral disorders than White youth presenting to PES. African American youth compared to white youth were also less likely to receive depressive disorder, bipolar disorder, and alcohol/substance abuse disorder diagnoses. Elsewhere, urban, African American youth have been found to present to pediatric emergency care with more behavioral problems than suicidal behaviors (e.g. suicide ideation, attempt).

Social/Clinical Factors Associated with Civil Commitment

Criteria for civil commitment (i.e., involuntary admission to inpatient psychiatric care) have been variously defined, but the general consensus among research studies has been that civil commitment includes various combinations of need for treatment and danger to self or others. In a meta-analytic study of involuntary admission to psychiatric care among adults, Nicholson found that older, nonwhite individuals with few resources were more likely to be involuntarily committed into psychiatric hospitalization. Involuntary patients were also more likely to have been involved in assaultive, non-suicidal behaviors prior to psychiatric hospitalization and were more likely to exhibit severe psychiatric symptoms. In a Canadian study of civil commitment among adults, higher scores on dangerousness to others, symptom severity regarding Axis I disorders, and difficulty with self-care were associated with an involuntary admission. Additionally, symptom severity was the strongest predictor, suggesting that those most in need appropriately received care. Interestingly, the availability of beds had no association with inpatient admission rates. Elsewhere, self-harm and perceived dangerousness by others have also been found to be associated with inpatient psychiatric admission among adults.

Studies of civil commitment in PES decision-making among adolescents have primarily focused on civil commitment in aggression management. For example, “perceived dangerousness” of adolescents with mental health problems, particularly among adolescents with behavioral disorders has been found to lead to the willingness of mental health professionals and families to use coercion in getting care for affected youth. In a study examining administrative pediatric mental health services data, the highest proportion of adolescents receiving involuntary psychiatric examinations were 15–17 years old, lived in metropolitan areas, were referred to psychiatric services by law enforcement officials, and exhibited self harm behaviors. Interestingly, in multivariate analyses, age (older) was the only significant predictor of involuntary psychiatric examinations. Aside from these few studies, however, the extant literature regarding the social/clinical factors that ultimately affect the disposition/civil commitment among urban or ethnic minority adolescents remains limited, mostly focusing on adults.

This study has three objectives. First, we describe the general background characteristics of African American youth presenting to PES. Second, we examine the demographic (i.e., age, gender), social (i.e., insurance status, arrival time) and clinical factors (i.e., suicidality, DSM-diagnosed disorder, substance abuse involvement) associated with a voluntary arrival to PES. Third, in two separate multivariate models (i.e., a model for youth with involuntary arrivals and a second model for youth with voluntary arrivals), we examine the demographic, social, and clinical factors associated with whether a civil commitment was upheld. The demographic, social, and clinical factors related to objectives two and three were chosen based on previous PES research, which primarily focused on adults. Thus, this study fills a critical gap in the literature on PES by examining these factors among African American adolescents, a group accessing PES at higher levels than other high risk adolescents. A better understanding of the factors influencing their PES presentations will help to identify where key intervention strategies may be developed to appropriately target care earlier in the illness career of African American adolescents with psychiatric needs, and improve their discharge planning upon accessing PES.

Method

Setting

The Albert Einstein Medical Center's Crisis Response Center (CRC) provides psychiatric emergency services (PES) treatment 24 hours a day, seven days a week. It is one of five Crisis Response Centers in the City of Philadelphia, and the only one that treats children and adolescents. The Albert Einstein PES is part of a large urban general hospital and is situated adjacent to the medical emergency department and is staffed by psychiatrists, social workers, registered nurses, and psychiatric assistants. This comprehensive PES provides psychiatric help for children and adolescents in emotional crisis, which may include drug and alcohol dependence, anxiety, depression, thoughts of harming oneself or others, or hallucinations. All patients who come to the CRC in need of help receive confidential assistance, thus Albert Einstein PES is not bounded by catchment area. About 85% of the children and adolescents seen are covered by county-funded medical assistance insurance. After undergoing a thorough psychiatric examination, patients receive emergency care for stabilization. When an evaluation is completed, staff assists individuals and their families in finding appropriate follow-up services and treatment. This CRC makes dispositions for ongoing care, which includes acute psychiatric hospitalization, drug and alcohol detoxification/rehabilitation, partial hospital programs, and outpatient services. Children and adolescents requiring inpatient services are transported to other hospitals. Those needing outpatient psychiatric services and partial hospital programs are referred to facilities in Philadelphia or adjacent counties. A staff member calls each patient as follow-up to make sure they are getting the help they need.

Sample

Between October 1, 2001, and September 30, 2002, the CRC had 1,922 African American patient visits; these patients were ages 22 and younger. To avoid duplication, only data for the first visit of patients who had multiple visits to the CRC were included in this analysis, thus reducing the potential adverse effect of clustering or non-independence of visits within patients. Further, patient records that were missing primary variables of interest for this study were excluded, resulting in a final sample size of 1621. An examination revealed no significant differences in age, gender, insurance status, arrival status, or civil commitment between cases that were either excluded or included in this study.

Measures

Information abstracted from patient records included age (developmentally distinct groups), gender, ethnicity (cases identified as African American), insurance status (public, private, or no insurance), arrival status (voluntary or involuntary), history of substance use (alcohol, drugs, or both), arrival time (morning shift, evening shift, overnight shift), and case disposition (inpatient, partial, outpatient). Patients are considered voluntary if they are brought to the CRC by a guardian or themselves (if they are age 14 and older). This also includes adolescents who may not have assented to being brought to the CRC, but no formal civil commitment petition was presented. An involuntary arrival status (code 302) requires a formal legal adjudication of a petition and examination. The 302 Emergency Involuntary Commitment Petition (EICP) is done when there is evidence showing reasonable probability that, without intervention, a dangerous behavior (harm to self or others) is probable. The medical records also indicated whether a client's status reverted to voluntary commitment (code 201), allowing the individual to sign themselves out of the psychiatric facility.

DSM-IV Diagnoses

Axis I DSM-IV diagnoses recorded in the medical record by the evaluating psychiatrist (e.g. child and adolescent attending and fellow/resident psychiatrists, general attending, and resident psychiatrists) and were collapsed into the following categories: mood (e.g., major depression, dysthymia, and bipolar disorder), disruptive behavior (e.g., conduct, impulsive, ADHD), and “all other diagnoses.” The latter category included any of the following recorded DSM-IV diagnoses: anxiety disorders, psychosis, other non-depressive mood disorders, V code for relational disorders, pervasive developmental disorders, learning disorders, and substance abuse disorders. This categorization was derived from preliminary analysis indicating that mood and behavior disorders were the primary diagnoses; the low frequency of some disorders of interest listed in the “other” category prevented their inclusion in any meaningful statistical analyses. Comorbid DSM-IV diagnoses were incomplete and therefore were not included in the analyses. Patient Global Assessment of Functioning (GAF) score was also recorded to measure psychological, social, and occupational functioning and level of danger to self or others. Although the GAF rating criteria are often described in nine broad categories, researchers have combined GAF score cutoffs into three categories.- Accordingly, patients' GAF scores were classified according to their clinical meaningfulness: 1 to 40, pervasive impairment; 41 to 60, serious impairment; 61 to 100, mild to minimal impairment.

Substance abuse was indicated if the medical record revealed either a substance abuse DSM-IV diagnosis or a positive history of substance abuse. The patient's suicidality was also noted in the chart and dichotomously coded to denote the presence of suicide-related visits (e.g., suicide ideation with no plan, suicide gesture, or plan). This study received IRB approval from the Albert Einstein Medical Center and the University of Pennsylvania. As a retrospective review, clinicians were not aware of the study at the time that the evaluation information and diagnostic and disposition decisions had been recorded. The data were subsequently de-identified and used for the current study.

Statistical Analysis

The analytical sample consisted of 1621 African American youth. Frequencies and percentages are reported for arrival status (voluntary or involuntary) and whether a commitment decision was upheld or dismissed among the involuntary users of PES. Cross-tabulations were used to examine characteristics between those who entered PES through involuntary or voluntary means. Demographic and social variables (gender, age, insurance status, and abuse history) and clinical variables (DSM diagnosis, substance abuse involvement, suicidality, and GAF scores) were entered stepwise into a logistic regression model to predict arrival status. Multivariate logistic regression was used to examine the factors associated with a civil commitment decision among the subset of the sample for which there was an official petition for civil commitment. Cross-tabulations were also used to examine characteristics between those who were committed to PES or had their case dismissed. In the multivariate model examining commitment decision, the independent influence of sociodemographic and clinical factors were cast as statistical controls. SPSS version 15 was used to perform the computations.

Results

Sample Characteristics

Table 1 indicates the characteristics of the sample and what typified their arrival status. There were more males (57%) than females visiting PES. Youth 13–17 years old had the highest number of visits (54%), followed by the 12 and under group (36%). Based on the prevalence of public insurance (75%), this was a fairly low income sample. Overall, the majority of sample youth arrived voluntarily (73%) to PES. Almost half (49%) were experiencing pervasive impairment (GAF scores 0–40), and the majority (54%) received a disruptive behavior disorder DSM diagnosis (e.g., conduct, impulsive, ADHD) at the time of their visit. The majority of PES users arrived during the evening shift--3 pm-11 pm (47.5%). Finally, the majority (48%) of youth who accessed PES were ultimately dispositioned to outpatient services.

Table 1.

Sample Characteristics of African American Youth Presenting at PES (N = 1621)

N (%)
Gender
 Male 928 57.2
 Female 693 42.8
Age Group
 12 yrs and under 583 36.0
 13–17 yrs 873 53.9
 18 and older 165 10.2
Medical Insurance
 No insurance 53 3.3
 Public insurance 1221 75.3
 Private insurance 347 21.4
Victim of Abuse Statusf
 Yes 247 15.2
 No 1365 84.2
DSM-Diagnosed Disorders
 Mood 607 37.4
 Behavior 882 54.4
 Other 132 8.1
Arrival Status
 Involuntary 426 26.4
 Voluntary 1188 73.6
Substance Abuse Involvement
 Drug/alcohol or both 188 11.6
 None 1433 88.4
Suicidality
 Yes 241 14.9
 No 1380 85.1
Global Assessment of Function
 Pervasive impairment (0–40) 798 49.2
 Serious impairment (41–60) 727 44.8
 Moderate to low impairment (61–90) 96 5.9
Arrival Time
 Morning shift (7 am–3 pm) 672 41.5
 Evening shift (3 pm–11 pm) 770 47.5
 Overnight shift (11 pm–7 am) 178 11.0
Disposition
 Inpatient 672 41.5
 Partial 142 8.8
 Outpatient 776 47.9

Percentages on some variables may not add up to 100% and the analytic sample may not total to 1621 because of missing values on selected variables.

Comparing voluntary and involuntary arrivals, Table 2 indicates that many of the risk characteristics (gender, age, insurance status, abuse victim, substance abuse, and arrival time) were proportionately similar. When examining arrival status more closely, however, a majority of involuntary patients arrived with pervasive impairment (72%; GAF scores 0–40), whereas a majority of voluntary patients arrived with serious impairment (52%; GAF scores 41–60) (χ2 = 123.17, df = 2, p < .001). More than twice as many youth who arrived voluntarily were dispositioned to outpatient services compared to involuntary arrivals (58% vs. 22%; χ2 = 208.8, df = 2, p < .001).

Table 2.

Social/Clinical Factors by Arrival Status (N = 162)

Voluntary [n = 1188] Involuntary [n = 426]

n (%) n (%)

Gender χ2 (1) = 5.13*
 Male 700 58.9 223 52.3
 Female 488 41.1 203 47.7
Age Group χ2 (2) = 47.6***
 12 yrs and under 486 40.9 95 22.3
 13–17 yrs 587 49.4 282 66.2
 18 and older 115 9.7 49 11.5
Medical Insurance χ2 (2) = 11.9**
 No insurance 30 2.5 23 5.4
 Public insurance 888 74.7 328 77.0
 Private insurance 270 22.7 75 17.6
Victim of Abuse Status χ2 (1) = 6.9**
 Yes 165 14.0 82 19.3
 No 1016 86.0 342 80.7
DSM-Diagnosed Disorders χ2 (2) = 0.26
 Mood 448 37.7 157 36.9
 Behavior 642 54.0 236 55.4
 Other 98 8.2.0 33 7.7
Substance Abuse Involvement χ2 (1) = 41.01***
 Drug/alcohol or both 102 8.6 86 20.2
 None 1086 91.4 340 79.8
Suicidality χ2 (1) = 0.14
 Yes 175 14.7 66 15.5
 No 1013 85.1 390 84.5
Global Assessment of Function χ2 (2) = 123.17***
 Pervasive impairment (0–40) 487 41.0 308 72.3
 Serious impairment (41–60) 617 51.9 106 24.9
 Moderate to low impairment (61–90) 84 7.1 12 2.8
Arrival Time χ2 (2) = 55.6***
 Morning shift (7 am–3 pm) 539 45.4 129 30.4
 Evening shift (3 pm–11 pm) 554 46.6 213 50.1
 Overnight shift (11 pm–7 am) 95 8.0 83 19.5
Disposition χ2 (2) = 208.8***
 Inpatient 368 31.6 301 72.2
 Partial 119 10.2 23 5.5
 Outpatient 679 58.2 93 22.3
*

p < .05

**

p < .01

***

p < .001

For some variables the percent total may not add up to 100% or the analytic sample may not total to 1621 because of missing values on selected variables.

Predictors of Voluntary Arrival

Significant predictors of voluntary arrival status included medical insurance, suicidality, DSM diagnosis (e.g., conduct, impulsive, ADHD), substance involvement, GAF scores, and time of day the youth arrived to PES (See Table 3). Youth with public insurance were less likely than youth with private insurance to voluntarily arrive at PES (OR = .52; CI 0.27, 0.99; p < .001). Youth classified as having suicidality had 1.7 times (OR = 1.71; CI 1.20, 2.44; p < .001) the adjusted odds of a voluntary admission compared with youth without suicidality. Youth with a DSM-diagnosed behavioral disorder were less likely (OR = 0.68; CI 0.52, 0.90; p < .001) to have a voluntary admission to PES than youth with a DSM-diagnosed mood disorder. Among youth with substance abuse histories, the odds of voluntary admission to PES were 2 times (OR = 2.25; CI 1.57, 3.22; p < .001) greater for youth with substance abuse involvement than those without substance abuse involvement. Both youth with serious impairment (GAF scores 41–60) (OR = 3.86; CI 2.92, 5.09; p < .001) and youth with moderate to low impairment (GAF scores 61–90) (OR = 5.17; CI 2.65, 10.11; p < .001) were more likely to enter PES through voluntary admission than youth with pervasive impairment (GAF scores 0–40). Finally, youth arriving at PES during the morning shift of 7 am-3 pm (OR = 2.41; CI 1.64, 3.54; p < .001) and the evening shift of 3pm-11pm (OR = 1.82; CI 1.27, 2.62; p < .001) had higher odds of arriving via voluntary admission than youth arriving during the overnight shift of 11 pm-7 am. Gender, age, and victim of abuse status were not associated with voluntary admission to PES. The Nagel Kirke R2 for this model was .20, and the overall percentage of cases correctly classified by the model (voluntary arrivals to PES) was 76%.

Table 3.

Social/Clinical Factors Predicting Voluntary Arrival (n = 1188)

Characteristics OR 95% CI
Gender
 Male 1.22 0.94, 1.58
 Female 1.00
Age Group
 12 yrs and under 1.56 0.97, 2.52
 13–17 yrs 0.75 0.50, 1.23
 18 and older 1.00
Medical Insurance
 No insurance .88 0.64, 1.21
 Public insurance .52 0.27, 0.99
 Private insurance 1.00
Suicidality
 Yes 1.71 1.20, 2.44
 No 1.00
Victim of Abuse Status
 Yes 0.79 0.57, 1.09
 No 1.00 --
DSM-Diagnosed Disorders
 Mood 1.00
 Behavior 0.68 0.52, 0.90
 Other 1.04 0.64, 1.68
Substance Abuse Involvement
 Yes 2.25 1.57, 3.22
 No 1.00
Global Assessment of Function
 Pervasive impairment (0–40) 1.00
 Serious impairment (41–60) 3.86 2.92, 5.09
 Moderate to low impairment (61–90) 5.17 2.65, 10.11
Arrival Time
 Morning shift (7 am–3 pm) 2.41 1.64, 3.54
 Evening shift (3 pm–11 pm) 1.82 1.27, 2.62
 Overnight shift (11 pm–7 am) 1.0

NagelKirke R2 0.20
Chi^2 df Sig.
Omnibus Test of Model/ 241.19 14 0.001
Hosmer-Leslow 2.12 8 0.98

Involuntary/Voluntary Commitment Characteristics

The majority of youth facing a commitment decision (i.e., involuntary commitment, voluntary commitment, or case dismissal) were between the ages of 13–17 (χ2 = 10.63, df = 4, p < .05) (See Table 4.). Additionally, the majority of youth were diagnosed with a behavioral disorder (χ2 = 11.56, df = 4, p < .05) in each of the commitment domains; however, most appeared not to have a substance abuse issue (χ2 = 13.38, df = 2, p < .001) or suicidality (χ2 = 19.53, df = 1, p < .001). Based on GAF scores, the majority of youth who were involuntarily committed (95%) experienced pervasive impairment (i.e., GAF scores in the range of 0–40), whereas the majority of youth who were voluntarily committed (51%) or had their case dismissed (75%) experienced serious impairment (i.e., GAF scores in the range of 41–60). Finally, the majority of the involuntary and voluntary committed youth arrived to PES during the evening shift (3pm–11pm), whereas the majority of the youth whose case was dismissed arrived during the morning shift (7am–3pm) (χ2 = 33.43, df = 4, p < .001).

Table 4.

Social/Clinical Factors by Type of Commitment Decision (N = 502)

Involuntary Committed [n = 300] Voluntary Committed [n = 83] Case Dismissal [n = 123]

n (%) n (%) N (%)
Gender χ2 (2) = .43
 Male 158 52.7 47 56.6 65 52.8
 Female 142 47.3 36 43.4 58 47.2
Age Group χ2 (4) = 10.63*
 12 yrs and under 59 19.7 29 34.9 35 28.5
 13–17 yrs 205 68.3 43 51.8 74 60.2
 18 and older 36 12.0 11 13.3 14 11.4
Medical Insurance χ2 (4)= 5.97
 No insurance 19 6.3 2 2.4 4 3.3
 Public insurance 227 75.7 59 71.1 97 78.9
 Private insurance 54 18.0 22 26.5 22 17.9
Victim of Abuse Status χ2 (2) = 0.80
 Yes 59 19.7 17 20.7 20 16.4
 No 240 80.3 65 79.3 102 86.6
DSM-Diagnosed Disorders χ2 (4) = 11.56*
 Mood 119 39.7 36 43.4 36 29.3
 Behavior 153 51.0 43 51.8 82 66.7
 Other 28 9.3.0 4 4.8 5 4.1
Substance Abuse Involvement χ2 (2) = 13.38***
 Drug/alcohol or both 61 20.3 3 3.6 25 20.3
 None 239 79.7 80 96.4 98 79.7
Suicidality χ2 (1) = 19.53***
 Yes 61 20.3 13 15.7 4 3.3
 No 239 79.7 70 84.3 119 96.7
Global Assessment of Function χ2 (4) = 288.7***
 Pervasive impairment (0–40) 287 95.7 35 42.2 18 14.6
 Serious impairment (41–60) 13 4.3 43 51.8 93 75.6
 Moderate to low impairment (61–90) 0 0 5 6.0 12 9.8
Arrival Time χ2 (4) = 33.43***
 Morning shift (7 am–3 pm) 71 23.7 23 27.7 59 48.0
 Evening shift (3 pm–11 pm) 164 54.8 54 65.1 49 39.8
 Overnight shift (11 pm–7 am) 64 21.4 6 7.2 15 12.2
*

p < .05

**

p < .01

***

p < .001

For some variables the percent total may not add up to 100% or the analytic sample may not total 506 because of missing values on selected variables.

Predictors of Involuntary/Voluntary Commitment (Commitment Decision Upheld)

Two multivariate logistic regression models examined the social/clinical factors predicting two outcomes: an involuntary commitment and a voluntary commitment (See Table 5). First, the model for an involuntary commitment fit the data well (χ2 = 332.60, df = 12; p < .001). In particular, only two factors were significant predictors of the decision to uphold an involuntary commitment: age and GAF scores. That is, for every year of age increase (e.g., 13 to 14 years old, 14 to 15 years old), a youth had odds 1.19 times (CI 1.07, 1.32; p < .001) greater for an involuntary commitment than youth of a younger age. In terms of GAF scores, the higher the GAF score (OR = 0.83; CI 0.80, 0.85; p < .001), the less likely a youth was involuntarily committed in terms of their case disposition. The NagelKirke R2 for this model was 0.65, and the overall percentage of cases correctly classified by the model (i.e., whether a commitment decision was upheld among those who involuntarily arrived) was 59.4%.

Table 5.

Social/Clinical Factors Predicting Commitment Decisions (n = 502) a,b,c

Involuntary Committed Voluntary Committed

Characteristics OR 95% CI OR 95% CI
Age 1.19 1.07, 1.32 0.89 0.81, 0.98
Gender
 Male 0.89 0.49, 1.61 1.14 0.65, 1.98
 Female 1.00 1.00
Global Assessment of Function 0.83 0.80, 0.85 1.05 1.03, 1.07
Medical Insurance
 No insurance 3.53 0.83, 15.02 0.39 0.81, 1.91
 Public insurance 1.27 0.59, 2.73 0.49 0.25, 0.94
 Private insurance 1.00 1.00
Suicidality
 Yes 1.00 1.00
 No 1.24 0.56, 2.73 0.65 0.30, 1.39
Victim of Abuse Status
 Yes 1.00 0.51, 1.98 1.22 0.64, 2.33
 No 1.00 -- 1.00 --
DSM-Diagnosed Disorders
 Mood 1.00
 Behavior 1.21 0.65, 2.26 0.61 0.34, 1.09
 Other 1.36 0.36, 5.18 0.64 0.19, 2.12
Substance Abuse Involvement
 Yes 1.50 0.65, 3.45 0.17 0.05, 0.59
 No 1.00 1.00
Arrival Time
 Morning shift (7 am–3 pm) 0.44 0.18, 1.07 1.69 0.61, 4.64
 Evening shift (3 pm–11 pm) 0.67 0.30, 1.52 2.89 1.15, 7.27
 Overnight shift (11 pm–7 am) 1.0 1.00

NagelKirke R2 0.65 .21
Chi^2 df Sig. Chi^2 df Sig.
Omnibus Test of Model/ 332.60 12 0.001 64.50 12 .001
Hosmer-Leslow 27.79 8 0.001 6.90 8 .547
a

Four subjects were dropped from the analyses because of missing values on selected variables.

b

In each model, the commitment status, involuntary vs. voluntary, is being compared to case dismissal; case dismissal was labeled 0 in both logistic regression models.

c

GAF scores were entered as continuous variables, unlike in previous models whereby GAF scores were entered as trichotomous predictor variables.

The model examining the voluntary committed status of youth also fit the data well (χ2 = 64.50; df = 12; p < .001). In this model, age, GAF scores, insurance status, substance abuse involvement, and arrival time were significant predictors of a voluntary commitment. In particular, for every year of age increase (e.g. 13 to 14 years old, 14 to 15 years old, etc.) youth were less likely to have a voluntary commitment (OR = 0.89; CI 0.81, 0.98; p < .001). Youth with public insurance were less likely than youth with private insurance to have a voluntary commitment (OR = 0.49; CI 0.25, 0.94; p < .001). Youth with substance abuse involvement were also less likely to have a voluntary commitment than youth with no substance abuse involvement (OR = 0.17; CI 0.05, 0.59; p < .001). Conversely, youth with higher GAF scores (OR = 1.05; CI 1.03, 1.07; p < .001) and youth who arrived during the evening shift (3pm–11pm) (OR = 2.89; CI 1.15, 7.27; p < .001) were more likely to have a voluntary commitment. The NagelKirke R2 for this model was .21, and the overall percentage of cases correctly classified by the model (whether a commitment decision was upheld among those who voluntarily arrived) was 83.7%.

Discussion

Findings in this study describe key demographic, social, and clinical characteristics of African American youth who present at a PES center. In particular, descriptive findings indicate that the majority of African American youth (73%) voluntarily arrived to PES. This finding is consistent with general findings in the pediatric emergency care literature that suggest that emergency departments and PES are perhaps pathways or gateway providers into the mental health system. Findings in this study also indicate that non-pervasively impaired youth (GAF scores 41–90) had odds 3 to 5 times higher to voluntarily arrive to PES (in contrast to GAF scores among the involuntary arrivals). Given the odds that a voluntary arrival increases as the level of impairment decreases, perhaps our findings suggest that youth in the sample were coming to PES for non-urgent matters.

Prior mental health services research studies and other factors (i.e., the availability of community-based services) help contextualize our findings. For example, dosReis found that among urban, African American children with ADHD and their families, community mental health service use may be delayed if caregivers are initially reluctant to view behaviors as problematic or they believe that their child's behavior does not warrant professional mental health treatment. Elsewhere, negative perceptions of community-based mental health treatment (and previous negative experiences) have been found to reduce the likelihood of seeking professional treatment among African American adolescents in community settings. Given the developmental level of this sample (i.e., adolescence), perhaps family factors (e.g. reluctance to identify symptoms as mental health problems needing professional treatment, or attempts by the family to resolve child mental health problems within the family) were influential in facilitating when and how treatment was sought. Goldstein and Howoritz found in their research on psychiatric presentations to emergency departments that formal care was sought as a last resort when child mental health concerns became too problematic for the family. Perceptions of the adolescents' danger to self or to others may also have influenced when, where, and how treatment was sought for youth in this sample. Finally, it could be the case that the lack of community-based mental health services was associated with the delayed access to treatment for this sample. That is, high use of PES may reflect a lack of community-based mental health service use or accessible and effective outpatient services. In the absence of ongoing, low-intensity treatment youth may have been more likely to appear at PES when untreated symptoms worsen or manifest in a crisis situation, reflecting in a crisis-oriented patterns of care for non-urgent reasons. From these data, however, we cannot determine what community-based services were available. Elsewhere, studies examining emergency room visits among adults suggest that emergency rooms are defacto service providers when low-intensity, community-based services are unavailable, particularly among low-income populations. More research is needed to determine whether the availability of community-based services is indeed associated with increased/decreased use of PES among adolescents.

In the logistic regression model examining predictors of arrival to PES, insurance status (public insurance less likely), suicidality, DSM-diagnosed disorder (behavioral disorder less likely), substance abuse involvement, GAF scores of 41 and higher, and arrival time (morning and evening shift) were associated with a voluntary arrival. The decreased likelihood of African American youth with DSM-diagnosed behavioral disorders to arrive voluntarily to PES suggests that externalizing behavioral problems (rather than mood or internalizing behaviors) exhibited by these youth met the standards of danger to self or others, warranting a 302 code. Janofsky and Tamburello found, among an adult sample referred to psychiatric services as a result of an emergency petition, that patients who could have been arrested based on behavioral expressions of danger to others or property were diverted from other sectors (e.g., criminal justice). Perhaps for a population (i.e., African American youth) more often likely to be overrepresented in the juvenile justice system, findings in this study might be a positive indication that mental health treatment is being sought in lieu of or in addition to other service sectors. Without information concerning the referral source, however, it is difficult to make a definitive conclusion.

Relative to the extant literature, there were two interesting findings regarding suicidality among youth in our sample. First, it is assumed that adolescents presenting for suicide-related problems (ideation, attempts) are more likely to use psychiatric emergency care. A majority of youth presenting at the PES in our study, however, had problems unrelated to a history of suicidal behavior or mood disorder, rather behavioral problems. Although suicidality is an important factor associated with PES use, previous research has consistently found that behavioral problems may be a stronger driver of PES use among samples of urban adolescents.

Second, while we were not able to determine whether suicidality increased adolescents' use of PES, we were able to determine that it does differentiate whether they arrived voluntarily or involuntarily. Given the increased rates of suicide-related problems (ideation, attempts) among African American adolescents, and the growing recognition of suicide as a significant public health issue for this group, - this finding suggests that more specific investigation of the suicide risk assessment skills of collaterals (e.g. caregivers; school, social welfare, or juvenile justice staff) making these referrals is warranted. It is plausible, however, that the mere mention of suicidality or indicators of an increased risk of self harm among adolescents in our sample to their collaterals is the reason why more presented voluntarily to PES. Additionally, suicidality did differentiate the arrival status among adolescents facing a commitment decision; that is, adolescents referred for hospitalization with suicidality were significantly more likely to have their commitment decision upheld than dismissed.

Age and GAF score accounted for 65% of the variance explained among predictors in the model for involuntary commitment to PES. That is, for every year of age increase, the likelihood of involuntarily hospitalization increased. This finding suggests that PES professionals may be sensitive to the issue of age. For example, as youth with severe mental health disorders get older, they are likely to contend increased risks (e.g. substance abuse, suicidality) indicating a need for mental health treatment. As a result, PES professionals might be less likely to involuntarily commit younger youth who might be less risk-exposed or perceived to be less dangerous to themselves or someone else than older adolescents. Moreover, findings that higher GAF scores decreased the likelihood of involuntary commitment were not surprising given previous studies that also indicate level of impairment as a primary reason for inpatient hospitalization. This finding might also be related to the fact that clinicians completing the GAF assessments were also likely to be making the decision regarding commitment. That GAF scores were such a strong predictor of lower likelihood of involuntary commitment may also explain why other mental health need indicators (e.g., suicidality, DSM disorders, and substance abuse involvement) did not reach significance in the model. It is believed that GAF scores essentially cancelled out the potential significant associations of the other independent variables. Or, as noted by Lincoln, it is possible that clinicians ultimately rule in favor of hospitalization because the patients had “high need” for services and it is the only way the clinician can link the adolescent to services, even though they might not have met the legal criteria for hospitalization.

Significant predictors in the voluntary commitment model included age (older age less likely), higher GAF scores, insurance status (public insurance less likely), substance abuse involvement (less likely), and arrival time (evening shift). These factors accounted for 21% of the variance explained in the model. Time of arrival seems to matter regarding a commitment decision. For example, the majority of voluntarily committed youth who arrived during the evening shift (3pm–11pm) had their commitment decision upheld. It might be the case that more intensive symptom expression occurred during the evening shift. Alternatively, it could be the case that youth were brought into PES because other community-based treatment services were not available. Segal and colleagues noted that in some cases emergency psychiatric facilities have less staff coverage during the evening shift. With fewer alternatives available for treatment, the PES, or Crisis Response Center, in this study might have been the only viable treatment option during evening hours. From the data, however, it cannot be determined whether this is truly the case. This finding warrants further investigation and has implications for the need to perhaps make more community-based services available during evening hours.

Limitations

The results should be interpreted with the following limitations in mind. First, this study relied exclusively on data retrieved from chart reviews rather than data gathered from semi-structured interviews or well supported reliable and valid measures. As has been discussed elsewhere much of the research on psychiatric emergency visits is based on retrospective chart reviews. Although the data at this PES are routinely collected and are a vital part of the accounting system, the reliability of the data collected was not established, and as such some data may have been inconsistently gathered. Additionally, the data analyzed herein was from visits seven to eight years ago and there may have been procedural, social and demographic changes since then. These limitations and the cross-sectional nature of the study reflects the need for future research in this area via more standardized methods that include reliability checks of data gathered from chart reviews, and data concerning the number of visits to indicate whether the findings observed in this study show consistent patterns over time.

Second, we were not able to explore the nature of the relationship between youth and the individual(s) responsible for bringing them into PES. Knowledge of the referral source provides greater context concerning the nature and course of illness and how the case might be dispositioned. This information would also be important in determining the extent of the social network's influence on help-seeking behaviors and may have important service use implications (e.g., treatment engagement, treatment adherence), especially considering the majority of youth in this study were voluntarily admitted.

Finally, a third set of limitations relate to the background characteristics of the sample. For example, referrals to PES among the sample may have been prompted by the perceived dangerousness of participants; however, there was no measure of this construct. Despite the large sample size, our findings likely represent characteristics of African American youth who are first time visitors to PES, rather than representative of characteristics of youth who use PES.

This study provides further support for the need to examine influences on help-seeking behaviors and formal service use in the context of community services for this population. In particular, further research should be conducted to identify the influence of families and gateway providers (e.g. juvenile corrections, child welfare) on entrée into PES among African American youth with serious psychiatric needs. For example, future research should examine both the availability of community-based treatment and the extent to which family members influence formal mental health help-seeking among low-income, African American youth with behavioral/mood disorders, particularly given the fact that fifty-eight percent of youth in the sample were ultimately dispositioned to community-based, outpatient treatment. Future research may also include qualitative methods to better understand the role of clinical and social factors in arrival status and hospitalization decisions. This research might explore caregivers and youths' perceptual barriers and other sociocultural factors that hinder service use earlier in the course of illness, particularly given our findings that the majority of youth arrived voluntarily to PES for non-urgent matters. Finally, PES practitioners might make better connections to juvenile justice and child welfare providers regarding post-discharge planning and coordination of community-based mental health treatment for African American adolescents.

Acknowledgments

This research was supported in part by funds from the National Institute of Mental Health, U.S. Department of Health and Human Resources awarded (K01-MH65499) to Dr. Joe

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry. 2003;60:837–844. doi: 10.1001/archpsyc.60.8.837. [DOI] [PubMed] [Google Scholar]
  • 2.Cooley-Quille M, Boyd RC, Frantz E, Walsh J. Emotional and behavioral impact of exposure to community violence in inner-city adolescents. J Clin Child Psychol. 2001;30:199–206. doi: 10.1207/S15374424JCCP3002_7. [DOI] [PubMed] [Google Scholar]
  • 3.dosReis S, Mychailyszyn MP, Myers M, Riley AW. Coming to terms with ADHD: How urban african-american families come to seek care for their children. Psychiatr Serv. 2007;58:636–641. doi: 10.1176/ps.2007.58.5.636. [DOI] [PubMed] [Google Scholar]
  • 4.Sadka S. Psychiatric emergencies in children and adolescents. New Directions in Mental Health Services. 1995;67:65–65. 74. doi: 10.1002/yd.23319650309. [DOI] [PubMed] [Google Scholar]
  • 5.Starling J, Bridgland K, Rose D. Psychiatric emergencies in children and adolescents: An emergency department audit. Australasian Psychiatry. 2006;14:403–407. doi: 10.1080/j.1440-1665.2006.02313.x. [DOI] [PubMed] [Google Scholar]
  • 6.Muroff J, Edelsohn GA, Joe S, Ford BC. The role of race in diagnostic and disposition decision making in a pediatric psychiatric emergency service. General Hospital Psychiatry. doi: 10.1016/j.genhosppsych.2008.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Snowden LR, Masland MC, Fawley K, Wallace N. Ethnic differences in children's entry into public mental health care via emergency mental health services. J Child Fam Stud. 2009;18:512–519. doi: 10.1007/s10826-008-9253-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Swanson J, Swartz M, Van Dorn R, Monahan J, McGuire T, Steadman H, et al. Racial disparities in involuntary outpatient commitment: Are they real? Health Affairs. 2009:816–826. doi: 10.1377/hlthaff.28.3.816. [DOI] [PubMed] [Google Scholar]
  • 9.Burns BJ, Phillips SD, Wagner HR, et al. Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child & Adolescent Psychiatry. 2004;43:960–970. doi: 10.1097/01.chi.0000127590.95585.65. [DOI] [PubMed] [Google Scholar]
  • 10.Pumariega AJ, Atkins DL, Rogers K, et al. Mental health and incarcerated youth. II: Service utilization. Journal of Child & Family Studies. 1999;8:205–215. [Google Scholar]
  • 11.Spurrell M, Hatfield B, Perry A. Characteristics of patients presenting for emergency psychiatric assessment at an english hospital. Psychiatr Serv. 2003;54:240–245. doi: 10.1176/appi.ps.54.2.240. [DOI] [PubMed] [Google Scholar]
  • 12.Lincoln A. Psychiatric emergency room decision-making, social control and the `undeserving sick'. Sociol Health Illn. 2006;28:54–75. doi: 10.1111/j.1467-9566.2006.00482.x. [DOI] [PubMed] [Google Scholar]
  • 13.Strakowski SM, Lonczak HS, Sax KW, West SA, Crist A, Mehta R, Thienhaus OJ. The effects of race on diagnosis and disposition from a psychiatric emergency service. The Journal Of Clinical Psychiatry. 1995;56:101. [PubMed] [Google Scholar]
  • 14.Way BB, Banks S. Clinical factors related to admission and release decisions in psychiatric emergency services. Psychiatr Serv. 2001;52:214–218. doi: 10.1176/appi.ps.52.2.214. [DOI] [PubMed] [Google Scholar]
  • 15.George L, Durbin J, Sheldon T, Goering P. Patient and contextual factors related to the decision to hospitalize patients from emergency psychiatric services. Psychiatr Serv. 2002;53:1586–1591. doi: 10.1176/appi.ps.53.12.1586. [DOI] [PubMed] [Google Scholar]
  • 16.Edelson GA, Braitman LE, Rabinovich H, sheves P, Melendez A. Predictors of urgency in a pediatric psychiatric emergency service. Journal of the American Academy of Child & Adolescent Psychiatry. 2003;42:1197–1202. doi: 10.1097/00004583-200310000-00010. [DOI] [PubMed] [Google Scholar]
  • 17.Evans ME, Boothroyd RA. A comparison of youth referred to psychiatric emergency services: Police versus other sources. The Journal Of The American Academy Of Psychiatry And The Law. 2002 [PubMed] [Google Scholar]
  • 18.Frosch E, McCulloch J, Yoon Y, Dosreis S. Pediatric emergency consultations: Prior mental health service use in suicide attempters. J Behav Health Serv Res. 2009 doi: 10.1007/s11414-009-9192-0. [DOI] [PubMed] [Google Scholar]
  • 19.Kaltiala-Heino R, Fröjd S. Severe mental disorder as a basic commitment criterion for minors. International Journal of Law & Psychiatry. 2007;30:81–94. doi: 10.1016/j.ijlp.2006.04.002. [DOI] [PubMed] [Google Scholar]
  • 20.Nicholson RA. Correlates of commitment status in psychiatric patients. Psychol Bull. 1986;100:241–250. [PubMed] [Google Scholar]
  • 21.Lyons JS, Stutesman J, Neme J, Vessey JT, O'Mahoney MT, Camper HJ. Predicting psychiatric emergency admissions and hospital outcome. Med Care. 1997;35:792–800. doi: 10.1097/00005650-199708000-00005. [DOI] [PubMed] [Google Scholar]
  • 22.Pescosolido BA, Fettes DL, Martin JK, Monahan J, McLeod JD. Perceived dangerousness of children with mental health problems and support for coerced treatment. Psychiatr Serv. 2007;58:619–625. doi: 10.1176/ps.2007.58.5.619. [DOI] [PubMed] [Google Scholar]
  • 23.Christy A, Kutash K, Stiles P. Short term involuntary psychiatric examination of children in florida. Administration & Policy in Mental Health & Mental Health Services Research. 2006;33:578–584. doi: 10.1007/s10488-006-0064-0. [DOI] [PubMed] [Google Scholar]
  • 24.Moos RH, McCoy L. Global assessment of functioning (GAF) ratings: Determinants and role as predictors of one-year. J Clin Psychol. 2000;56:449–461. doi: 10.1002/(sici)1097-4679(200004)56:4<449::aid-jclp1>3.0.co;2-8. [DOI] [PubMed] [Google Scholar]
  • 25.Moos RH, Nichol AC, Moos BS. Global assessment of functioning ratings and the allocation and outcomes of mental health services. Psychiatr Serv. 2002;53:730–737. doi: 10.1176/appi.ps.53.6.730. [DOI] [PubMed] [Google Scholar]
  • 26.Hillard JR, Slomowitz M, Levi LS. A retrospective student of adolescents' visits to a general hospital psychiatric emergency service. American Journal of Psychiatry. 1987;144 doi: 10.1176/ajp.144.4.432. [DOI] [PubMed] [Google Scholar]
  • 27.Hillard JR. The past and future of psychiatric emergency services in the U.S. Hosp Community Psychiatry. 1994;45:541–543. doi: 10.1176/ps.45.6.541. [DOI] [PubMed] [Google Scholar]
  • 28.Stewart C, Spicer M, Babl FE. Caring for adolescents with mental health problems: Challenges in the emergency department. Journal of Paediatrics & Child Health. 2006;42:726–730. doi: 10.1111/j.1440-1754.2006.00959.x. [DOI] [PubMed] [Google Scholar]
  • 29.Sills MR, Bland SD. Summary statistics for pediatric psychiatric visits to US emergency departments, 1993–1999. Pediatrics. 2002;110:e40–e40. doi: 10.1542/peds.110.4.e40. [DOI] [PubMed] [Google Scholar]
  • 30.Soto EC, Frederickson AM, Trivedi H, et al. Frequency and correlates of inappropriate pediatric psychiatric emergency room visits. J Clin Psychiatry. 2009;70:1164–1177. doi: 10.4088/JCP.08m04839. [DOI] [PubMed] [Google Scholar]
  • 31.Lindsey MA, Korr WS, Broitman M, Bone L, Green A, Leaf PJ. Help-seeking behaviors and depression among african american adolescent boys. Soc Work. 2006;51:49. doi: 10.1093/sw/51.1.49. [DOI] [PubMed] [Google Scholar]
  • 32.Scott LDJ, Munson MR, McMillen JC, Snowden LR. Predisposition to seek mental health care among black males transitioning from foster care. Children and Youth Services Review. 2007;29:870–882. doi: 10.1016/j.childyouth.2007.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Goldstein AB, Horwitz SH. Child and adolescent psychiatric emergencies in nonsuicide-specific samples. Pediatric Emergency Care. 2006:379. doi: 10.1097/01.pec.0000216565.26123.34. [DOI] [PubMed] [Google Scholar]
  • 34.Pescosolido BA, Fettes DL, Martin JK, Monahan J, McLeod JD. Perceived dangerousness of children with mental health problems and support for coerced treatment. Psychiatric Services. 2007;58:619–625. doi: 10.1176/ps.2007.58.5.619. [DOI] [PubMed] [Google Scholar]
  • 35.Gibson RW. Utilization and costs of psychiatric care: Key issues in the coming debate on national health insurance. Am J Orthopsychiatry. 1973;43:628–631. doi: 10.1111/j.1939-0025.1973.tb00832.x. [DOI] [PubMed] [Google Scholar]
  • 36.Neighbors HW. Ambulatory medical care among adult black Americans: The hospital emergency room. J Natl Med Assoc. 1986;78:275–282. [PMC free article] [PubMed] [Google Scholar]
  • 37.Janofsky JS, Tamurello AC. Diversion to the mental health system: Emergency psychiatric evaluations. The Journal of American Academy of Psychiatry and the Law. 2006;34:284. [PubMed] [Google Scholar]
  • 38.Teplin LA, Abram KM, McClelland GM, Washburn JJ, Pikus AK. Detecting mental disorder in juvenile detainees: Who receives services. Am J Public Health. 2005;95:1773–1780. doi: 10.2105/AJPH.2005.067819. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Joe S, Baser RS, Neighbors HW, Caldwell CH, Jackson JS. 12-month and lifetime prevalence of suicide attempts among black adolescents in the national survey of american life. Journal of the American Academy of Child & Adolescent Psychiatry. 2009;48:271–282. doi: 10.1097/CHI.0b013e318195bccf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Joe S. Explaining changes in the patterns of black suicide in the united states from 1981 to 2002: An age, cohort, and period analysis. J Black Psychol. 2006;32:262–284. doi: 10.1177/0095798406290465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Centers for Disease Control and Prevention (CDC) Suicide among black youths--united states, 1980–1995. MMWR Morb Mortal Wkly Rep. 1998;47:193–196. [PubMed] [Google Scholar]
  • 42.Segal SP, Laurie TA, Segal MJ. Factors in the use of coercive retention in civil commitment evaluations in psychiatric emergency services. Psychiatr Serv. 2001;52:514–520. doi: 10.1176/appi.ps.52.4.514. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES