Development and Validation of the GAIN Short Screener (GSS) for Internalizing, Externalizing and Substance Use Disorders and Crime/Violence Problems Among Adolescents and Adults

Abstract

The Global Appraisal of Individual Needs (GAIN)¹ is a 1–2 hour standardized biopsychosocial that integrates clinical and research assessment for people presenting to substance abuse treatment. The GAIN – Short Screener (GSS) is 3–5 minute screener to quickly identify those who would have a disorder based on the full 60–120 minute GAIN and triage the problem and kind of intervention they are likely to need along four dimensions (internalizing disorders, externalizing disorders, substance disorders, and crime/violence). Data were collected from 6,177 adolescents and 1,805 adults as part of 77 studies in three dozen locations around the United States that used the GAIN. For both adolescents and adults the 20-item total disorder screener (TDScr) and its four 5-item sub-screeners (internalizing disorders, externalizing disorders, substance disorders, and crime/violence) has good internal consistency (alpha of .96 on total screener), is highly correlated (r = .84 to .94) with the 123-item longer scales in the full GAIN. The GSS also does well in terms of its receiver operator characteristics (90% or more under the curve in all analyses) and has clinical decision-making cut points with excellent sensitivity (90% or more) for identifying people with a disorder and excellent specificity (92% or more) for correctly ruling out people who did not have a disorder. The GSS has good potential as an efficient screener for identifying people with co-occurring disorders across multiple systems and routing them to the right services and more detailed assessments.

Among people in the community with psychiatric and substance disorders, multiple co-occurring diagnoses are the norm.^2–7 Co-occurrence of psychiatric disorders, substance use disorders, and crime/violence are an even more prominent characteristic of those entering the addiction treatment, mental health services, and the criminal/juvenile justice system.^6,8–16 Individuals with multiple co-occurring problems are more likely to experience problems with treatment and medication adherence, shorter lengths of stay, administrative discharges, functional status, community adjustment, quality of life, and worse outcomes following treatment for their substance use disorder.^17–24

Unfortunately, it is estimated that one- to two-thirds of people with serious disorders do not access treatment and most of those who are un- or under-treated are young males with poor education.²⁵ Among people with substance use disorders, less than 1 in 5 adults and 1 in 10 adolescents enter treatment.²⁶ While research suggests that 70 to 80% of people entering substance abuse treatment have one or more co-occurring psychiatric disorders only 16% of adults and 28% of adolescents have a co-occurring disorder documented in their intake assessments.^27–28 This has led to calls to introduce standardized assessment of co-occurring disorders among people presenting to mental health, substance abuse, welfare and justice systems.^29–32 The problem is that such assessments often take hours of client and staff time that press limited public resources. Thus, there is a need to develop and evaluate a short initial screener to (1) identify who has a disorder and rule out those who do not, (2) make an initial approximation of the type of problems and severity, and (3) guide referral to further assessment and treatment.

There are hundreds of diagnoses (each with multiple specifiers) identified in the Diagnostic and Statistical Manual Version IV text revised (DSM-IV-TR).³³ Studies in general population^34–37 and clinical samples,^38–39 however, demonstrate the existence of three primary dimensions along which the prevalence of the more common mental disorders vary: (1) internalizing disorders (e.g., depression, anxiety, somatic disorder, traumatic distress, suicide), (2) externalizing disorders (e.g., attention deficit, hyperactivity, conduct and other impulse control disorders), and (3) substance use disorders (e.g., abuse, dependence, other substance induced health or psychiatric problems). While externalizing disorders typically onset in childhood, there is increasing recognition that they extend well into adulthood and are related to substance use severity.^5,39–43 Among substance users and people in the juvenile/criminal justice system there is often a fourth dimension, crime and violence (e.g., interpersonal violence, drug related crime, property crime, interpersonal crime).^38,44–46 While each disorder/problem is unique, those that vary along the same dimensions often share more etiology (e.g., age of onset, course), consequence, treatment, and outcomes with each other than with disorders in other dimensions. Methodologically, grouping symptoms and disorders into these four statistical dimensions increases the efficiency and accuracy of a screener.

The full Global Appraisal of Individual Needs (GAIN)¹ is a standardized biopsychosocial that integrates clinical and research assessment for people presenting to substance abuse or other behavioral health treatment. It asks about symptoms from DSM-IV-TR that can be used to generate dimensional symptom count measures or categorical diagnostic impressions of specific disorders in the four main dimensions of interest here (i.e., internalizing, externalizing, substance, and crime/violence). While well received (it is currently in use by over 400 agencies across the U.S., Canada, and Mexico), it typically takes 2–3 months of training and feedback to get a staff person certified on GAIN administration and then takes 90 to 120 minutes per patient/staff person to actually administer. For many situations, this involves more resources, more control of programs/staff than is feasible over a large spread out system, and is substantively too extensive an assessment for a client coming in on a generic referral. Even if we pulled out just the 123 main diagnostic questions from the GAIN, this would still be 8 pages long and take 20–30 minutes. This is often still too long for use as a screener in settings like school, workplace, internet based health risk assessments, welfare, or justice systems where it may only be one of several components, there is limited time or limited staff resources. Thus, there was a need to develop a GAIN-Short Screener (GSS) that could be (a) easily trained, (b) used in 5 minutes or less to identify people who have a disorder and rule out people who do not, and (c) provide guidance for referral to further assessment and treatment. Consistent with the full GAIN, the GSS is designed to (a) be valid for both adolescent and adult populations, (b) provide measures of severity overall and in the four main dimensions of emotional/behavioral problems (internalizing, externalizing, substance, crime/violence), and (c) triage these dimensions to provide guides to support clinical decision making about diagnosis and treatment needs.

The aims of this paper are to (1) describe the development of the 20-item GSS (2) validate its use as a dimensional measure and (3) identify the optimal clinical cut points for clinical decision making about the likelihood of a disorder or crime/violence problem. For each aim we examine the GSS’s use for both adolescents and adults, as a measure of total severity, and as a measure within each of the four main dimensions of emotional/behavioral problems (internalizing, externalizing, substance, crime/violence) measured by the full GAIN.

METHODS

Data Source

The data are from the 77 studies in three dozen locations around the United States that used the GAIN.¹ Over two thirds of these studies were conducted by independent investigators. They were funded by a wide range of organizations (e.g., the Center for Substance Abuse Treatment, National Institute on Alcohol Abuse and Alcoholism, National Institute on Drug Abuse, Robert Wood Johnson Foundation and Interventions Foundation) and conducted in a variety of institutional settings, including across adolescent and adult levels of care, student assistance programs, criminal and juvenile justice agencies, mental health agencies, and child protective service and family service agencies. All data were collected as part of general clinical practice or specific research studies under their respective voluntary consent procedures and were subsequently de-identified.

Sample Characteristics

The pooled data are based on GAIN interviews with 6,177 adolescents (age 10–17) and 1,805 adults (age 18–69). Table 1 shows the demographic and clinical characteristics of the sample by adolescent (average age 16) and adults (average age 34). The adolescents were significantly more likely than adults to be male and involved in the justice system, but less likely to be African American or entering residential treatment. Clinically, adolescents had slightly more total past-year symptoms on the GAIN’s General Individual Severity Scale and had a different pattern of symptoms as compared to adults; on average, adolescents reported significantly fewer past-year symptoms on the Internal Mental Distress Scale (internalizing disorder symptoms) and Substance Problem Scale (abuse/dependence/induced symptoms) and significantly more past-year symptoms than adults on the Behavior Complexity Scale (externalizing disorder symptoms) and Crime/Violence Scale. The bottom half of the table shows the difference in the patterns using the items to create diagnostic/categorical groups. Adolescents were significantly less likely than adults to meet criteria for internalizing disorders and substance dependence; adolescents were more likely to meet criteria for substance abuse, externalizing disorders, and crime/violence problems.

TABLE 1.

Sample Characteristics Based on Full GAIN Assessment by Age^a

Characteristic	Adolescent (n = 6,177)	Adult (n = 1,805)
Age – Mean (SD)	16 (1.5)	34 (9.1)
Male (%)	73	49
Race-Ethnicity: African American (%)	17	52
Caucasian (%)	48	40
Hispanic (%)	14	3
Mixed/Other (%)	21	5
Current Involvement in Juvenile/Criminal Justice System	71	52
Residential Treatment	34	56
General Individual Severity Scale (GISS) – Mean (SD)b	37 (22.9)	35 (23.6)
Internal Mental Distress Scale (IMDS) – Mean (SD)	10 (9.1)	13 (11.1)
Behavior Complexity Scale (BCS) – Mean (SD)	12 (8.3)	8 (8.6)
Substance Problem Scale (SPS) – Mean (SD)	8 (4.9)	10 (5.0)
Crime/Violence Scale (CVS) – Mean (SD)	8 (6.2)	4 (4.8)
Any Past Year Disorderc (%)	94	94
Any Past Year Internalizing Disorder (%)	56	68
Depression (%)	37	53
Anxiety (%)	21	44
Traumatic Distressd (%)	42	50
Suicidale (%)	16	20
Any Past Year Externalizing Disorder (%)	66	40
Attention Deficit/Hyperactivity Disorder (%)	48	33
Conduct Disorder (%)	58	30
Any Past Year Substance Use Disorder (%)	86	90
Dependence (%)	56	76
Abuse (%)	29	13
Any Past Year Crime/Violence Problems (%)	66	41
High Physical Conflictf (%)	22	11
Mod/High General Crimeg (%)	64	37

^aAll differences by age are significant at p < .05 level using a Wilcoxon Rank order test for continuous measures and Chi-square test for nominal variables.

^bTotal symptom count which includes all of the symptoms in the four scales listed below.

^cIncludes any of the diagnoses listed under the four domains below.

^dIncludes Post Traumatic Distress, Acute Traumatic Distress and traumatic distress associated with Child Maltreatment / Interpersonal Violence that may be complex and both in the past and ongoing.

^eAny suicidal thoughts, plans, means or attempts.

^fPhysical violence towards others based on the GAIN’S variation of the Conflict Tactic Scale.

^gAny illegal activities across the FBI’s Uniform Crime Report categories beyond simple possession/use.

The GAIN Family of Measures

The participant characteristics, dimensional measures and clinical diagnoses/problems were all based on participant self report to in-person interviews with the GAIN.¹ The full GAIN is a 90- to 120-minute standardized biopsychosocial measure that integrates clinical and research assessment into one comprehensive structured interview with eight main sections (background, substance use, physical health, risk behaviors, mental health, environment, legal, and vocational). The full GAIN has 103 scales and indices to assess problems, measure change and document service utilization. The GAIN incorporates DSM-IV-TR³³ symptoms for common disorders (discussed further below), American Society of Addiction Medicine’s (ASAM),⁴⁷ Patient Placement Criteria for the Treatment of Substance-Related Disorders, Joint Commission on Accreditation of Healthcare Organization (JCAHO)⁴⁸ epidemiological questions from the National Household Survey on Drug Abuse (NHSDA),⁴⁹ and items which have been economically valued for benefit cost analysis with adults and adolescents by Dr. Michael French over the past decade.^50–51 The GAIN’s main scales have demonstrated excellent to good internal consistency (alpha over .90 on main scales, .70 on subscales), and test-retest reliability (Rho over .70 on problem counts, Kappa over .60 on categorical measures). GAIN measures have been validated with time line follow-back methods, urine tests, collateral reports, treatment records, and blind psychiatric diagnosis (Rho of .70 or more; Kappa of .60 or more).^{24,38,52–57} The GAIN has also been applied to predict substance use relapse and recidivism to crime up to 30 months later.^58–59 A more detailed list of studies, copies of the actual GAIN instruments/items (full and short screener), and the syntax for creating the scales and diagnostic group variables are publicly available at www.chestnut.org/li/gain.

The GAIN Model of Emotional/Behavioral Problems

Figure 1 shows the substantive-statistical model of emotional and behavioral problems on which the full GAIN is based. It presumes that there is an underlying second order factor of multi-morbidity that can be represented by a total symptom count called General Individual Severity Scale (GISS; 123 items, alpha of .97 in adolescents and .97 in adolescent). These symptoms can be divided into four main dimensions represented by the:

1. The Internal Mental Distress Scale (IMDS; 43 items, alpha of 0.94 with adolescents and .96 with adults), which is a count of past-year symptoms related to internalizing disorders, including somatic, anxiety, depression, traumatic stress and suicide thoughts; it is based on the DSM-IV-TR (APA),³³ the Hopkins Symptom Checklist 90 (HSCL-90),^60–61 the Mississippi Scale of Post Traumatic Stress Disorder (PTSD)^62–65 and common screening items for suicide risk (homicidal thoughts, suicidal thoughts, plans, means, attempts).

2. The Behavior Complexity Scale (BCS; 33 items, alpha of 0.94 with adolescents and .96 with adults), which is a count of past-year symptoms related to externalizing disorders, including attention deficit, hyperactivity/impulsivity, and conduct disorder; it is based on the DSM-IV-TR (APA)³³ symptoms/disorders that are most common in adolescents but still persist into adulthood are correlated with substance use severity.³⁹

3. The Substance Problem Scale (SPS; 16 items, alpha of 0.90 with adolescents and .92 with adults), which is a count of past-year symptoms related to any alcohol or drug use disorders, including abuse, dependence, substance induced health and psychiatric problems; it is based on the DSM-IV-TR (APA)³³ and is associated with increased odds of internalizing and externalizing disorders.³⁹

4. Crime/Violence Scale (CVS; 31 items, alpha of 0.90 with adolescents and .89 with adults), which is a count of increasingly violent strategies used for resolving interpersonal conflict in the past year and the types of drug related, property, and interpersonal crimes the respondent has committed; it is based on the Conflict Tactic Scale introduced in the Family Violence Survey⁶⁶ and lay versions of the Federal Bureau of Investigations⁶⁷ uniform crime report (UCR) categories introduced in the 1995 NHSDA⁴⁹ and predicts future crime and violence.^58–59

FIGURE 1.

GAIN Model of Emotional and Behavioral Problems

Under each of these four dimensions, there are 15 specific subscales that relate to a given type of diagnosis or problem (e.g., depression, conduct disorder, dependence, property crime) that can be important for deciding the most appropriate clinical intervention. For the purpose of the screener, however, we have focused on the fact that symptoms/disorders within the same dimension tend to be more highly correlated (e.g., depression and anxiety are more related than depression and violent crime).

Confirmatory factor analysis of the model in Figure 1 suggests that it closely fits the actual data with two caveats. First, there is a further loading between Crime/Violence and Conduct Disorder. Second, though structurally similar, adolescent and adult models fit better when their coefficients are calculated separately in terms of both confirmatory fit index (CFI; .83 constrained to the same coefficients across age vs. .92 if coefficients are allowed to vary by age) and Root Mean Square Error of Approximation (RMSEA; .08 constrained to the same vs. .06 if allowed to vary by age). Hu and Bentler⁶⁸ suggest CFI should be over .9 and RMSEA under .6, so in this paper we have analyzed adult and adolescent data separately.

Development of the GAIN-Short Screener (GSS) Measures

As noted above, the GSS was developed to quickly identify who is likely to have an internalizing, externalizing, substance use or crime/violence diagnosis overall and to triage them by the kinds of emotional/behavioral health interventions that are likely to be appropriate. To be clear, it presumes that once you get people to the right program, a full GAIN or other more detailed assessment would be done there. Thus, this is a screener in the traditional sense, not a replacement for a detailed assessment.

We developed a 20-item GSS using 5 items for each of the four factors in Figure 1. Within each factor we selected at least one item from each of the subscales in the factor. The initial selection of items was based on the highest item-total correlation with the rest of the sub-scale and scale. A Rasch⁶⁹ analysis with Winsteps⁷⁰ was then applied to determine the relative item severity on the latent dimension represented by the four respective dimensions of Figure 1. For each dimension, we then constructed a chart with Rasch severity on the x-axis and the cumulative percent with a diagnosis on the y-axis. Using the resulting S-curve, we identified the transitional range of diagnostic uncertainty. Below this range, less than 10% of the patients had a diagnosis in this dimension; above this range over 90% of the patients had a diagnosis in this dimension. To achieve a high correlation between the screener and the full dimensional measure, we then adjusted the selection of items to make sure that the five items were spread as evenly as possible across this range of diagnostic uncertainty. (This technique was actually borrowed from a computer adaptive testing procedure developed by Riley and colleagues⁷¹ that has been used with the GAIN’s Substance Problem Scale.) The above process was done separately for adolescents and adults. The item that worked the best on average in both was used. The final 20-item version has a Total Disorder Screener (TDScr) with four 5-item subset screeners that correspond to the model in Figure 1 (Internalizing Disorder Screener (IDScr), Externalizing Disorder Screener (EDScr), Substance Disorder Screener (SDScr) and Crime/Violence Screener (CVScr). While the GSS (and all GAIN measures) are copyrighted and there is a license fee of $100, there is no per use fee, it has open syntax for scoring, and is available on line to preview for free (http://www.chestnut.org/LI/gain/GAIN_SS/index.html).

Validation Procedures

The 20 items in the GSS will be used to predict the dimensional and/or categorical values from the 123 diagnostic items in the full GAIN. For this analysis, if a single item was missing on any GSS or full GAIN measure, it would be replaced based on the average value for the relevant subscale or screener. Individual items were missing in less than 2% of the cases.

The GSS dimensional measures were validated based on a confirmatory factor analysis, a comparison of the Pearson correlation matrix between the full and screener scales, internal consistency based on Cronbach’s alpha and a measure “efficiency” as defined in equation (1).

$$\text{Efficiency}=(\#\text{of screener items}/\#\text{of full scale items})/(\text{diagonal correlation})$$ (1)

Thus, efficiency goes down (good) the fewer items that are used and up (bad) the less the screener is correlated with the full scale. To verify that each sub-screener has discriminant validity, we compared the correlation of the sub-screener with the full GAIN scale that it was drawn from with the average correlation of the sub-screener with the other three GAIN dimensional scales.

The GSS cut points were validated based on sensitivity (the percentage of people with disorders on the full GAIN correctly identified by the GSS), specificity (percentage of people without a disorder on the full GAIN correctly excluded by the GSS) and the percentage of area under the curve (AUC) in a Receiver Operating Characteristics (ROC) analysis plotting the sensitivity (y-axis) against 1-specificity (x-axis). The optimal cut point is the one closest to or above 90% sensitivity, 90% specificity and 90% under the curve in the upper left corner (with values of 80% being good and 70% being fair). These analyses were done separately for adolescents and adults for the total screener and each of the four 5-item sub-screeners.

RESULTS

Validation of Dimensional Measures

Table 2 compares the GAIN Short Screener (GSS) measures in columns with the full GAIN measures in rows. The 20-item Total Disorder Screener represents only 17% of the 123 items in the full GAIN’s General Individual Severity Scale. While this made it less internally consistent (Cronbach’s alpha of .89 vs. .97), the GSS Total Disorder Screener was still highly correlated with the full GAIN’s General Individual Severity Scale (.94 for both adolescents and adults) and very efficient (efficiency = .17 for both, where lower than 1 is good). The four sub-screeners followed a similar pattern ranging from an 88% reduction in items (43 to 5) for internalizing disorders to a 68% reduction (16 to 5) for substance use disorders. This generally reduced the alpha from the excellent (.89 to .97) to moderate (.72 to .87) range with one major exception – the Crime/Violence Scale alpha for adults dropped from .89 to .65. The four sub-screeners, however, were still highly correlated with their respective longer version (Diagonal correlation of .84 to .90). Thus, the total and four sub-screeners reduced the number of items with minimal loss of information and are more efficient.

TABLE 2.

Internal consistency, efficiency and correlations of GAIN-SS with GAIN-I

			Correlationsa
Full GAIN-I Scale	GAIN-I No. of Items	GAIN-I Cronbach’s Alpha	Total Disorder Screener (TDScr)b	Internalizing Disorder Screener (IDScr)	Externalizing Disorder Screener (EDScr)	Substance Disorder Screener (SDScr)	Crime/ Violence Screener (CVScr)
Adolescent (n = 6,177)
General Individual Severity Scale (GISS)c	123	0.97	0.94	0.73	0.77	0.66	0.68
Internal Mental Distress Scale (IMDS)	43	0.94	0.74	0.89	0.53	0.44	0.38
Behavior Complexity Scale (BCS)	33	0.94	0.81	0.52	0.88	0.49	0.55
Substance Problem Scale (SPS)	16	0.90	0.77	0.46	0.46	0.92	0.49
Crime/Violence Scale (CVS)	31	0.90	0.72	0.33	0.55	0.43	0.86
GAIN-SS Number of items			20	5	5	5	5
GAIN-SS Cronbach’s Alpha			0.87	0.74	0.76	0.76	0.72
GAIN-SS Efficiencyd			0.17	0.13	0.17	0.34	0.19
Average non-diagonal Re				0.44	0.51	0.45	0.47
Adults (n = 1,805)
General Individual Severity Scale (GISS)c	123	0.97	0.94	0.80	0.81	0.64	0.52
Internal Mental Distress Scale (IMDS)	43	0.96	0.79	0.90	0.62	0.46	0.27
Behavior Complexity Scale (BCS)	33	0.96	0.80	0.60	0.90	0.44	0.40
Substance Problem Scale (SPS)	16	0.92	0.75	0.47	0.43	0.93	0.36
Crime/Violence Scale (CVS)	31	0.89	0.62	0.31	0.49	0.35	0.84
GAIN-SS Number of items			20	5	5	5	5
GAIN-SS Cronbach’s Alpha			0.87	0.79	0.81	0.78	0.65
GAIN-SS Efficiencyd			0.17	0.13	0.17	0.33	0.19
Average non-diagonal Re				0.46	0.51	0.42	0.34

^aAll correlations are significant, p < .001.

^bThe Total Disorder Screener includes all the items in the four screeners to the right.

^cThe General Individual Severity Scale includes all of the items in the four scales below it.

^dEfficiency = (# of screener items/# of full scale items). *1/(diagonal correlation).

^eThe average correlation of the (sub) screener with the three (non-diagonal) full GAIN scales in the column.

To test the ability of the four sub-screeners to differentiate the types of problems we also examined their discriminant validity and overall structure (Table 2). The diagonal correlation with the scale that the screener items were drawn from was 1.7 to 2.5 times larger than the correlation of each screener with the other three scales from the full GAIN for the internalizing disorder screener (.89 diagonal vs. .44 average non diagonal for adolescents and .90 vs. .46 for adults), externalizing disorder screener (.88 vs. .51 for adolescents and .90 vs. .51 for adults), substance disorder screener (.92 vs. .45 for adolescents and .93 vs. .42 for adults) and crime/violence screener (.86 vs. .47 for adolescents and .84 vs. .34 for adults).

A confirmatory factor analysis of the structure of the GSS shows that it is also consistent with the regular GAIN model in Figure 1 after allowing adolescents and adults to vary and the cross loading of conduct disorder items with crime/violence. The confirmatory factor analysis was slightly worse than the full-scale version in terms of the confirmatory fit index (CFI; .87 for GSS vs. .92 for the full GAIN, where up is good) and slightly better in terms of Root Mean Square Error of Approximation (RMSEA; .05 for GSS vs. .06 for the full GAIN, where down is good). This suggests that each of the sub-screeners has good discriminant validity and that the total structure is consistent with the model used with the full GAIN.

Optimization and Validation of Cut Points

Table 3 gives the sensitivity, specificity, and area under the curve for cut points of 1, 2, and 3 or more symptoms on the total and each of the four sub-screeners relative to respective diagnoses for that dimension. For the Total Disorder Screener it takes a cut point of 3 or more to get a balance of 90% + sensitivity and 90% + specificity. Figure 2 shows this graphically with the results of the Receiver Operator Characteristics (ROC) analysis. The Total Disorder Screener was very well behaved, with 96% of area under the curve for adolescents and 97% for adults. These differences were not significantly different by age (X²₍₁₎ = 3.2, n.s.d.). For both adolescents and adults, 3 or more past-year symptoms on the GSS is the closest to the optimal point of 90% sensitivity and specificity).

TABLE 3.

Sensitivity, specificity, and area under the curve (AUC) between GAIN-SS cut points and self-reported GAIN-I diagnoses by age^a

		Adolescent (n = 6,177)			Adult (n = 1,805)
Screener	Cut off	% Sensitivity	% Specificity	% AUC	% Sensitivity	% Specificity	% AUC
Total Disorder Screener (TDScr)b	> = 1	99	47		99	57
	> = 2	96	80	96	95	88	97
	> = 3	91	90		90	92
Internalizing Disorder Screener (IDScr)c	> = 1	94	71		97	74	96*
	> = 2	77	95	92	87	94
	> = 3	55	99		71	100
Externalizing Disorder Screener (EDScr)d	> = 1	98	75		97	81
	> = 2	86	88	94	85	93	95*
	> = 3	66	96		66	98
Substance Disorder Screener (SDScr)e	> = 1	96	73		97	73
	> = 2	86	93	95	91	96	97*
	> = 3	68	100		80	100
Crime/Violence Screener (CVScr)f	> = 1	94	76		91	87
	> = 2	70	99	93	52	100	92
	> = 3	49	99		28	100

^aSensitivity is the percent of people with a diagnosis on the GAIN-I correctly identified by screener at a given cut point or above; specificity is the percent of people without the diagnosis correctly rejected by the screener at a given cut point or above; and AUC is the area under the curve formed by 1-specificity on the x-axis and sensitivity on the y-axis (see Figure 1 & 2).

^*These AUC are all significantly different by age at p < .05.

^bRelative to detecting any of the disorders listed below.

^cRelative to detecting any internalizing disorder (Depression, Anxiety, Trauma, Suicide Ideation) on the full GAIN.

^dRelative to detecting any externalizing disorder (Attention Deficit, Hyperactivity, Impulse Control, Conduct) on the full GAIN.

^eRelative to detecting any substance use disorder (Abuse, Dependence) on the full GAIN.

^fRelative to detecting any crime/violence problem (high interpersonal conflict, any drug, property, or interpersonal/violent crime).

FIGURE 2.

Receiver Operator Characteristics (ROC) of the GSS Total Disorder Screener (TDScr) Relative to the Full GAIN

All four of the sub-screeners have 92% or more of their area under the curve for both adolescents and adults. However, as illustrated in Table 3 and Figure 3, there is generally no one optimal cut point on the sub-screeners with 90% + sensitivity and specificity. The one exception is the Substance Disorders Screener that meets this criterion at the 2 + cut-point for adults only. In the absence of a single optimal cut point, it is possible to triage the score into three groups: 0 (unlikely), 1–2 (possible), and 3–5 (probable) for which the lower cut point (1 +) has 90% sensitivity and the upper cut point (3 +) has 90% specificity. Comparisons of the ROC curves by ages suggest that the GSS worked slightly better for adults than adolescent on the Internalizing Disorder Screener (X²₍₁₎) = 45.0, p < .0001), Externalizing Disorder Screener (X²₍₁₎) = 5.4, p < .05), and the Substance Disorder Screener (X²₍₁₎) = 18.2, p <. 0001).

FIGURE 3.

ROC of the GAIN Short Screener (GSS) Subscales Relative to the Full GAIN

DISCUSSION

Reprise

In this study we described the development of the 20-item GAIN-Short Screener (GSS) and validated its ability to serve as an efficient tool for reliably assessing psychopathology and crime/violence as a dimensional measure and as a screener to identify people who are likely to have diagnoses and require clinical services. While it would not be sufficient to render a specific diagnosis, the GSS appears capable of quickly identifying who has a possible or probable diagnosis/problem and triaging the problem into four dimensions of severity: internalizing disorders, externalizing disorder, substance disorders, and crime/violence problems.

Recommended Cut Points

Based on these analyses we recommend that the total and four sub-screeners be triaged into three groups for clinical decision making:

1. Low (0), who are unlikely to need services;

2. Moderate (1–2), who have a possible diagnosis and are likely to benefit from a brief intervention; and

3. High (3 or more) who have a high probability of a diagnosis and need more formal assessment and intervention (either directly or through referral).

These same ranges apply to the total and sub-screeners. As shown in the results section, moderate or above (1+) will have 90% or more sensitivity within a sub-screener and high will have 90% or more specificity. A manual is available on line to further support placement and treatment planning based on these cut points.⁷²

In practice, most clinicians will want to look for a score of 3 or more on the Total Disorder Screener and then assume that services are needed in each of the four domains with scores of 1 or more. Thus, if you reported 5 symptoms − 3 on the Externalizing Disorder Screener and 2 on the Substance Disorder Screener, you would meet criteria for needing services and be referred to a program(s) to meet your externalizing and substance disorder needs. One very important caveat: this is just a screener based on self-reports. Clinicians should still be encouraged to combine it with other information (e.g., from a parent, spouse, probation officer, supervisor, records) and/or to override it on individual cases where they suspect other problems. It is designed to be a tool to facilitate clinical judgment, not override it.

Strengths and Limitations

This paper has several strengths including a large/diverse sample, being based on a full instrument that has a formal measurement model, and advanced statistical analyses. There are, however, some notable limitations of this paper that should be acknowledged. First, the analysis is based entirely on self-reported data. Second, the comparisons are between a part and whole scale collected at the exact same time. If there were alternative data available and/or more elapsed time, the comparisons would presumably come out lower. We are currently working with several treatment, school, workplace, mental health, justice and welfare programs to collect such data and hope to report those findings in the near future. Third, while GSS is an efficient screener to identify who is likely to have a disorder in a given area and might need further assessment, more information will typically need to be collected later to make a full diagnosis.

Next Steps

Because of its efficiency and ease of implementation (minimal training and 2 pages per administration), the GSS also has the potential to help with policy and program planning. This is important for the increasing number of federal, state and local initiatives to mandate or encourage standardized screening across a wide range of systems. This might include school, workplace, welfare and justice system that are increasingly interested in early identification and intervention related to behavioral health issues. The GSS can be used as a simple form of needs assessment to help guide program planning and the purchase of further training or technical assistance. For program evaluation, this simple measure can also be used to “predict” the number of people with each diagnosis or types of services needed and to evaluate/manage penetration and referral rates. It may also prove useful as a short measure of change that could be used to track reductions in symptoms in each of the four dimensions. Use of the GSS in these other settings and applications, however, still needs to be evaluated.

Conclusion

In conclusion, it is important to recognize that assessment costs money and staff/client time. Rather than one size fits all, we have consistently advocated a more progressive approach to assessment - screening general populations, a brief assessment for targeted populations and full assessment for more complicated/multi-morbid populations. We believe that the GAIN Short Screener (GSS) has great potential to serve as both a cost-effective front door screener to identify people with co-occurring disorders across multiple systems and route them to the right services and more detailed assessments.