Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Oct 3.
Published in final edited form as: Crim Justice Behav. 2007 Sep;34(9):1198–1216. doi: 10.1177/0093854807304430

CJDATS CO-OCCURRING DISORDERS SCREENING INSTRUMENT (CODSI) FOR MENTAL DISORDERS (MD)

A Validation Study

STANLEY SACKS 1, GERALD MELNICK 1, CARRIE COEN 1, STEVE BANKS 2, PETER D FRIEDMANN 3, CHRISTINE GRELLA 4, KEVIN KNIGHT 5, CARON ZLOTNICK 6
PMCID: PMC3184768  NIHMSID: NIHMS313487  PMID: 21976780

Abstract

Three standardized screening instruments—the Global Appraisal of Individual Needs Short Screener (GSS), the Mini-International Neuropsychiatric Interview–Modified (MINI-M), and the Mental Health Screening Form (MHSF)—were compared to two shorter instruments, the 6-item Co-Occurring Disorders Screening Instrument for Mental Disorders (CODSI-MD) and the 3-item CODSI for Severe Mental Disorders (CODSI-SMD) for use with offenders in prison substance-abuse treatment programs. Results showed that the CODSI screening instruments were comparable to the longer instruments in overall accuracy and that all of the instruments performed reasonably well. The CODSI instruments showed sufficient value to justify their use in prison substance-abuse treatment programs and to warrant validation testing in other criminal justice populations and settings.

Keywords: co-occurring disorders, criminal justice, mental disorders, substance abuse, screening, instrument

Inmates with co-occurring substance use and mental disorders (COD) represent a significant problem in the criminal justice system, given the increasing numbers of prisoners diagnosed with co-occurring disorders and the difficulty of managing and treating offenders with concurrent disorders. It is estimated that, compared to the general population, prisoners are 2 to 4 times more likely to suffer from psychotic illness or major depression (Fazel & Danesh, 2002). Estimates from U.S. Department of Justice statistics indicate that 16% of those in state prisons, jails, and on probation suffer from some form of mental disorder (Ditton, 1999).

Furthermore, the number of correctional clients with mental disorders appears to be increasing. As an example, reports from the Colorado Department of Corrections chronicle a steadily rising proportion of inmates with mental illness, from 4% in 1991, to 14% in 2001 (Kleinsasser & Michaud, 2002), to 20% more recently (J. Stommel, personal communication, March 16, 2005). Among the 20% of inmates with mental illness, three quarters were estimated to have a co-occurring substance-use disorder.

The need to identify and treat these individuals is evinced in the risk to the community, problems of behavior management within the prison system, and poorer treatment outcomes. The increased risk of violence among individuals with COD has now been well-documented (Melnick, Sacks, & Banks, 2006; Monahan et al., 2000; Monahan et al., 2001; Monahan et al., 2005; Steadman et al., 1998). Furthermore, studies have found that among inmates treated in prison for schizophrenia and followed for as long as 10 years, 67% re-offended violently and 30% were recidivated for violent crime. From the perspective of the current article, it is noteworthy that polydrug abuse or alcohol problems increased the odds of reconviction by a factor of 3 or more (Baxter, Rabe-Hesketh, & Parrott, 1999). In addition to the risk for violence in the community, individuals with COD pose a particular challenge to the criminal justice system for several reasons. Prisoners with mental disorders generally adapt less well to the complexity of prison life (Morgan, Edwards, & Faulkner, 1993), frequently face stigmatization or isolation from other prisoners, and have high rates of victimization (Correctional Association of New York, 2004), and are more likely to be written up in an incident report during their first 3 months of incarceration (DiCataldo, Greer, & Profit, 1995).

Moreover, studies show that COD presents unique challenges to treatment programs. The elevated risk factors and poorer treatment outcomes of those with COD are exemplified by their higher rates of HIV infection, relapse to substance use, rehospitalization, depression, and suicide risk, as opposed to those with a single disorder (Drake et al., 1998; Office of the Surgeon General, 1999). Numerous studies have documented the effects of COD in criminal justice populations, including higher rates of dropout from treatment alternatives to prison (Lang & Belenko, 2000), shorter lengths of stay in substance-abuse treatment among probationers (Hiller, Knight, & Simpson, 1999), and the failure to complete jail-based substance-abuse treatment programs by ratios as high as 3 to 1 when compared to prisoners who did not have a history of mental disorder (Brady, Krebs, & Laird, 2004).

Although COD clients constitute a population that has proven difficult to treat, promising interventions now offer the very real prospect of positive change. For example, Sacks and colleagues demonstrated significantly lower reincarceration rates and significantly greater reduction in substance abuse for offenders with COD in a modified therapeutic community (TC) program as compared to those in standard mental health treatment (Sacks, Sacks, McKendrick, Banks, & Stommel, 2004; Sullivan, McKendrick, Sacks, & Banks, in press). Before treatment can be matched to a client population that can derive its benefits, however, an instrument capable of identifying appropriate individuals for referral must be available.

At present, no single brief instrument has been validated to screen for both substance use and co-occurring mental disorders among prisoners. The lack of a standardized and uniformly accepted COD screening instrument has hindered both the ability to plan programs for offenders with COD and to identify prisoners in need of specialized COD treatment. Peters and colleagues evaluated several frequently used screening instruments for alcohol and drug disorders in prison settings (Peters, Greenbaum, Steinberg, & Carter, 2000). They found the Texas Christian University Drug Screen (TCUDS; Broome, Knight, Joe, & Simpson, 1996; Simpson, 1995; Simpson, Joe, Rowan-Szal, & Greener, 1997), the Simple Screening Instrument (SSI; Center for Substance Abuse Treatment, 1994), and the ASI–Drug Screen (McLellan et al., 1985; McLellan et al., 1992; McLellan, Luborsky, Woody, & O'Brien, 1980) in combination with the Alcohol Dependence Scale1 (ADS; Skinner & Horn, 1984) to be about equally effective in detecting substance-use disorders among prison inmates and that best met their criteria of being accurate overall, brief, in the public domain, and capable of administration by non-(mental health) professional staff.

Instruments commonly used to screen for mental disorders include the Global Appraisal of Individual Needs (Dennis, 1999), the Mini-International Neuropsychiatric Interview–Modified (MINI-M; Sheehan et al., 1998), and the Mental Health Screening Form (MHSF; Carroll & McGinley, 2001). The K6 (Kessler et al., 2003) provides a very brief screen that was 96% accurate in identifying individuals who did not have a severe mental disorder, but was only 36% accurate in identifying those with a severe mental disorder. More recently, the K6 has been found to have a sensitivity of 76% and specificity of 81% when testing for severe mental disorders in population with a substance-use disorder (Swartz & Lurigio, 2006). The Psychiatric Diagnostic Screening Questionnaire (Zimmerman & Mattia, 2001) is another instrument widely used in criminal justice settings. The Brief Jail Mental Health Screen, which the authors recommend for men only (Steadman, Scott, Osher, Agnese, & Robbins, 2005), and the Jail Screening Assessment Tool, a semistructured interview used to assess women prisoners in jails (Nicholls, Lee, Corrado, & Ogloff, 2004) have been introduced recently to screen for mental disorders in criminal justice settings. Earlier reviews of screening instruments for mental disorders (Peters & Green Bartoi, 1997; Peters & Hills, 1997) recommended the use of two instruments, the Referral Decision Scale (Teplin & Schwartz, 1989) and the Brief Symptom Inventory (Derogatis, 1993).

Screeners for substance use and mental disorders have often been administered in separate settings, and the information has often not been consolidated to identify offenders who need specialized COD assessment and treatment services and to provide information to administrators regarding the need for specialized COD services. A more effective approach would be to have a consolidated, integrated screening instrument effective in identifying both disorders, so that this information could be gathered at the same time and by the same staff, resulting in a more comprehensive picture of the offender's needs for specialized COD services. This article reports on the first validation study in a planned development of a brief screening instrument for both mental and substance use disorders (i.e., COD) for use in a variety of criminal justice settings. The substance-use-screening component of the instrument (not examined in the present study) is composed of the first 9 items of the TCUDS, an instrument widely used in criminal justice research. The selection of the TCUDS was based on a previous validation study (Peters et al., 2000) and on the recommendation of two panels, one composed of criminal justice substance-abuse stakeholders and the other of experts in COD Sacks et al., 2007 The study described in this article was conducted to validate the cutoff points for a mental-health screening instrument to serve as a companion to the TCUDS in screening for COD. The validation study includes cutoff points for standardized (GSS, MHSF, and MINI-M) and unstandardized (CODSI-MD and CODSI-SMD) screening instruments previously identified in a pilot study of offenders in prison substance-abuse programs.

METHOD

SAMPLE

The research study was part of the National Institute on Drug Abuse (NIDA)-funded Criminal Justice Drug Abuse Treatment Studies (CJDATS) initiative. Under this initiative, nine regional research centers, a coordinating center, and NIDA work with federal, state, and local criminal justice partners to develop and test new approaches to meeting the needs of offenders with substance-use disorders for prison and reentry services. The present research project is 1 of 13 current CJDATS studies that include screening and referral, treatment interventions for reentering offenders, engagement and retention in treatment, improving coordination and linking services in community, meeting the needs of special populations of offenders, and understanding the organizational and contextual factors and current treatment practices for offenders with histories of substance abuse.

The project consisted of an initial pilot study and this validation study, which employed a geographically diverse sample of 280 consecutive new admissions to prison substance-abuse treatment programs. The pilot study, which established cutoff scores for each of the instruments among program admissions (reported elsewhere; Sacks et al., 2007), included a total of 100 subjects. The remaining 180 cases were used in this study to validate these cutpoints. A study subject was considered to be a “new admission” for 14 days from his or her entry to the treatment program. Exceptions were made under special circumstances (e.g., potential subjects were missed when a lockdown prevented interviews from being scheduled so that the initial test battery could not be completed within 2 weeks of entry to the program). Four CJDATS Research Centers were involved in the data collection and 13 different prison substance-abuse treatment programs were used. The participating centers and the number of subjects drawn from each were as follows: NDRI Rocky Mountain in Colorado (N = 117), Lifespan at Brown University in Rhode Island (N = 75), the Institute for Behavioral Research at Texas Christian University in Texas (N = 60), and the Integrated Substance Abuse Programs at UCLA in California (N = 28). The sample was stratified to include one third women, which represented an oversampling compared to the actual percentage of women in state prison populations (7%; Harrison & Beck, 2005) that was incorporated to make the screening instrument sensitive to any possible gender differences in the accuracy of the screening instruments in the prison population.

The large numbers of individuals with substance-abuse disorder entering these prison programs afforded a unique opportunity to develop and test a measure for mental disorders appropriate to this population. For example, previous research indicated a high prevalence rate of mental disorders among admissions to prison substance-abuse treatment programs: For example, 59% reported previous psychological treatment and 68% reported severe depression, 61% severe anxiety, 46% trouble controlling violent behavior, and 11% a previous suicide attempt (Prendergast, Hall, Wexler, Melnick, & Cao, 2004).

A total of 311 inmates were approached to participate in the study. In addition to the 280 subjects who constituted the full sample (pilot plus validation samples), 29 (9%) refused to participate in the study and a communication barrier2 prevented 2 inmates (0.6%) from participating. The 2 inmates who reported a problem understanding the questions were replaced by the next subjects to enter the treatment program. The 29 inmates who refused to participate cited reasons that were consistent across sites and included the following: not interested in participating in studies, denies drug history, had attention-deficit hyperactivity disorder (ADHD) and couldn't sit for long periods of time, and didn't want to spend the time. Because this rate of refusal was relatively low and was not a threat to validity, the authors did not collect any further information on those who declined to participate.

The CODSI study was approved by the Institutional Review Boards at each of the four CJDATS Research Centers, and each received certification from the Office for Human Research Protections (OHRP). The project was also reviewed by a Data and Safety Monitoring Board and received a Certificate of Confidentiality from NIDA. Prior to taking part in the study, each inmate completed an informed consent process, guided by a research interviewer, and signed a consent form; all research staff has been trained in the protection of human subjects in research and in the particular concerns for the protection of prisoners. The consent process was free from any coercion; participation was entirely voluntary and had no bearing on the inmate's circumstances, either within the treatment program or as a prisoner.

DEMOGRAPHIC AND BACKGROUND VARIABLES

A modified (shorter version) of the CJDATS Intake Interview (CJDATS, 2005), a structured interview used to collect sociodemographic background information, including education and employment, criminal history, health and psychological status, and drug history, was administered to all subjects.

SCREENING INSTRUMENTS

The selection of screening instruments was based on an extensive review of the screening literature, stakeholder input, and the recommendations of a panel of experts in COD (Sacks et al., 2006). A literature search identified screening instruments with good reliability and validity that were in the public domain, required no more than 20 minutes to complete, and were suitable for administration by staff members without extensive training. More than 150 instruments were reviewed; those that best met the criteria were submitted to a panel consisting of 12 stakeholders (criminal justice professionals working in the area of substance abuse) and to a panel consisting of 10 experts in COD. These panels evaluated the instruments based on the type of personnel and training needed for administration, the maximum time available for administration, the ease of use and scoring, acceptability of questions to respondents, the type of profile generated, and utility estimates (e.g., the tradeoffs between the length of the instrument, the desired level of accuracy). The process identified three instruments that were selected for pilot testing: the 22-item MINI-M (Sheehan et al., 1998), the 24-item MHSF (Carroll & McGinley, 2001), and the 15-item GSS version 1.03 (Dennis, Chan, & Funk, 2006).

CRITERION INSTRUMENT

The Structured Clinical Interview for DSM-IV (SCID; First, Spitzer, Gibbon, & Williams, 2002) was used as the criterion measure for mental disorders. The SCID is widely accepted as the standard for assessing substance use and mental disorders (Baldassano, 2005; Blackburn, 2000; Maffei et al., 1997; Magruder, Sonne, Brady, Quello, & Martin, 2005; Ramirez Basco et al., 2000), providing DSM diagnoses for Axis I and Axis II disorders using 30-day and lifetime information. Assessment of a mental disorder in the current study is based on the lifetime information. A screening instrument is considered to be accurate if it concurs with the SCID on the presence (or absence) of a disorder.

ADDITIONAL INSTRUMENT DEVELOPMENT

The development of the Co-Occurring Disorder Screening Instrument for Mental Disorders (CODSI-MD) and for Severe Mental Disorders (CODSI-SMD) is fully described in Sacks and colleagues 2007. Briefly, items from the MINI-M, MHSF, and GSS were tested in logistic regression analyses to determine the best combination of items to predict a SCID diagnosis of either an Axis I or Axis II mental disorder. The rationale for item selection for the CODSI screeners was to make use of existing items from standardized scales rather than creating and testing new items.

The 6 items of the CODSI-MD were selected by this procedure with coefficients ranging from 0.194 to 0.389 (disregarding the signs). The items and their original instruments were as follows:

  • 1

    Have you ever felt you needed help with your emotional problems, or have other people told you that you should get help for your emotional problems? (MHSF)

  • 2

    Have you ever talked to a psychiatrist, psychologist, therapist, social worker, or counselor about an emotional problem? (MHSF)

  • 3

    Have you been told by teachers, guidance counselors, or others that you have a special learning problem? (MHSF, reverse scored)

  • 4

    Have you ever been advised to take medication for anxiety, depression, hearing voices, or for any other emotional problem? (MHSF, reverse scored)

  • 5a

    Have you had one or more occasions when you felt intensely anxious, frightened, uncomfortable, or uneasy even when most people would not feel that way?

  • 5b

    Do you feel anxious or uneasy in place or situations where you might have the panic-like symptoms we just spoke about? Or do you feel anxious or uneasy in situations where help might not be available or escape might be difficult? (MINI-M: This item is scored based on response to 5b.)

  • 6

    During the past 12 months, have you been a bully or threatened other people two or more “The two items that require reverse scoring are used to compensate for overestimation of mental disorder in the remaining items by reducing the positive score.”

An additional logistic regression analysis Sacks et al. 2007 was conducted to determine the items most associated with severe mental disorders (schizophrenia, major depression, bipolar disorder) as well as suicide potential. One item from the MINI-M and 2 items from the MHSF were selected in the regression to form the CODSI-SMD: (a) Have you felt sad, low, or depressed most of the time for the past 2 years? (MINI-M); (b) Did you ever attempt to kill yourself? (MHSF); and (c) Have you ever had a period of time when you were so full of energy and your ideas came very rapidly, when you talked nearly nonstop, when you moved quickly from one activity to another, when you needed little sleep, and when you believed you could do almost anything? (MHSF).

Test Administration

Testing was conducted in two face-to-face sessions within 1 month of each other. Oral administration of the interviews avoided questions of literacy in this population. The first session consisted of completing the informed consent, the CJDATS Intake Interview and the CODSI screening battery, which included the three mental disorder instruments: the MHSF, MINI-M, and GSS (the CODSI-MD and the CODSI-SMD items were nested within these other instruments). The order of administrating the three mental health screeners was randomized to control for possible ordering effects. The second session was the administration of the SCID. As the SCID diagnosis used for the study was based on lifetime information, this window was of sufficient brevity to have a minimal impact on the data though permitting the flexibility needed to accommodate prison and interviewer schedules. The initial session was conducted by experienced interviews who received standardized training (provided separately by each of the participating research centers) in the procedures for collecting informed consent and administering the CJDATS Intake Interview and screening battery. Rater assessment or interpretation was not obtained in the administration of the screening instruments; therefore, training consisted of reviewing the items on each of the screeners, reading manuals about proper interviewing techniques, mandatory training on human subjects, the observation of interviews by experienced staff, and completing multiple practice interviews with supervisors to achieve fluency in presentation. The absence of the need for more specialized training, which reduces the demands on potential users, is interpreted as a strength of the screening instruments. A different group of interviewers who were trained and experienced in the use of the SCID conducted the second session. All SCID interviews were reviewed by a SCID supervisor for completeness and accuracy. To avoid possible influence on the results, SCID interviewers and supervisors were not informed of the results of the first session (intake interview and CODSI screening battery).

ANALYTIC STRATEGY

Precision

The planned analysis for this validation study determined the sensitivity, specificity, and overall accuracy of the screening instruments. Sensitivity is the percentage of individuals identified by the SCID as having a mental disorder who are correctly identified as such by the screening instrument. Specificity is the percentage of individuals who are not classified by the SCID as having a mental disorder who are correctly identified as not having a disorder by the screening instrument. Overall accuracy is the percentage of individuals who are correctly identified by the screening instrument as either having or not having a mental disorder as determined by the SCID.

Cutoff Scores

The analytic plan validated the cutpoints that the pilot study had established for each instrument. A cutpoint (or cutoff score) is the number of positively scored items that produces the best overall accuracy, sensitivity, and specificity. Receiver operating characteristics (ROC; Metz, 1978) curves were developed for the pilot study (N = 100) to determine the predictive value of the instruments along with the full range of possible cutpoints. A range of cutoff scores was calculated that was wide enough to produce a curve of ascending to descending overall accuracy. When cutpoints produced equivalent overall accuracy, the cutpoint with the highest sensitivity was chosen (i.e., the cutoff score that was best at determining those who screened in for a mental disorder). The cutpoints identified in the pilot study were as follows:

  • MHSF: A cutpoint of 3 or greater exhibited a sensitivity of 90.0%, specificity of 47.5%, and overall accuracy of 73.0%.

  • MINI-M: A cutoff score of 5 or greater resulted in a sensitivity of 80.0%, specificity of 52.5%, and overall accuracy of 69.0%.

  • GSS: A cutpoint of 2 or greater demonstrated sensitivity of 83.3%, specificity of 45.0%, and overall accuracy of 71.0%.

  • CODSI-MD: A cutpoint of 3 or greater produced a sensitivity score of 86.7%, specificity of 72.5%, and overall accuracy of 81.0%.

The second set of cutpoints developed in the pilot study was to screen for severe mental disorders (i.e., schizophrenia, bipolar disorder, and major depressive disorder) as well as for suicide potential, as determined by the MDE section of the SCID-IV. The CODSI-SMD with a cutpoint of 2 had an overall accuracy of 82%, with sensitivity at 60.7% and specificity at 90.3%. The MHSF, with a cutoff score of 11, had an overall accuracy of 76%, with sensitivity at 42.9% and specificity at 88.9%. The MINI-M, with a cutpoint of 10, had an overall accuracy of 72%, with sensitivity at 46.4% and specificity at 81.9%. Finally, the GSS Internal Disorder Screener (GSS-IDS; the subsection of the GSS that pertains to severe mental disorder), with a cutoff score of 5, had an overall accuracy of 68%, with sensitivity at 25% and specificity at 84.7%.

The two-tailed nonparametric McNemar test (Siegel, 1956) for related samples (as in the present instance when multiple measures are taken on the same individual) was used to test the significance of differences between the instruments in sensitivity and specificity. Differences in overall accuracy were not tested as these are affected by the prevalence of the disorder in each specific sample; tests higher in sensitivity will show improved overall accuracy when the prevalence of the disorder is high, and tests with high specificity will show greater overall accuracy when the prevalence is low. Therefore, although meaningful in the context of the particular population sampled, overall accuracy is dependent on conditions external to the validity of the instruments in the wider population in which the instrument would be used. Results of the McNemar tests are reported in the text to avoid creating confusion in the tables by presenting multiple comparisons.

RESULTS

Table 1 shows the demographic characteristics of the validation sample. Males comprised two thirds of the study sample by desing. The median number of all arrests was 10, with drug-related arrests accounting for half that number. Physical health problems were prevalent, with almost two thirds (62%) reporting hospitalization for physical health problems. Half of the sample reported experiencing severe depression, and a similar proportion reported severe anxiety in their lifetime. One fifth of the sample had serious thoughts about suicide, and more than 18% had at least one suicide attempt. Twenty-two percent of the sample had at least one psychiatric hospitalization. Substance use, as expected in this population, was significant, with a lifetime prevalence of alcohol use of 98%, marijuana use of 94%, cocaine use of 71%, methamphetamine use of 61%, intravenous drug use of 34%, and heroin use of 25%. Sixty-seven percent of study subjects had received prior treatment for substance abuse during the course of their life.

TABLE 1.

Validation Sample Demographics (N = 180)

Background Characteristics Number
Age 34.51
Male (%) 58.9
Hispanic (%) 36.4
White (%) 51.5
Black (%) 23.4
Other race (Asian, Native American, biracial; %) 25.2
Legally married (%) 23.9
Employed past 6 months before incarceration (%) 56.1
Mean number of children younger than 18 1.95
Mean years of education 11.49
Mean age at first arrest 18.14
Median number of lifetime arrests 10.00
Median number of drug-related arrests 5.00
Hospitalized for physical health problems—lifetime (%) 62.2
Mean number of hospitalizations for psychiatric problems—lifetime 1.74
Ever hospitalized for psychiatric/emotional problems (%) 22.2
Report severe depression (%) 51.1
Report severe anxiety (%) 47.2
Report hallucinations (%) 5.0
Report trouble concentrating, understanding, remembering (%) 43.9
Trouble controlling violent behavior (%) 24.4
Serious thoughts of suicide (%) 20.0
Attempted suicide (%) 18.3
Received prior treatment for drug abuse (%) 66.7
Median number of times treated for substance use 1.00
Use of alcohol—lifetime (%) 97.8
Use of marijuana—lifetime (%) 93.9
Use of heroin—lifetime (%) 25.0
Use of cocaine—lifetime (%) 71.1
Use of methamphetamines—lifetime (%) 61.1
Use of IDU—lifetime (%) 34.4
Open in a new tab

SCREENING FOR MENTAL DISORDER SUFFICIENT FOR A DIAGNOSIS OF COD

Table 2 shows the sensitivity, specificity, and overall accuracy for the four instruments in screening for any mental disorder after weighting the results to adjust for the oversampling of females in the stratified sample. In this table (and those that follow), the N for overall accuracy includes all of the subjects in the validation sample. The N for sensitivity reflects only those subjects with a SCID diagnosis of mental disorder and the N for specificity includes only those subjects who do not have a SCID diagnosis of mental disorder. As shown in Table 2, the overall accuracy was the similar for the MHSF and GSS. The CODSI-MD was within 3 percentage points of the MHSF and GSS and within 4 percentage points of the MINI-M. With reference to sensitivity, the MHSF and GSS produced the highest scores, followed by the CODSI-MD. All three of these instruments produced higher scores than the MINI-M, with the CODSI-MD exceeding the MINI-M by approximately 4% (ns), the GSS exceeding the MINI-M by 10% (p < .005), and the MHSF exceeding the MINI-M by 10 (p < .05). The MINI-M produced the highest specificity score with the CODSI-MD approximately 7% lower and the MHSF and GSS approximately 13% lower, but none of these differences reached statistical significance.

TABLE 2.

Comparison of Precision of Screening Battery Instruments, Any Mental Disorder, Weighted for Gender

Any Mental Disorder Weighted
Instrument Cutoff Score Sensitivity (n = 141) Specificity (n = 39) Overall Accuracy (N = 180)
CODSI-MD 3 75.1% 54.1% 69.6%
MHSF 3 80.9% 47.5% 72.1%
MINI-M 5 71.3% 60.7% 68.5%
GSS 2 81.6% 47.1% 72.6%
Open in a new tab

Note. CODSI-MD = Co-Occurring Disorders Screening Instrument for Mental Disorders; MHSF = Mental Health Screening Form; MINI-M = Mini-International Neuropsychiatric Interview-Modified; GSS = Global Appraisal of Individual Needs Short Screener.

Table 3 shows the results by gender. The high prevalence rates for mental disorder in the study resulted in a relatively small sample for specificity when the data were disaggregated by gender, reducing the power to demonstrate statistical significance for this variable. All four instruments demonstrated higher overall accuracy for females than for males. Given the lower N values when the data were disaggregated for gender, a McNemar two-tailed 0.10 level of significance was used.

TABLE 3.

Comparison of Precision of Screening Battery Instruments, Any Mental Disorder, by Gender

Female
 Male
Sensitivity (n = 64) Specificity (n = 10) Overall Accuracy (N = 74) Sensitivity (n = 77) Specificity (n = 29) Overall Accuracy (N = 106)
CODSI-MD
 Score of 3 or greater 85.9% 30.0% 78.4% 74.0% 55.2% 68.9%
MHSF
 Score of 3 or greater 96.9% 30.0% 87.8% 79.2% 48.3% 70.8%
MINI-M
 Score of 5 or greater 82.8% 30.0% 75.7% 70.1% 62.1% 68.9%
GSS
 Score of 2 or greater 92.2% 20.0% 82.4% 80.5% 48.3% 71.7%
Open in a new tab

Note. CODSI-MD = Co-Occurring Disorders Screening Instrument for Mental Disorders; MHSF = Mental Health Screening Form; MINI-M = Mini-International Neuropsychiatric Interview–Modified; GSS = Global Appraisal of Individual Needs Short Screener.

For males, the GSS, MHSF, CODSI-MD, and MINI-M all scored within 3 percentage points of each other in overall accuracy. The GSS and MHSF produced the highest values for sensitivity among the men, with the CODSI-MD producing an intermediate value, and the MINI-M the lowest value; the only significant difference was the performance of the GSS compared to the MINI-M. For specificity among men, the highest values were produced by MINI-M; the CODSI-MD produced an intermediate score approximately 7% lower, and the MHSF and GSS produced scores approximately 14% lower than the MINI-M, although this difference did not reach statistical significance. For females, the MHSF produced the highest level of overall accuracy and the MINI-M the lowest. Results were similar for sensitivity, with the MHSF again producing the highest values and the MINI-M the lowest (p < .01). With respect to specificity, the N of 10 precluded a meaningful test of significance, but the GSS score was 10% less than the other instruments.

In general, of the four instruments, the MHSF was tied with the GSS in highest overall accuracy in the weighted sample, and each of these two instruments showed superiority in at least one other score disaggregated by gender. The MHSF was superior in overall accuracy for females, and the MHSF and GSS produced the highest accuracy in males. The MINI-M and the 6-item CODSI-MD performed reasonably well in comparison to the other instruments; the CODSI-MD offered the advantage of being a shorter instrument than the MHSF or GSS, which contain 17 and 10 items, respectively.

SCREENING FOR SEVERE MENTAL DISORDERS

The present study also validated previously identified cutoff scores for severe mental ill-ness for the MHSF, MINI-M, GSS, and the CODSI-SMD (Sacks et al., 2007). Table 4 shows the sensitivity, specificity, and overall accuracy for the four instruments in screening for severe mental disorders after weighting the results to adjust for the oversampling of females. The Internal Disorder Screener (IDS) subscale of the GSS, designated as the GSS-IDS, was substituted for the entire GSS because this subset of items was the most relevant to severe mental disorders.

TABLE 4.

Comparison of Precision of Screening Battery Instruments, Severe Mental Disorder, Weighted for Gender

Validation, Severe Mental Disorder Weighted
Instrument Cutpoint Sensitivity (n = 75) Specificity (n = 105) Overall Accuracy (N = 180)
CODSI-SMD 2 50.8% 91.2% 78.1%
MHSF 11 35.4% 93.3% 74.5%
MINI-M 10 41.8% 83.9% 70.3%
GSS-IDS 5 37.5% 95.9% 76.7%
Open in a new tab

Note. CODSI-SMD = Co-Occurring Disorders Screening Instrument for Severe Mental Disorders; MHSF = Mental Health Screening Form; MINI-M = Mini-International Neuropsychiatric Interview–Modified; GSS-IDS = Global Appraisal of Individual Needs Short Screener—Internal Disorder Screener.

The CODSI-SMD produced the highest overall accuracy across the entire sample. The GSS-IDS was the second-highest scoring instrument for overall accuracy, coming within 2 percentage points of the CODSI-SMD. The CODSI-SMD produced the highest sensitivity; the other instruments produced values that were approximately 9% to 15% lower. With respect to sensitivity, the CODSI-SMD was significantly higher than the MHSF (p < .05) and approached significance when compared to the GSS-IDS (p < .10). The GSS-IDS produced the highest specificity score, followed by the MHSF and the CODSI-SMD. The specificity score of the MINI-M was 7% to 12% lower than that of the other instruments (p < .05).

When the scores were disaggregated by gender (see Table 5), the CODSI-SMD produced the highest overall accuracy score for males and tied the MINI-M for highest overall accuracy score for females; the GSS-IDS was almost 7% lower for females. The highest sensitivity scores were achieved by the CODSI-SMD for both males and females, with none of the comparisons reaching statistical significance except the CODSI-SMD and MHSF (< .05). The GSS-IDS, followed by the MHSF, produced the highest specificity scores for men; the CODSI-SMD produced an intermediate score 4% lower than the GSS-IDS; these three instruments showed higher scores than the MINI-M, with p values of < .01, < .10, and < .10, respectively. Among females, the MHSF showed the highest specificity score and the CODSI-SMD the lowest, although these differences did not reach statistical significance.

TABLE 5.

Comparison of Precision of Screening Battery Instruments, Severe Mental Disorder, by Gender

Female
 Male
Sensitivity (n = 43) Specificity (n = 31) Overall Accuracy (N = 74) Sensitivity (n = 32) Specificity (n = 74) Overall Accuracy (N = 106)
CODSI-SMD
 Score of 2 or greater 55.8% 77.4% 64.9% 50.0% 91.9% 79.2%
MHSF
 Score of 11 or greater 41.9% 93.5% 63.5% 34.4% 93.2% 75.5%
MINI-M
 Score of 10 or greater 48.8% 87.1% 64.9% 40.6% 83.8% 70.8%
GSS-IDS
 Score of 5 or greater 37.2% 87.1% 58.1% 37.5% 95.9% 78.3%
Open in a new tab

Note. CODSI-SMD = Co-Occurring Disorders Screening Instrument for Severe Mental Disorders; MHSF = Mental Health Screening Form; MINI-M = Mini-International Neuropsychiatric Interview–Modified; GSS-IDS = Global Appraisal of Individual Needs Short Screener–Internal Disorder Screener.

RELIABILITY

Test/retest reliability for each instrument was calculated from a randomly selected sample of 60 subjects drawn from the Colorado sample and tested 1 week apart. The most meaningful form of reliability was considered replication based on whether an individual met the cutpoint for referral to assessment. Because these data were categorical, Kappa was used to determine agreement between the test and retest using the cutpoints selected for the study. All four instruments produced statistically significant Kappas, with the MHSF and MINI-M producing Kappas superior to those of the GSS as follows: MHSF with a cutoff score 3, κ = 0.625 (p < .000), MINI-M with a cutoff score of 5, κ = 0.618 (p < .000), and the GSS with a cutoff score of 2, κ = 0.381 (p < .01); the 6-item CODSI-MD screener was κ= 0.526 (p < .000). Kappa reliability was established on test–retest data on the same sample described above using the cutpoints for screening for severe mental disorders. Kappa reliability for the each of the instruments was: CODSI-SMD κ = 0.660, p < .000; MHSF κ = 0.761, p < .000; MINI-M κ = 0.682, p < .000, GSS κ = 0.487, p < .000. The percentage of times that the instrument consistently determines whether or not the individual falls below the specified cutoff point is another indication of the intertest agreement on whether the cutoff-score criterion was met at both Time 1 and Time 2. These percentages were calculated as follows: for any mental disorder, the percentage agreement between Time 1 and Time 2 was CODSI-MD (80.0%), MHSF (85.0%), MINI-M (81.7%), and GSS (76.7%); for any severe mental disorder, the percentage agreement between Time 1 and Time 2 was CODSI-SMD (86.7%), MHSF (91.7%), MINI-M (88.3%), and GSS-IDS (83.3%).

DISCUSSION

SCREENING FOR ANY MENTAL DISORDER

Four instruments—the MHSF, GSS, CODSI-MD, and the MINI-M—provided reasonable and comparable overall accuracy in screening for any mental disorder. Results were similar when the data were disaggregated by gender. The MHSF showed the highest overall accuracy for females, but few sizable differences were evident between the MHSF, GSS, CODSI-MD, and the MINI-M on the sensitivity and specificity scores for either males or females. Generally, each of the four instruments did best under some circumstances, such as an emphasis on sensitivity or specificity or for a particular gender, so that the choice of an instrument will depend on the particular requirements of the user. The CODSI-MD had the advantage of being shorter than the other instruments while producing comparable results, yet the use of the previously standardized instruments may well be justified because, besides performing reasonably well in this study, they are well established, already in use in many settings, and provide other information that is often valuable in a particular situation.

SCREENING FOR SEVERE MENTAL DISORDER

When screening for a severe mental disorder (i.e., schizophrenia, bipolar disorders, and depressive disorders) and suicide-risk potential, all instruments, the MHSF, GSS-IDS, CODSI-MD, and MINI-M provided reasonable accuracy. The CODSI-SMD produced a higher sensitivity score than either the MHSF or the GSS-IDS. The CODSI-SMD ruled out 91% of those without a severe disorder and identified 51% of those with a severe disorder. Although the sensitivity of the CODSI-SMD could be substantially increased (to 84%) by using a cutoff score of 1, the specificity score would be halved (to 45%), resulting in lower overall accuracy; therefore, retaining the selected cutoff score of 2 was judged to be best for balancing the need to identify cases with the need to avoid overstressing scarce assessment resources. Attempts to add additional items to the CODSI-SMD also resulted in lower overall accuracy, as specificity was reduced to a greater extent than sensitivity was increased. One could argue that, in this case, the choice of instruments would depend on the prevalence of mental disorders in the population of interest and on the importance of identifying positive cases. Under conditions of very low prevalence in the population being screened, the MHSF and the GSS-IDS would be expected to produce better overall accuracy and, when prevalence was high, the CODSI-SMD would be expected to produce better overall accuracy.

CLINICAL UTILITY OF THE CODSI INSTRUMENTS

Clinical utility of the instruments will be determined by the prevalence rates for mental disorders in the population and the availability of diagnostic assessment and treatment resources. Generally, diagnostic assessment resources are scarce so that, when prevalence rates are low, specificity is valued for the ability to screen out individuals without the disorder. When prevalence is high and adequate diagnostic resources are available, sensitivity is preferred as a means of identifying the highest number of individuals possessing the disorder. The CODSI-MD approximates the overall accuracy of the two best-performing instruments—the MHSF and the GSS. No instrument demonstrated significantly higher sensitivity or specificity scores than the CODSI-MD in the analyses of the entire sample or in the separately reported gender findings.

The CODSI-SMD offers comparable overall accuracy to the other instruments while demonstrating the best balance between sensitivity and specificity. Thus, the CODSI-SMD identifies nearly 91% of those without a severe mental disorder as not having a disorder and 50% of those with a severe mental disorder as having the disorder. The ability to identify all cases with a disorder is particularly important when prevalence is high or when the need to identify as many of the positive cases as practicable is compelling. The three-item CODSI-SMD also provides greater efficiency in its use of screening resources; it consists of fewer items and, consequently, its response times are lower than the other instruments.

Additional analyses of the unweighted data (as differentiated from the weighted data in Tables 2 and 4) showed that the CODSI screening instruments offer users a high degree of efficiency. The CODSI-MD identified 132 individuals as having a disorder, 85% (112) of whom had a SCID diagnosis of a mental disorder. Thus, only 20 cases (15%) who would have been referred for assessment would not have had a mental disorder. The CODSI-SMD, in particular, offers users with limited assessment resources a highly efficient means of identifying those prisoners most in need of further assessment. The CODSI-SMD identified 53% (40 of 75) of the subjects who met the criteria for a severe mental disorder. Furthermore, these analyses found that all 40 (100%) of the individuals who screened positively for severe mental disorder on the CODSI-SMD met SCID criteria for a severe mental disorder. Thus, using the CODSI-SMD screener would have maximized the identification of severe mental disorders and produced little or no wasted assessment resources in identifying individuals who met the mental disorder criteria for COD.

PREVALENCE OF MENTAL DISORDERS

Although this validation study uses a convenience sample of admissions to prison substance-abuse treatment programs, the sample was geographically diverse, and the total of 141 (78.3%) admissions who were diagnosed with a mental disorder (using the SCID) was consistent with the 50% to 75% of admissions to community-based programs who are estimated to have at least some mental disorder (Compton, Cottler, Phelps, Abdallah, & Spitznagel, 2000; Havassy, Alvidrez, & Owen, 2004; Prendergast et al., 2004; Sacks, Sacks, De Leon, Bernhardt, & Staines, 1997). Furthermore, of the 141 mental disorder diagnoses, more than half were for severe mental disorders (75 of 141). Such high prevalence rates emphasize the importance of screening offenders to inform policy makers of the need for specialized services for inmates housed in prisons under their jurisdiction and to identify individuals who need further assessment to identify the potential need for specialized mental health services.

It should be further noted that the clinical utility of any screening instrument will be affected by the prevalence rates for mental disorders in the population. Although sensitivity and specificity are independent of prevalence, the overall accuracy of the screening instruments is not. For example, in a population with a high prevalence of mental disorders, sensitivity will be more influential in determining overall accuracy, whereas in a population with low prevalence of mental disorders, specificity will be more influential. In the present study population, sensitivity was favored because of the high prevalence of mental disorder, and specificity was comparatively more important in determining overall accuracy on the less prevalent severe mental disorders.

A further example of the effect of the prevalence of mental disorder can be seen in the reversal of the relationship between sensitivity and specificity scores between the two CODSI screening instruments. The CODSI-MD produced better overall accuracy for females, whereas the CODSI-SMD produced better overall accuracy for males. This may be attributed to any of several reasons, but one explanation for the difference lies in the differential prevalence rates for any mental disorder and for severe mental disorders. The higher prevalence for any mental disorder among females favors instruments with high sensitivity scores such as the CODSI-MD. On the other hand, the lower prevalence rates for severe mental disorders favors instruments that score high in specificity, such as the CODSI-SMD. In this instance the lower prevalence rate among males resulted in higher overall accuracy. In addition to gender, minority status may influence the overall accuracy of the results either because of the nature of the questions or the prevalence rates for any or for severe mental disorder(s). This issue is being addressed in an additional data collection and will be reported once that study has concluded.

LIMITATIONS

The reader should exercise caution in interpreting the generalizability of these findings beyond the specific population studied (e.g., admissions to prison substance-abuse treatment programs). High prevalence rates of mental illness resulted in relatively small sample sizes on which to estimate specificity, particularly when the data were disaggregated by gender. Furthermore, although cutpoints were established in the pilot study within this population, the standard mental-health screening instruments studied had been designed originally to screen different populations. The MHSF, for example, was created to assist community-based substance-abuse treatment programs to determine the mental health needs of program admissions, a population that was similar, but not identical, to the offender population used in the pilot and validation studies described here. The other instruments (the MINI-M and the GSS) had been created to be screening devices for use in a variety of settings. It is reasonable to assume that these instruments may perform differently in other populations or other settings, particularly where the populations are markedly dissimilar from the present sample. Another possible drawback consists of the presentation of the items in both CODSI instruments as embedded in the other instruments. It is likely that these effects were small and will be corrected in planned future studies in which the CODSI instruments will be administered as distinct and separate instruments.

FUTURE DIRECTIONS

The investigators plan a future study to assess the validity of the CODSI-MD and the CODSI-SMD in conjunction with the TCUDS as a screener for offenders entering prison. It is expected that such study will undertake analysis for the following mental disorder categories: Axis I, severe; Axis I, all; Axis II with Axis I; and Axis II.

CONCLUSION

The high prevalence rates for mental disorders among admissions to prison substance-abuse treatment programs found in the present study and others emphasizes the need for screening, assessment, and specialized treatment services in criminal justice populations. The current study provides evidence for the overall accuracy of the 6-item CODSI-MD in determining the presence of any mental disorder and demonstrates particular strength for the 3-item CODSI-SMD in determining the presence of a severe mental disorder. The CODSI instruments can be used as a basis for referring prisoners for further assessment and as a means of collecting mental health data in prison substance-abuse treatment programs. In addition, the present study can be seen as validating the use of three standardized mental-health screening instruments (MHSF, the MINI-M, and GSS) in prison substance-abuse settings and as establishing the best cutoff scores for this purpose. Overall, the CODSI instruments show sufficient value in terms of brevity and efficiency to be combined with the TCUDS to form a screening device for co-occurring mental and substance-use disorders in prison substance-abuse treatment programs and to warrant future validation studies in other criminal justice populations and settings.

Acknowledgments

AUTHORS' NOTE: This study was funded (Grant No. U01 DA16200) under a cooperative agreement from the U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, National Institute on Drug Abuse (NIH/NIDA). The authors gratefully acknowledge the collaborative contributions by federal staff from NIDA, the 10 research centers, and participating field sites in the Criminal Justice Drug Abuse Treatment Studies (CJDATS) project, as described in more detail in the introduction to this special volume of Criminal Justice and Behavior.

Footnotes

1

Although the ADS is not in the public domain, it is relatively inexpensive for bulk administration.

2

Reading facility was not a consideration as a research interviewer read all questions aloud to each subject (the instruments were available in English only).

3

During the course of the study, a new version of the GSS was released; all findings discussed here refer to version 1.0 of the GSS.

Views and opinions are those of the authors and do not necessarily reflect those of Department of Health and Human Services, the NIH, NIDA, or other participants in CJDATS. This article has not been published elsewhere nor has it been submitted simultaneously for publication elsewhere.

REFERENCES

  1. Baldassano CF. Assessment tools for screening and monitoring bipolar disorder. Bipolar Disorders. 2005;7(1):8–15. doi: 10.1111/j.1399-5618.2005.00189.x. [DOI] [PubMed] [Google Scholar]
  2. Baxter R, Rabe-Hesketh S, Parrott J. Characteristics, needs, and reoffending in a group of patients with schizophrenia formerly treated in medium security. Journal of Forensic Psychiatry. 1999;10:69–83. [Google Scholar]
  3. Blackburn R. Classification and assessment of personality disorders in mentally disordered offenders: A psychological perspective. Criminal Behaviour and Mental Health. 2000;10:S8–S33. [Google Scholar]
  4. Brady TM, Krebs CP, Laird G. Psychiatric comorbidity and not completing jail-based substance abuse treatment. American Journal on Addictions. 2004;13(1):83–101. doi: 10.1080/10550490490265398. [DOI] [PubMed] [Google Scholar]
  5. Broome KM, Knight K, Joe GW, Simpson DD. Evaluating the drug-abusing probationer: Clinical interview versus self-administered assessment. Criminal Justice & Behavior. 1996;23(4):593–606. [Google Scholar]
  6. Carroll JFX, McGinley JJ. A screening form for identifying mental health problems in alcohol/other drug dependent persons. Alcoholism Treatment Quarterly. 2001;19(4):33–47. [Google Scholar]
  7. Center for Substance Abuse Treatment . Simple screening instruments for outreach for alcohol and other drug abuse and infectious diseases. U.S. Department of Health and Human Services; Rockville, MD: 1994. (Treatment Improvement Protocol Series, No. 11). [PubMed] [Google Scholar]
  8. Compton WM, Cottler LB, Phelps DL, Abdallah AB, Spitznagel EL. Psychiatric disorders among drug dependent subjects: Are they primary or secondary? American Journal on Addictions. 2000;9:126–134. doi: 10.1080/10550490050173190. [DOI] [PubMed] [Google Scholar]
  9. Correctional Association of New York . Mental health in the House of Corrections: A study of mental health care in New York State prisons by the Correctional Association of New York. Author; New York City: 2004. [Google Scholar]
  10. Criminal Justice Drug Abuse Treatment Studies (CJDATS) Criminal Justice Drug Abuse Treatment Studies Adult Intake Interview. National Institutes of Health, Division of Epidemiology, Services and Prevention Research; Rockville, MD: 2005. Retrieved October 5, 2006, from http://cjdats.org/ka/ka-3.cfm?content_item_id=771&folder_id=23. [Google Scholar]
  11. Dennis ML. Global Appraisal of Individual Needs (GAIN): Administration guide for the GAIN and related measures. Chestnut Health Systems; Bloomington, IL: 1999. Retrieved October 24, 2006, from http://www.chestnut.org/li/gain/gadm1299.pdf. [Google Scholar]
  12. Dennis ML, Chan Y, Funk RR. Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. American Journal on Addictions. 2006;15(S1):80–91. doi: 10.1080/10550490601006055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Derogatis LR. Brief Symptom Inventory administration, scoring, and procedures manual. 3rd ed. National Computer Systems, Inc.; Minneapolis, MN: 1993. [Google Scholar]
  14. DiCataldo F, Greer A, Profit WE. Screening inmates for mental disorder: An examination of the relationship between mental disorders and prison adjustment. Bulletin of the American Academy of Psychiatry and the Law. 1995;23(4):573–585. [PubMed] [Google Scholar]
  15. Ditton PM. Mental health and treatment of inmates and probationers. Bureau of Justice Statistics; Washington, DC: 1999. Retrieved September 21, 2005, from http://www.ojp.usdoj.gov/bjs/pub/ascii/mhtip.txt. [Google Scholar]
  16. Drake RE, McHugo GJ, Clark RE, Teague GB, Xie H, Miles K, et al. Assertive community treatment for patients with co-occurring severe mental illness and substance use disorder: A clinical trial. American Journal of Orthopsychiatry. 1998;68(2):201–215. doi: 10.1037/h0080330. [DOI] [PubMed] [Google Scholar]
  17. Fazel S, Danesh J. Serious mental disorder in 23000 prisoners: A systematic review of 62 surveys. Lancet. 2002;359(9306):545–550. doi: 10.1016/S0140-6736(02)07740-1. [DOI] [PubMed] [Google Scholar]
  18. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I disorders. research version, patient edition. Biometrics Research, New York State Psychiatric Institute; New York: Nov, 2002. SCID-I/P. [Google Scholar]
  19. Harrison PM, Beck AJ. Prisoners in 2004 (Bureau of Justice Statistics Bulletin, NCJ 210677) 2005 October; Retrieved November 1, 2006, from http://www.ojp.usdoj.gov/bjs/pub/pdf/p04.pdf.
  20. Havassy BE, Alvidrez J, Owen KK. Comparisons of patients with comorbid psychiatric and substance-use disorders: Implications for treatment and service delivery. American Journal of Psychiatry. 2004;161(1):139–145. doi: 10.1176/appi.ajp.161.1.139. [DOI] [PubMed] [Google Scholar]
  21. Hiller M, Knight K, Simpson DD. Risk factors that predict dropout from corrections-based treatment for drug abuse. The Prison Journal. 1999;79:411–430. [Google Scholar]
  22. Kessler RC, Barker PR, Colpe LJ, Epstein JF, Gfroerer JC, Hiripi E, et al. Screening for serious mental illness in the general population. Archives of General Psychiatry. 2003;60:184–189. doi: 10.1001/archpsyc.60.2.184. [DOI] [PubMed] [Google Scholar]
  23. Kleinsasser LD, Michaud J. Identifying and tracking mentally ill offenders in the Colorado Department of Corrections. Presentation to Mental Health Corrections Consortium Conference; Kansas City, MO. May, 2002. [Google Scholar]
  24. Lang MA, Belenko S. Predicting retention in a residential drug treatment alternative to prison program. Journal of Substance Abuse Treatment. 2000;19:145–160. doi: 10.1016/s0740-5472(00)00097-0. [DOI] [PubMed] [Google Scholar]
  25. McLellan AT, Kushner H, Metzger D, Peters R, Smith I, Grissom G, et al. The 5th edition of the Addiction Severity Index. Journal of Substance Abuse Treatment. 1992;9:199–213. doi: 10.1016/0740-5472(92)90062-s. [DOI] [PubMed] [Google Scholar]
  26. McLellan AT, Luborsky L, Cacciola J, Griffith J, Evans F, Barr H, et al. New data from the Addiction Severity Index: Reliability and validity in three centers. Journal of Nervous and Mental Disease. 1985;173(7):412–423. doi: 10.1097/00005053-198507000-00005. [DOI] [PubMed] [Google Scholar]
  27. McLellan AT, Luborsky L, Woody GE, O'Brien CP. An improved diagnostic evaluation instrument for substance abuse patients. Journal of Nervous and Mental Disease. 1980;168:26–33. doi: 10.1097/00005053-198001000-00006. [DOI] [PubMed] [Google Scholar]
  28. Maffei C, Fossati A, Agostonid I, Barraco A, Bagnato M, Deborah D, et al. Interrater reliability and internal consistency of the Structured Clinical Interview for DSM-IV Axis II personality disorders (SCID-II), version 2.0. Journal of Personal Disorders. 1997;11(3):279–284. doi: 10.1521/pedi.1997.11.3.279. [DOI] [PubMed] [Google Scholar]
  29. Magruder KM, Sonne SC, Brady KT, Quello S, Martin RH. Screening for co-occurring mental disorders in drug treatment populations. Journal of Drug Issues. 2005;193:658–664. [Google Scholar]
  30. Melnick G, Sack S, Banks S. Predicting violent behavior among individuals with co-occurring substance use and mental disorders. Paper presented at the GAINS Conference; Boston, MA. Apr, 2006. [Google Scholar]
  31. Metz CE. Basic principles of ROC analysis. Seminars in Nuclear Medicine. 1978;8(4):283–298. doi: 10.1016/s0001-2998(78)80014-2. [DOI] [PubMed] [Google Scholar]
  32. Monahan J, Bonnie RJ, Appelbaum PS, Hyde PS, Steadman HJ, Swartz MS. Mandated community treatment: Beyond outpatient commitment. Psychiatric Services. 2001;52(9):1198–1205. doi: 10.1176/appi.ps.52.9.1198. [DOI] [PubMed] [Google Scholar]
  33. Monahan J, Steadman HJ, Appelbaum PS, Robbins PC, Mulvey EP, Silver E, et al. Developing a clinically useful actuarial tool for assessing violence risk. The British Journal of Psychiatry. 2000;176:312–319. doi: 10.1192/bjp.176.4.312. [DOI] [PubMed] [Google Scholar]
  34. Monahan J, Steadman HJ, Robbins PC, Appelbaum P, Banks S, Grisso T, et al. An actuarial model of violence risk assessment for persons with mental disorders. Psychiatric Services. 2005;56(7):810–815. doi: 10.1176/appi.ps.56.7.810. [DOI] [PubMed] [Google Scholar]
  35. Morgan DW, Edwards AC, Faulkner LR. The adaptation to prison by individuals with schizophrenia. Bulletin of the American Academy of Psychiatry and the Law. 1993;21(4):427–433. [PubMed] [Google Scholar]
  36. Nicholls TL, Lee Z, Corrado RR, Ogloff JRP. Women inmates' mental health needs: Evidence of the validity of the Jail Screening Assessment Tool (JSAT) International Journal of Forensic Mental Health. 2004;3(2):167–184. [Google Scholar]
  37. Office of the Surgeon General . Report on Mental Health. Superintendent of Documents; Washington, DC: 1999. Publication No. 017-024-01653-5. [Google Scholar]
  38. Peters R, Green Bartoi M. Screening and assessment of co-occurring disorders in the justice system. Policy Research; Delmar, NY: 1997. Monograph of the National GAINS Center for People with Co-Occurring Disorders In the Justice System. [Google Scholar]
  39. Peters RH, Greenbaum PE, Steinberg ML, Carter CR. Effectiveness of screening instruments in detecting substance use disorders among prisoners. Journal of Substance Abuse Treatment. 2000;18(4):349–358. doi: 10.1016/s0740-5472(99)00081-1. [DOI] [PubMed] [Google Scholar]
  40. Peters RH, Hills HA. Intervention strategies for offenders with co-occurring disorders: What works? National GAINS Center; Delmar, NY: 1997. Retrieved January 9, 2001, from http://www.prainc.com/s/publications/intervention.htm. [Google Scholar]
  41. Prendergast ML, Hall EA, Wexler HK, Melnick G, Cao Y. Amity prison-based therapeutic community: Five-year outcomes. The Prison Journal. 2004;84(1):36–60. [Google Scholar]
  42. Ramirez Basco MR, Bostic JQ, Davies D, Rush AJ, Witte B, Hendrickse W, et al. Methods to improve diagnostic accuracy in a community mental health setting. American Journal of Psychiatry. 2000;157(10):1599–1605. doi: 10.1176/appi.ajp.157.10.1599. [DOI] [PubMed] [Google Scholar]
  43. Sacks S, Melnick G, Coen C, Banks S, Friedmann PD, Grella C, et al. CJDATS Co-Occurring Disorders Screening Instrument for Mental Disorders (CODSI-MD): A pilot study. The Prison Journal. 2007;87(1):86–110. doi: 10.1177/0032885506299044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Sacks S, Sacks JY, De Leon G, Bernhardt AI, Staines GL. Modified therapeutic community for mentally Ill chemical abusers: Background; influences: Program description: Preliminary findings. Substance Use and Misuse. 1997;32(9):1217–1259. doi: 10.3109/10826089709035472. [DOI] [PubMed] [Google Scholar]
  45. Sacks S, Sacks JY, McKendrick K, Banks S, Stommel J. Modified TC for MICA Offenders: Crime outcomes. Behavioral Sciences & The Law. 2004;22:477–501. doi: 10.1002/bsl.599. [DOI] [PubMed] [Google Scholar]
  46. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry. 1998;59(Suppl. 20):22–33. [PubMed] [Google Scholar]
  47. Siegel S. Nonparametric statistics for the behavioral sciences. McGraw-Hill; New York: 1956. [Google Scholar]
  48. Simpson DD. Toward the year 2000: Issues in treatment process and services research. International Journal of the Addictions. 1995;30(7):875–879. doi: 10.3109/10826089509067012. [DOI] [PubMed] [Google Scholar]
  49. Simpson DD, Joe GW, Rowan-Szal GA, Greener JM. Drug abuse treatment process components that improve retention. Journal of Substance Abuse Treatment. 1997;14(6):565–572. doi: 10.1016/s0740-5472(97)00181-5. [DOI] [PubMed] [Google Scholar]
  50. Skinner HA, Horn JL. Alcohol Dependence Scale: Users guide. Addiction Research Foundation; Toronto, Canada: 1984. [Google Scholar]
  51. Steadman HJ, Mulvey EP, Monahan J, Robbins PC, Appelbaum PS, Grisso T, et al. Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry. 1998;55(5):393–401. doi: 10.1001/archpsyc.55.5.393. [DOI] [PubMed] [Google Scholar]
  52. Steadman HJ, Scott JE, Osher F, Agnese TK, Robbins PC. Validation of the Brief Jail Mental Health Screen. Psychiatric Services. 2005;56(7):816–822. doi: 10.1176/appi.ps.56.7.816. [DOI] [PubMed] [Google Scholar]
  53. Sullivan CJ, McKendrick K, Sacks S, Banks SM. American Journal of Drug and Alcohol Abuse. Modified TC for MICA offenders: Substance use outcomes. in press. [DOI] [PubMed] [Google Scholar]
  54. Swartz JA, Lurigio AJ. Screening for serious mental illness in populations with co-occurring substance use disorders: Performance of the K6 scale. Journal of Substance Abuse Treatment. 2006;31:287–296. doi: 10.1016/j.jsat.2006.04.009. [DOI] [PubMed] [Google Scholar]
  55. Teplin LA, Schwartz J. Screening for severe mental disorders in jails. Law & Human Behavior. 1989;13:1–18. [Google Scholar]
  56. Zimmerman M, Mattia JI. A self-report scale to help make psychiatric diagnoses: The Psychiatric Diagnostic Screening Questionnaire. Archives of General Psychiatry. 2001;58:787–794. doi: 10.1001/archpsyc.58.8.787. [DOI] [PubMed] [Google Scholar]

RESOURCES