Reliability and validity of severity dimensions of psychopathology assessed using the Structured Clinical Interview for DSM‐5 (SCID)

Abstract

This study examined whether the Structured Clinical Interview for DSM (SCID), a widely used semistructured interview designed to assess psychopathology categorically, can be adapted to identify reliable and valid severity dimensions of psychopathology. The present study also examined whether these severity dimensions have better psychometric properties (internal consistency, test–retest reliability, and concurrent and predictive validity) than categorical diagnoses. Participants (N = 234) were recruited from the community and clinics. Retest reliability and prospective predictive validity (symptoms and functioning 1 year later) were examined in subsamples of participants. Dimensional severity scales were created from an adapted version of the SCID for both current and lifetime major depression, alcohol, substance, post‐traumatic stress disorder, panic, agoraphobia, social anxiety, specific phobia, obsessive–compulsive disorder, and generalized anxiety disorder. The SCID's severity scales demonstrated substantial internal consistency (all Cronbach's αs >.80), test–retest reliability, and concurrent and predictive validity. Symptom severity scales demonstrated significant incremental validity over and above categorical diagnoses for both current and prospective outcomes. The psychometric properties of SCID‐identified symptom scales were far superior to the psychometrics of categorical diagnoses for both current and lifetime psychopathology. These results highlight the feasibility and utility of the SCID to assess reliable and valid symptom severity dimensions of both current and lifetime psychopathology.

1. INTRODUCTION

Although the modern Diagnostic and Statistical Manuals of Mental Disorders (DSM; post‐1980) have provided stakeholders of mental health services (researchers, clinicians, patients, etc.) a common nomenclature to describe psychopathology, the system has received numerous critiques (Thyer, 2015; Wakefield, 2016; Widiger & Clark, 2000). One of the major critiques of the DSM is that it conceptualizes disorders as discrete (i.e., categorical) entities in which hard cut points are used to classify individuals as either “with” or “without” the disorder (Helzer, Kraemer, & Krueger, 2006; Insel et al., 2010). Indeed, numerous taxometric and simulation studies have shown that most psychopathologies are not discrete entities and are better conceptualized as continuous (i.e., dimensional) constructs (Haslam, Holland, & Kuppens, 2012; Ruscio & Marcus, 2007) or perhaps hybrids of dimensional and categorical constructs (Muthen, 2006). Additionally, a meta‐analysis reported that compared to when psychopathology is assessed categorically, psychopathology defined dimensionally has 15% greater reliability and 37% greater validity (Markon, Chmielewski, & Miller, 2011).

Despite these psychometric advantages, the best method to assess psychopathology dimensionally is lacking. Most measures designed to assess psychopathology dimensionally are paper‐and‐pencil questionnaires that either only assess current psychopathology (e.g., a depression scale that assesses symptoms during previous 2 weeks) or broader traits (e.g., personality disorder measures such as the Personality Inventory for DSM‐5; Krueger, Derringer, Markon, Watson, & Skodol, 2012). Compared to paper‐and‐pencil questionnaires, semistructured interviews such as the Structured Clinical Interview for DSM (SCID; First, Williams, Karg, & Spitzer, 2015) allow for the probing and clarification of responses and are not as impacted by respondents' reading level, an important concern of many questionnaires (Schinka, 2012). Although there are several interviews designed to assess one or more dimensions of a single area of psychopathology (e.g., Positive and Negative Syndrome Scale for Schizophrenia; Kay, Fiszbein, & Opler, 1987), there is no semistructured interview designed to assess a broad array of psychopathology both categorically and dimensionally and for both current and past symptoms. Some researchers have used semistructured interviews such as the SCID or Anxiety Disorders Interview Schedule (ADIS‐IV; Brown, DiNardo, & Barlow, 1994) to assess psychopathology dimensionally, but they have only focused on a single symptom domain (Abela, Skitch, Auerbach, & Adams, 2005) and have not examined past, as well as current, psychopathology (Brown, Chorpita, & Barlow, 1998; Kotov, Gamez, & Watson, 2005). Additionally, and most importantly, it is unknown whether these semistructured interviews yield symptom severity dimensions that are reliable and valid. Having one instrument that can assess an array of psychopathology domains dimensionally would also be useful for studies adopting NIMH's Research Domain Criteria perspective that encourages a dimensional conceptualization of psychopathology (Sanislow et al., 2010).

The purpose of this study is therefore to (a) adapt a widely used categorical assessment interview of adult psychopathology, the SCID, so that it can also assess the severity of psychopathology dimensionally and (b) compare the reliability and validity of SCID‐assessed dimensions versus SCID‐assessed categories for both current and lifetime psychopathology. We chose to use the SCID for the present study rather than create an entirely new interview as the SCID is the most widely used semistructured diagnostic interview for adult psychopathology and its adaptation would not require additional training beyond what SCID users typically receive (SCID‐101, 1998).

2. METHOD

2.1. Participants

Two hundred thirty‐four participants were drawn from an ongoing study on neurophysiological vulnerability factors for psychopathology (Gorka et al., 2016; Weinberg, Liu, Hajcak, & Shankman, 2015). The study recruited participants from the community and area mental health clinics (via fliers, Internet postings, etc.) and aimed to obtain a sample of young adults with a wide range of psychiatric diagnoses and symptoms as well as healthy controls. Thus, the inclusion criteria were broad, only requiring age between 18 and 30 years and having at least one full biological sibling within the same age range who was eligible to enroll. Participants were excluded from the larger study if they had a personal or family history of psychosis or mania; were unable to read or write English; had a history of head trauma with loss of consciousness; or were left‐handed. These criteria were included to ensure participants were able to provide consent and to mitigate potential confounds to psychophysiological data. Two hundred thirty‐four sibling pairs were assessed, but to avoid nonindependence of observations, data from only one sibling per family were used for the present analyses. In order to maximize the prevalence of psychopathology, the sibling with the higher sum of lifetime ADIS‐IV ratings across the assessed psychopathologies (see below) was selected for the present analyses. Randomly selected subsamples of these 234 individuals were used for the retest reliability (n = 51) and prospective predictive validity (n = 71) studies (see below). This investigation was carried out in accordance with the latest version of the Declaration of Helsinki, and all participants provided written informed consent after review of the procedures as approved by the university Institutional Review Board. Participants were paid approximately $20/hour for their participation. Demographic characteristics for these two subsamples and the overall sample are described in Table 1.

Table 1.

Sample characteristics

	Whole sample (N = 234)	Test–retest sample (N = 51)	Predictive validity sample (N = 71)
Age (M, SD)	22.46 (3.19)	22.27 (3.20)	22.23 (3.36)
Gender
Male	76 (32.5%)	21 (41.2%)	24 (33.8%)
Female	158 (67.5%)	30 (58.8%)	47 (66.2%)
Ethnicity
Caucasian	96 (41.0%)	21 (41.2%)	29 (40.8%)
Hispanic	53 (22.6%)	6 (11.8%)	19 (26.8%)
African American	34 (14.5%)	12 (23.5%)	9 (12.7%)
Asian	25 (10.7%)	5 (9.8%)	8 (11.3%)
Middle eastern	10 (4.3%)	1 (2.0%)	2 (2.8%)
Mixed race	14 (6.0%)	5 (9.8%)	3 (4.2%)
Other	2 (.9%)	1 (2.0%)	1 (1.4%)
Overall symptom severity and functioning
Mean GAF (M, SD)	68.06 (14.00)	67.44 (10.88)	68.38 (16.04)
Mean SOFAS (M, SD)	70.80 (13.36)	70.54 (10.77)	70.85 (14.69)
Education
Did not graduate high school	10 (4.2%)	1 (2.0%)	2 (2.8%)
Graduated high school (or equivalent)	10 (4.3%)	6 (11.8%)	2 (2.8%)
Some college	120 (51.3%)	24 (47.1%)	44 (62.0%)
Graduated 2 years in college	12 (5.1%)	5 (9.8%)	0 (.0%)
Graduated 4 years in college	43 (18.4%)	11 (21.6%)	15 (21.1%)
Some graduate or professional school	28 (12.0%)	4 (7.8%)	5 (7.0%)
Completed graduate or professional school	10 (4.3%)	0 (.0%)	3 (4.2%)
Employment status
Full time	59 (25.2%)	10 (19.6%)	19 (26.8%)
Part time	55 (23.5%)	11 (21.6%)	23 (32.4%)
Student	106 (45.3%)	24 (47.1%)	26 (36.6%)
Unemployed	14 (6.0%)	6 (11.8%)	3 (4.2%)
Lifetime psychiatric hospitalization	17 (7.3%)	6 (11.8%)	8 (11.3%)
Lifetime psychiatric treatment	99 (42.3%)	20 (39.2%)	32 (45.1%)

Note. GAF = Global Assessment of Functioning. SOFAS = Social and Occupational Functioning Assessment Scale.

2.2. Baseline measures

2.2.1. Structured clinical interview for DSM‐5

Lifetime and current psychopathology were assessed by interviewers using an adapted version of the research version of the SCID (First et al., 2015). Specifically, the following 10 disorders were assessed—major depressive disorder (MDD), alcohol use disorder (AUD), substance use disorder (SUD), post‐traumatic stress disorder (PTSD), panic disorder (PD), agoraphobia, social anxiety disorder (SAD), specific phobia, obsessive–compulsive disorder (OCD), and generalized anxiety disorder (GAD; see Supporting Information for results for the anorexia, bulimia, and binge eating disorder modules, which were also administered, but are not reported in the main analyses due to small N). As personal history of psychosis or mania was an exclusion criterion, lifetime psychosis and mania were assessed using the psychotic screening module and manic episode section from the SCID's Mood Disorders module. The SCID for the present study was identical to the SCID‐5, with the following exceptions. First, the instrument used a slightly different structure than the SCID‐5. The SCID‐5 sometimes assesses lifetime diagnostic criteria and then current criteria for some disorder modules, and sometimes does current and then lifetime (only if the current diagnosis was not met). To aid in the symptom severity assessment of psychopathology, each module of the adapted SCID was organized to always assess lifetime symptoms first and then current symptoms. Second, the separate parts of multicomponent symptoms were coded independently (e.g., the MDD symptom “worthlessness or guilt” [Symptom 7] yielded separate ratings for worthlessness and guilt). Third, to increase sensitivity to individuals with subthreshold psychopathology and facilitate the calculation of symptom severity scales, we modified some of the skip‐out rules in the SCID. Specifically, interviewers ignored all but the first “skip out” for all disorders except MDD and GAD. For example, for PTSD, if a subject received either a 2 or 3 for Criterion A (exposure to a trauma), interviewers ignored all subsequent “skip outs” and assessed all lifetime PTSD symptoms even if the subject did not fully meet criteria B, C, D, and/or E (e.g., they only had one symptom from Criterion D). However, if a subject received a “1” for Criterion A for PTSD (i.e., they never experienced a trauma), the rest of the PTSD symptoms were not assessed as it would not make sense to assess a person's reactions to a trauma if they never experienced a trauma. Using PD as another example, it would not make sense to assess whether a person worried about additional attacks (Criterion B) if they never had a panic attack (Criterion A). Interviewers also ignored the first “skip out” at the beginning of the MDD and GAD sections because MDD and GAD contain symptoms whose presence are not dependent on the cardinal symptom(s). For example, all of the symptoms of MDD were assessed even if the cardinal symptoms (depressed mood and anhedonia) were not present. Fourth, additional items related to previous course of illness (e.g., duration of longest episode and time since last symptomatic) were also added to each module (data not included in this report). Fifth, the time frames for current psychopathology were modified for some disorders. Rather than assessing current agoraphobia, SAD, and specific phobia using the past 6 months and assessing current AUD and SUD using the past 12 months (as specified in the SCID‐5), we elected to use the past 1 month to assess these current disorders. This modification allowed for the assessment of current symptoms using the same time window as our measures of current functioning (e.g., World Health Organization Disability Assessment Schedule [WHODAS], Global Assessment of Functioning [GAF], and Social and Occupational Functioning Assessment Scale [SOFAS]), thus facilitating concurrent validity analyses of current psychopathology. Sixth, although the SCID‐5 includes questions assessing functional impairment and perceived distress associated with some disorders, several disorders do not include clinically significant impairment or distress as a criterion and do not have questions assessing functional impairment and distress due to symptoms of the specific disorder. Therefore, prompts and rating scales from the ADIS‐IV (Brown et al., 1994) were added for each disorder, and for both lifetime and current psychopathology. This provided measures of disorder‐specific impairment and distress that were consistent across disorders, thereby facilitating analyses of concurrent validity. These ratings consisted of separate assessments of perceived distress and three forms of impairment (social, occupational, and “other” impairment that resulted from the specific psychopathology) made along a 9‐point scale ranging from 0 (none) to 8 (severe), with (as per ADIS‐IV convention) ratings of 2 or higher signifying clinically significant distress or impairment. To provide measures of overall symptom severity and functioning, the GAF Scale (American Psychiatric Association, 1994) was modified slightly to only focus on overall symptom severity at the time of the interview and the SOFAS (Goldman, Skodol, & Lave, 1992) was used to focus solely on overall functional impairment due to current psychopathology. This is consistent with previous recommendations that the GAF be divided into two scales (Aas, 2010).

Interviewers were trained to criterion by viewing the SCID‐101 training videos (SCID‐101, 1998), observing two or three SCID interviews with an experienced interviewer, and completing three SCID interviews (observed by the first author or an advanced interviewer) in which diagnoses were in full agreement with those of the observer.

A subset of participants (n = 51) completed a second SCID with a different interviewer blind to the first interview within 3 weeks of their first SCID (M = 8.51 days, SD = 4.31) to assess the test–retest reliability of symptom dimensions and categorical diagnoses.

2.2.2. World Health Organization Disability Assessment Schedule 2.0

To assess more global disability and impairment, the 36‐item WHODAS 2.0 (Ustün et al., 2010) interview was administered immediately after the SCID. The 36‐item WHODAS is the gold standard assessment of disability published by the World Health Organization and contains six domains of current functioning (cognition, mobility, self‐care, getting along, life activities, and participation), which are calculated by summing the values for the items in each domain. These domain level scores have previously shown excellent internal consistency and test–retest reliability, as well as concurrent, construct, and discriminant validity (Ustün et al., 2010). In the present study, Cronbach's αs for the WHODAS domains ranged from .81–.92.

2.3. One‐year follow‐up assessment

A subset of participants (n = 71) completed the Longitudinal Interval Follow‐up Evaluation (LIFE) interview (the gold standard assessment of longitudinal symptom assessment) and 12‐item WHODAS interview 1 year after their initial SCID interview (data were only available for a subset as data collection is ongoing). The 12‐item WHODAS interview retrospectively assessed impairment since baseline. It accounts for 81% of the variance of the 36‐item version (Ustün et al., 2010), but only produces an overall disability score and not domain level scores. WHODAS impairment ratings were obtained for each month between the baseline assessment and the follow‐up.

The LIFE (Keller et al., 1987) interview was administered face‐to‐face (n = 65) or over the phone (n = 6) at 1‐year follow‐up to retrospectively assess symptoms of psychopathology since baseline. For each week since baseline, the LIFE yields a measure of symptom severity called a psychiatric status rating for each disorder assessed at baseline, ranging from 1 (absent) to 6 (severe), with ratings of 5 or 6 signifying that the full diagnostic criteria for the disorder are met. The LIFE has exhibited good to excellent interrater reliability and excellent test–retest reliability over a 1‐year period (Keller et al., 1987; Warshaw, Keller, & Stout, 1994).

2.4. Data analysis

We measured lifetime and current psychopathology using both categorical diagnoses and symptom severity scales. After all interviews were completed, symptom severity scales were electronically computed by summing each symptom within a disorder, with each symptom rated on a 1–3 scale (except for the percentage time symptom from OCD). The severity scales included the following symptoms : MDD—9 core symptoms; AUD and SUD—11 core symptoms; PD—Symptoms A, B1, and B2; PTSD—Symptom A, the 20 symptoms in Criteria B through E; agoraphobia—the 5 potentially feared situations under Symptoms A to and E; SAD and specific phobia (scored separately)—Symptoms A to E; OCD—Symptoms A1, A2, B1, B2, and the percentage of time spent engaging in OCD thoughts or behaviors (all z‐scored); GAD—Symptoms A, B, and the six symptoms under Symptom C; Anorexia Nervosa (AN)—Symptoms A to C; Bulimia Nervosa (BN)—Symptoms A1, A2, B, C, and D; Binge‐Eating Disorder (BED)—Symptoms A1 and A2, the five symptoms under Criterion B, and Symptoms C and D.

Internal consistencies of each symptom severity scale were determined using Cronbach's αs. One‐way random single‐measure intraclass correlation coefficients were used to assess the test–retest reliability of severity scales, and Cohen's kappas were used to assess test–retest reliability of categorical diagnoses. For these reliability analyses, we used Shrout's (1998) standards for determining strength of reliabilities—although these standards are not without controversy (Kraemer, Kupfer, Clarke, Narrow, & Regier, 2012), specifically, 0.00–0.10 (virtually none), 0.11–0.40 (slight), 0.41–0.60 (fair), 0.61–0.80 (moderate), 0.81–1.0 (substantial). For these analyses, there were nine sibling pairs in the test–retest reliability subsample (but not the baseline [n = 234] or predictive validity [N = 71] sample). The inclusion of sibling pairs in the test–retest reliability subsample infringes on the statistical assumption that observations are independent. However, results were comparable in a smaller sample using only one sibling from each pair.

Concurrent validity for lifetime and current psychopathology was assessed using correlations between each diagnosis or symptom severity scale (point‐biserial and Pearson correlations, respectively) and the corresponding ADIS ratings, with the idea that assessments of psychopathology should be associated with functional impairment. Concurrent validity was also assessed with the domain level scores of the 36‐item WHODAS and the GAF current symptom severity and SOFAS current impairment ratings. The key analyses regarding concurrent validity tested incremental validity of symptom dimensions over and above categorical diagnoses. To test this, we computed multiple regressions with both symptom severity scales and categorical diagnoses as simultaneous predictors of the various validators (ADIS ratings, etc.).

The predictive validity of each diagnosis and severity scale was assessed via (a) the maximum LIFE severity rating for the corresponding disorder over the 1‐year follow‐up, and (b) the overall impairment index from the 12‐item WHODAS across the follow‐up (specifically, the average of the maximum rating of each of the 12 items across the follow‐up). These measures were used to measure longitudinal risk with the assumption that SCID assessments should be associated with longitudinal risk for psychopathology. Although the relationships between SCID assessments and the maximum LIFE severity rating are likely influenced by the persistence of each disorder, the specific disorders' persistence were equated in the key analyses that examined whether dimensional and categorical assessments (entered simultaneously in a multiple regression) contributed incremental validity over and above each other.

3. RESULTS

Table 2 presents the internal consistency and test–retest reliability for categorical and dimensional severity assessments of psychopathology. Using Shrout's (1998) conventions for judging reliability coefficients, the internal consistencies for the 20 dimensional assessments of lifetime and current psychopathologies were all in the substantial range (above .80) with the sole exception of current AUD (Cronbach's α = .78 [moderate range]). For the test–retest reliabilities, all 10 of the dimensional assessments for lifetime psychopathology were in the substantial range, whereas kappas for categorical diagnoses demonstrated poorer reliability (2 were substantial, 4 were moderate, 2 were fair, and 1 was slight). A similar pattern emerged for test–retest reliability of current psychopathology with 9 of the 10 dimensional severity assessments demonstrating substantial reliability (the 10th was moderate). Test–retest reliability was poorer for current categorical diagnoses (none were substantial, 4 were moderate, 2 were fair, and 1 was slight).

Table 2.

Internal consistency (n = 234) and test–retest reliability of current and lifetime psychopathology (N = 51)

	Cronbach's α (n = 234)	Test–retest reliability (ICCs [95% CI] for Sx severity scales; kappas [95% CI] for categorical Dx—n = 51)	# of participants diagnosed in Time 1	# of participants diagnosed in Time 2
Lifetime MDD
Categorical Dx	‐	κ = .69 [.49, .89]	26	28
Sx severity	.91	.91** [.84, .95]
Current MDD
Categorical Dx	‐	κ = .74 [.46, 1.00]	8	5
Sx severity	.91	.85** [.76, .91]
Lifetime AUD
Categorical Dx	‐	κ = .84 [.68, .99]	21	19
Sx severity	.88	.79** [.76, .91]
Current AUD
Categorical Dx	‐	κ = .54 [.08, 1.00]	3	4
Sx severity	.78	.38** [.13, .59]
Lifetime SUD
Categorical Dx	‐	κ = .87 [.73, 1.00]	19	18
Sx severity	.91	.92** [.87, .96]
Current SUD
Categorical Dx	‐	κ = .64 [.26, 1.00]	5	4
Sx severity	.89	.76** [.61, .85]
Lifetime PTSD
Categorical Dx	‐	κ = .65 [.20, 1.00]	2	4
Sx severity	.94	.85** [.75, .91]
Current PTSD
Categorical Dx	‐	Cannot be computed because one set had no one diagnosed	1	0
Sx severity	.89	.73** [.57, .84]
Lifetime panic
Categorical Dx	‐	κ = .46 [.12, .79]	9	6
Sx severity	.86	.74** [.58, .84]
Current panic
Categorical Dx	‐	Cannot be computed because one set had no one diagnosed	1	0
Sx severity	.85	.12 [−.16, .38]
Lifetime agoraphobia
Categorical Dx	‐	Cannot be computed because one set had no one diagnosed	2	0
Sx severity	.91	.63** [.44, .77]
Current agoraphobia
Categorical Dx	‐	Cannot be computed because one set had no one diagnosed	1	0
Sx severity	.92	.31* [.04, .53]
Lifetime social anxiety
Categorical Dx	‐	κ = .18 [−.15, .43]	13	14
Sx severity	.97	.61** [.40, .75]
Current social anxiety
Categorical Dx	‐	κ = .29 [−.10, .68]	6	5
Sx severity	.98	.58** [.36, .73]
Lifetime specific phobia
Categorical Dx	‐	κ = .71 [.44, .97]	10	6
Sx severity	.97	.72** [.55, .83]
Current specific phobia
Categorical Dx	‐	κ = .70 [.38, 1.00]	7	4
Sx severity	.97	.77** [.63, .86]
Lifetime OCD
Categorical Dx	‐	κ = .49 [.11, .88]	5	6
Sx severity	.92	.61** [.40, .76]
Current OCD
Categorical Dx	‐	κ = .64 [.26, 1.00]	4	5
Sx severity	.88	.57** [.35, .73]
Lifetime GAD
Categorical Dx	‐	κ = .77 [.51, 1.00]	7	8
Sx severity	.94	.79** [.67, .88]
Current GAD
Categorical Dx	‐	κ = .56 [.18, .94]	4	6
Sx severity	.95	.76** [.61, .85]

Note. Sx = symptoms; Dx = diagnosis; MDD = major depressive disorder, AUD = alcohol use disorder; SUD = substance use disorder, PTSD = post‐traumatic stress disorder, OCD = obsessive–compulsive disorder; GAD = generalized anxiety disorder. Hyphens signify that a statistic was not applicable.

^* p < .05.

^** p < .01.

Table 3 shows that both dimensional severity scales and categorical diagnoses of lifetime psychopathology were associated with significant concurrent validity for all validators. Table 3 also shows the results for multiple regression models in which both dimensional severity and categorical assessments of psychopathology were predictors. The symptom severity scales remained significant for 29 of the 30 regression models, whereas the categorical assessment was only significant for 8 of the 30 models. Additionally, symptom severity scales had larger independent effects than categorical diagnoses for 27 of the 30 models. There was a similar pattern for the concurrent validity of the current psychopathology assessment (see Table 4), albeit slightly weaker compared to lifetime psychopathology given the lower levels of current psychopathology. Specifically, severity scales remained significant for 66 of the 90 (73%) multiple regression models where both dimension severity and categorical assessments were entered simultaneously, compared to only 15 of the categorical effects (15 of 90 [16.7%]). Dimensional assessments had larger effects than categorical diagnoses for 84 of the 90 models (93%).

Table 3.

Concurrent validity for lifetime psychopathology (N = 234)

	Lifetime ADIS social	Lifetime ADIS occupational	Lifetime ADIS distress
Lifetime MDD
Categorical Dx (prev = 46.2%)	−.02 (.66***)	−.09 (.58***)	−.12* (64***)
Sx severity	.83*** (.81***)	.82*** (.74***)	.92*** (.82***)
Lifetime AUD
Categorical Dx (prev = 32.1%)	−.14* (.52***)	.18** (.69***)	−.12 (.48***)
Sx severity	.80*** (.69***)	.62*** (.77***)	.73*** (.63***)
Lifetime SUD
Categorical Dx (prev = 26.1%)	.04 (.64***)	.00 (.65***)	.04 (.54***)
Sx severity	.66*** (.71***)	.73*** (.73***)	.57*** (.60***)
Lifetime PTSD
Categorical Dx (prev = 12.0%)	.00 (.63***)	.04 (.57***)	.03 (.63***)
Sx severity	.83*** (.83***)	.70*** (.72***)	.78*** (.81***)
Lifetime panic
Categorical Dx (prev = 11.1%)	−.03 (.69***)	.03 (.66***)	−.01 (.73***)
Sx severity	.84*** (.82***)	.73*** (.76***)	.87*** (.86***)
Lifetime agoraphobia
Categorical dx (prev = 2.1%)	−.01 (.52***)	−.03 (.46***)	−.05 (.52***)
Sx severity	.85*** (.84***)	.79*** (.77***)	.92*** (.89***)
Lifetime social anxiety
Categorical Dx (prev = 21.4%)	.18*** (.56***)	.11 (.52***)	.08* (.61***)
Sx severity	.56*** (.68***)	.62*** (.69***)	.77*** (.83***)
Lifetime specific phobia
Categorical Dx (prev = 23.5%)	.24*** (.47***)	.38*** (.44***)	.36*** (.70***)
Sx severity	.32*** (.49***)	.08 (.36***)	.47*** (.73***)
Lifetime OCD
Categorical Dx (prev = 9.8%)	.41*** (.66***)	.08 (.48***)	.48*** (.75***)
Sx severity	.32*** (.64***)	.50*** (.57***)	.33*** (.72***)
Lifetime GAD
Categorical Dx (prev = 14.5%)	−.111** (.47***)	−.07 (.50***)	−.14*** (.48***)
Sx severity	.89*** (.82***)	.87*** (.82***)	.96*** (.87***)

Note. In each cell, zero‐order correlations are presented inside the bracket and standardized betas from multiple regressions are presented outside of bracket. Betas are adjusted for other assessment (i.e., categorical adjusted for severity and severity adjusted for categorical). Cat = categorical; prev = prevalence; MDD = major depressive disorder; AUD = alcohol use disorder; SUD = substance use disorder; PTSD = post‐traumatic stress disorder; OCD = obsessive–compulsive disorder, ADIS = Anxiety Disorders Interview Schedule; Sx = symptoms; Dx = diagnosis.

^* p < .10.

^** p < .05.

^*** p < .01.

Table 4.

Concurrent validity for current psychopathology (N = 234)

	Current ADIS social	Current ADIS occupational	Current ADIS distress	GAF symptoms	SOFAS impairment	WHODAS cognition	WHODAS getting along	WHODAS life activities	WHODAS participation
Current MDD
Cat. Dx (prev = 8.1%)	−.41*** (.50***)	−.38*** (.49***)	−.31*** (.57***)	.19* (−.31***)	.21** (−.31***)	−.08 (.35***)	−.20* (.23***)	−.23** (.27***)	−.19* (.33***)
Sx Sev.	1.13*** (.80***)	1.08***(.77***)	1.09***(.84***)	−.62*** (−.46***)	−.64*** (−.47***)	.53*** (.46***)	.54*** (.38***)	.63*** (.44***)	.65*** (.50***)
Current AUD
Cat. Dx (prev = 6.0%)	−.03 (.38***)	.44*** (.55***)	−.26*** (.47***)	−.07 (−.12*)	.01 (−.10)	.07 (.07)	−.06 (.07)	−.12 (.07)	−.06 (.03)
Sx Sev.	.48*** (.45***)	.13 (.51***)	.85*** (.63***)	−.06 (−.12*)	−.13 (−.12*)	−.01 (.06)	.15 (.10)	.22* (.12*)	.10 (.05)
Current SUD
Cat. Dx (prev = 6.8%)	−.01 (.53***)	.02 (.63***)	−.14 (.51***)	.08 (−.16**)	.13 (−.13**)	.02 (.13*)	−.24* (.09)	−.28** (.06)	−.26** (.00)
Sx Sev.	.64*** (.63***)	.70*** (.72***)	.75*** (.63***)	−.38*** (−.29**)	−.30** (−.20***)	.13 (.14**)	.38*** (.18***)	.40*** (.16**)	.30** (.08)
Current PTSD
Cat. Dx (prev = 2.1%)	.10* (.52***)	.06 (.40***)	−.22*** (.31***)	.04 (−.20***)	.10 (−.16**)	−.06 (.05)	−.06 (.05)	−.04 (.14**)	.01 (.15**)
Sx Sev.	.66*** (.73***)	.55*** (.58***)	.84*** (.70***)	−.35*** (−.30***)	−.40*** (−.34***)	.17** (.13**)	.18** (.14**)	.28*** (.26***)	.22*** (.23***)
Current panic
Cat. Dx (prev = 3.8%)	.16*** (.74***)	.41*** (.82***)	.16** (.69***)	.19* (−.24***)	.14 (−.22***)	−.01 (.12*)	.06 (.21***)	.12 (.20***)	.09 (.23***)
Sx Sev.	.72*** (.84***)	.51*** (.84***)	.66*** (.79***)	−.52*** (−.37***)	−.43*** (−.32***)	.17 (.16**)	.18 (.23***)	.10 (.19***)	.17 (.24***)
Current agoraphobia
Cat. Dx (prev = 1.3%)	−.07 (.43***)	−.24*** (.29***)	−.16*** (.42***)	.14* (−.10)	.10 (−.12*)	−.09 (.07)	−.10 (.08)	.00 (.06)	.06 (.12*)
Sx Sev.	.84*** (.80***)	.89*** (.74***)	.97*** (.88***)	−.41*** (−.32***)	−.36*** (−.31***)	.26*** (.21***)	.30*** (.24***)	.10 (.10)	.10 (.14**)
Current social anxiety
Cat. Dx (prev = 12.0%)	.15** (.56**)	.15** (.57***)	.18*** (.66***)	−.05 (−.25***)	−.14* (−.32***)	.07 (.19***)	.00 (.25***)	−.03 (.09)	−.06 (.11*)
Sx Sev.	.62*** (.72***)	.62*** (.72***)	.73*** (.84***)	−.30*** (−.34***)	−.27*** (−.36***)	.18** (.23***)	.37*** (.28***)	.17* (.16**)	.26*** (.22***)
Current spec. Phobia
Cat. Dx (prev = 16.2%)	.29*** (.49***)	.31*** (.47***)	.39*** (.69***)	−.19* (−.32***)	−.14 (−.26*)	.16* (.18***)	.13 (.17**)	.19** (.18***)	.13 (.14**)
Sx Sev.	.29*** (.49***)	.23*** (.44***)	.43*** (.70***)	−.18** (−.31***)	−.17* (−.27***)	.03 (.14**)	.05 (.14**)	−.01 (.12*)	.01 (.10)
Current OCD
Cat. Dx (prev = 6.8%)	.62*** (.40***)	.46*** (.07)	.64*** (.22***)	−.25*** (−.04)	−.20*** (−.01)	.04 (−.04)	.11 (.04)	.13* (.04)	.16** (.12*)
Sx Sev.	.51*** (.05)	.53*** (.30***)	.68*** (.36***)	−.29*** (−.14*)	−.25*** (−.15**)	.08 (.07)	.11 (.04)	.14** (.05)	.10 (−.03)
Current GAD
Cat. Dx (prev = 6.4%)	−.08* (.47***)	.02 (.55***)	−.09** (.50***)	.01 (−.28***)	.05 (−.25***)	.01 (.27***)	.13* (.32***)	−.04 (.18***)	.04 (.29***)
Sx Sev.	.89*** (.84***)	.87*** (.88***)	.96*** (.91***)	−.49*** (−.48***)	−.49*** (−.46***)	.42*** (.42***)	.32*** (.40***)	.35*** (.33***)	.41*** (.43***)

Note. In each cell, zero‐order correlations are presented inside the bracket and betas adjusted for other assessment are presented outside the bracket (i.e., categorical adjusted for severity and severity adjusted for categorical). Cat = categorical; Sev = severity; prev = prevalence; MDD = major depressive disorder; AUD = alcohol use disorder; SUD = substance use disorder; PTSD = post‐traumatic stress disorder; OCD = obsessive–compulsive disorder; ADIS = Anxiety Disorders Interview Schedule; GAF = Global Assessment of Functioning; WHODAS = World Health Organization Disability Assessment Schedule; Sx = symptoms; Dx = diagnosis.

^* p < .10.

^** p < .05.

^*** p < .01.

Table 5 presents the predictive validity of categorical and dimensional severity assessments. Categorical and dimensional assessments for lifetime and current psychopathology prospectively predicted higher symptoms of the corresponding disorder over the course of the 1‐year follow‐up. Table 5 also shows the results for multiple regression models in which both dimensional and categorical assessments were included. Dimensional severity assessments demonstrated stronger incremental predictive validity for 17 of 20 lifetime psychopathology models and 16 of 18 current psychopathology models. Fewer symptom severity scales and categorical diagnoses prospectively predicted WHODAS functioning. However, when both dimensional severity and categorical assessments were included in the same model, only dimensional (and never categorical) assessments demonstrated significant incremental predictive validity for functioning.

Table 5.

Current and lifetime psychopathology predicting symptoms and functioning at 1‐year follow‐up (predictive validity)

	Max PSR over 1 year	WHODAS (meanmaxs) at 1‐year FU
Lifetime MDD
Categorical Dx (prev. = 45.1%)	.12 (.48***)	−.19 (.28**)
Dimensional Sx	.44** (.54***)	.57*** (.42***)
Current MDD
Categorical Dx (prev. = 8.5%)	−.11 (.40***)	−.01 (.31***)
Sx severity	.62*** (.54***)	.41** (.40***)
Lifetime AUD
Categorical Dx (prev. = 35.2%)	−.09 (.32***)	−.20 (−.11)
Sx severity	.40** (.42***)	.11 (−.06)
Current AUD
Categorical Dx (prev. = 9.9%)	.27 (.54***)	.08 (−.07)
Sx severity	.31 (.54***)	−.18 (−.10)
Lifetime SUD
Categorical Dx (prev. = 23.9%)	−.05 (.57***)	−.08 (.03)
Sx severity	.69*** (.65***)	.12 (.05)
Current SUD
Categorical Dx (prev. = 11.3%)	.22 (.64***)	.24 (.01)
Sx severity	.49*** (.68***)	−.27 (−.06)
Lifetime PTSD
Categorical Dx (prev. = 12.7%)	.25* (.62***)	.34* (.25**)
Sx severity	.47*** (.67***)	−.12 (.14)
Current PTSD
Categorical Dx (prev. = 2.8%)	−.08 (.43***)	−.18 (−.02)
Sx severity	.81*** (.76***)	.26 (.14)
Lifetime panic
Categorical Dx (prev. = 12.7%)	−.16 (.38***)	−.26 (.25**)
Sx severity	.63*** (.49***)	.61*** (.38***)
Current panic
Categorical Dx (prev. = 4.2%)	−.14 (.45***)	−.01 (.04)
Sx severity	.73*** (.62***)	.06 (.05)
Lifetime agoraphobia
Categorical Dx (prev. = 1.4%)	−.37*** (−.01)	−.21 (−.05)
Sx severity	.58*** (.35***)	.26* (.13)
Current agoraphobia
Categorical Dx (prev. = 0.0%)	N/A	N/A
Sx severity	N/A (.54***)	N/A (−.03)
Lifetime social anxiety
Categorical Dx (prev. = 18.3%)	.40*** (.65***)	.08 (.23*)
Sx severity	.37*** (.65***)	.23 (.28**)
Current social anxiety
Categorical Dx (prev. = 9.9%)	.11 (.63***)	.07 (.28**)
Sx severity	.77*** (.85***)	.31** (.36***)
Lifetime specific phobia
Categorical Dx (prev. = 21.1%)	.22 (.52***)	−.03 (.10)
Sx severity	.41*** (.57***)	.17 (.15)
Current specific phobia
Categorical Dx (prev. = 15.5%)	.29*** (.66***)	.10 (.09)
Sx severity	.54*** (.74***)	−.02 (.05)
Lifetime OCD
Categorical Dx (prev. = 12.7%)	−.18 (.48***)	.12 (.36***)
Sx severity	.83*** (.68***)	.31 (.40***)
Current OCD
Categorical Dx (prev. = 11.3%)	−.06 (.51***)	.24 (.29**)
Sx severity	.72*** (.67***)	.06 (.26**)
Lifetime GAD
Categorical Dx (prev. = 14.1%)	−.02 (.46***)	.09 (.27**)
Sx severity	.73*** (.72***)	.28* (.34***)
Current GAD
Categorical Dx (prev. = 9.9%)	.02 (.48***)	.13 (.31***)
Sx severity	.76*** (.77***)	.31** (.39***)

Note. In each cell, zero‐order correlations are presented inside of bracket and betas adjusted for the other assessment are presented outside the bracket (i.e., categorical adjusted for severity, and severity adjusted for categorical). Dx = diagnosis; Sx = symptoms; prev = prevalence; MDD = major depressive disorder; AUD = alcohol use disorder; SUD = substance use disorder; PTSD = post‐traumatic stress disorder; OCD = obsessive–compulsive disorder; PSR = psychiatric status rating (from the LIFE interview) for the psychopathology in the respective row; WHODAS = World Health Organization Disability Assessment Schedule; Sx = symptoms; Dx = diagnosis.

^* p < .10,

^** p < .05,

^*** p < .01,

4. DISCUSSION

The present study found that symptom severity dimensions of psychopathology identified by the SCID demonstrate substantial internal consistency, test–retest‐reliability, concurrent, and predictive validity, and far superior psychometric properties than categorical diagnoses for most disorders. Although studies of personality disorders and research utilizing paper‐and‐pencil questionnaires have previously demonstrated similarly superior psychometrics for dimensional assessments of psychopathology (Markon et al., 2011), this is the first study to our knowledge that has examined the ability of a semistructured interview to identify reliable severity dimensions of MDD, GAD, OCD, PTSD, PD, agoraphobia, social anxiety, specific phobia, AUDs, and SUDs for both current and past psychopathology.

These results suggest that if research labs are seeking to use a semistructured interview to assess psychopathology dimensionally (or with a categorical/dimensional hybrid model, Muthen, 2006), they can simply adapt the SCID (one of the most widely adopted tools to assess psychopathology) to do so, without additional training beyond the standard SCID. The SCID might be a particularly useful measure for labs conducting Research Domain Criteria inspired research that define psychopathology dimensionally, but may also need to classify participants' categorical diagnostic status (e.g., for exclusion criteria; Cuthbert & Kozak, 2013).

There are, of course, other measures to assess current psychopathology dimensionally (e.g., Kotov et al., 2005; Watson et al., 2008), but most of these are self‐report (which have limitations as discussed above) or do not assess a broad range of psychopathology. Another advantage of the SCID over these instruments is that it not only assesses current (i.e., last several weeks or month) but also lifetime psychopathology and the present results suggest that severity dimensions for both can be assessed reliably and validly. It is interesting that symptom severity scales of lifetime psychopathology were reliable and valid given questions about the validity of categorically defined lifetime psychopathology assessments (Olino et al., 2012). These prior studies did not assess lifetime psychopathology dimensionally, however, and this may have contributed to their poorer validity.

We do not assume that the DSM‐based severity scales used in this study provide the most parsimonious classification system. We took a “top‐down” approach to defining severity dimensions, using the structure of the SCID (which is organized by DSM categories), while most studies examining the structure of psychopathology take a “bottom‐up” approach, examining how symptoms of psychopathology co‐vary into parsimonious dimensions and/or types (Eaton, Krueger, Docherty, & Sponheim, 2014). It will be important for future studies to develop semistructured interviews that measure “higher order” dimensions (such as the frequently observed dimensions of internalizing and externalizing psychopathology; Farmer, Seeley, Kosty, Olino, & Lewinsohn, 2013; Krueger & Markon, 2006).

Strengths of the study included a well‐characterized sample of individuals with broad range of symptomatology, a longitudinal follow‐up, retest assessments with a different interviewer, and numerous measures of functioning to validate the measures of psychopathology. There were also several notable limitations. First, individuals with a personal or family history of bipolar disorders or psychosis were excluded, thus limiting the study's ability to examine those psychopathologies. Second, impairment was reported by the respondent and was not corroborated by independent report, although interviewers attempted to elicit examples from respondents in order to validate these ratings during the interview. Third, several disorders that were assessed had very few cases for the categorical analyses (particularly for the analyses of current psychopathology). Fourth, the present study only examined the dimension of symptom severity and not other dimensions that are important to psychopathology assessment (e.g., duration of symptoms; Shankman & Klein, 2002). Fifth, the sample was not systematically recruited for the purposes of this study and had a limited age range (18–30), which limits the generalizability of these findings to other clinical or community samples or other age groups.

In sum, the present study suggests that even though the SCID was designed to assess psychopathology categorically, it can be adapted to yield severity dimensions of the common forms of psychopathology (depression, anxiety, substance, and alcohol use) that are reliable and valid. Most importantly, these dimensions consistently showed incremental validity over and above SCID‐defined categories of psychopathology. As the field considers moving towards frameworks that define psychopathology dimensionally (Helzer et al., 2006), additional studies such as the present one are valuable in assessing the psychometrics (and ultimately feasibility and utility) of dimensional assessment approaches.

ACKNOWLEDGEMENTS AND FUNDING SOURCES

This work was supported by a grant from the National Institute of Mental Health (R01MH098093 [PI: Shankman]) and the National Center for Advancing Translational Sciences (UL1TR002003).

Supporting information

Table S1. Internal consistency (n = 234) and test–retest reliability of current and lifetime eating disorders (N = 51)

Table S2. Concurrent validity for lifetime eating disorders (N = 234)

Table S3. Concurrent validity for current eating disorders (N = 234)

Table S4. Current and Lifetime Eating Disorders Predicting Symptoms and Functioning at 1‐year follow‐up (predictive validity)

Shankman SA, Funkhouser CJ, Klein DN, Davila J, Lerner D, Hee D. Reliability and validity of severity dimensions of psychopathology assessed using the Structured Clinical Interview for DSM‐5 (SCID). Int J Methods Psychiatr Res. 2018;27:e1590 10.1002/mpr.1590