Assessing the performance of the short screening scale for post‐traumatic stress disorder in a large nationally‐representative survey

Abstract

The performance of the short screening scale for the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM‐IV) post‐traumatic stress disorder (PTSD), has not been assessed in an independent general population sample, although it has been used in epidemiological as well as clinical research. In this report we evaluate the short screening scale in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC), a population‐based survey of US household and group quarter residents. DSM‐IV PTSD was assessed via symptom questions in the Alcohol Use Disorder and Associated Disabilities Interview Schedule‐DSM‐IV (AUDADIS‐IV) version. Sensitivity, specificity, positive and negative predictive value, and percent correctly classified were calculated, using the interview‐based diagnosis as the standard. Replicating findings from the initial report, a score of four or more on the short screening scale identifies cases of PTSD with sensitivity of 78%, specificity of 97%, positive predictive value of 75%, and negative predictive value of 98%. The percentage of correctly classified respondents was 96%. The findings support the utility of the seven‐item scale for screening PTSD in clinical and general population samples. Copyright © 2011 John Wiley & Sons, Ltd.

Introduction

The advent of structured diagnostic interviews is responsible, in part, for the scientific evidence base that has been amassed in the fields of psychiatry and psychiatric epidemiology (Robins, 1985). The need for shortened versions of these structured psychiatric interviews that balance validity with expedited assessment has increased with the incorporation of psychiatric disorders into the catalog of conditions monitored in large general health surveys (e.g., the National Health Interview Survey), as well as, the increased importance of mental health screening in primary care settings. One disorder for which shortened structured interviews have been developed is post‐traumatic stress disorder (PTSD), which affects an estimated 7% of US residents over their lifetime, according to a recent report (Kessler et al., 2005).

Breslau and colleagues have developed an empirically‐derived short screening scale for the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM‐IV) PTSD (Breslau et al., 1999) based on data from a general population survey (n = 2181) that used a structured interview, the Diagnostic Interview Schedule (DIS)/Composite International Diagnostic Interview PTSD section, to ascertain lifetime PTSD. To select a short subset of items from the 17 PTSD criterion symptoms, the analysis proceeded in four steps. Step 1 selected the five best combinations for each of several scale lengths. Step 2 used estimates of the area under the receiver operating characteristic curve to identify the single best combination of items out of the five combinations selected in Step 1 for each of three candidate scale lengths. Step 3 applied receiver operating characteristic analysis to each of the three best models identified in Step 2 in order to select the best final scale. Step 4 simplified the classification rule in Step 3. The simpler rule was based on the total number of symptoms reported by the respondent, with equal weights given to each symptom. Step 4 estimated the sensitivity, specificity, positive predictive value, and negative predictive value for this final scale for varying scale scores, calculated by this rule. Based on these four steps, a scale comprising seven symptoms, each coded one if present and zero if absent, was selected. The utility of the screening scale was tested against the diagnosis of PTSD as elicited by the full interview and classified according to DSM‐IV criteria, including the criterion that the respondent's subjective response involved intense fear, helplessness, or horror (A2), one‐month duration (E), and clinical significance (F). Using a cutoff value of four, the scale's characteristics for identifying cases of PTSD, compared to the interview‐based diagnoses, were as follows: sensitivity = 80%; specificity = 97%; positive predictive value = 71%; negative predictive value = 98%. Complete detail on the development of the scale is available in the original article (Breslau et al., 1999).

Since its publication, the short screening scale has been used in several clinical and epidemiologic studies (Coid et al., 2003; Fogarty et al., 2008; Iversen et al., 2005; Lawler et al., 2005; Maercker et al., 2008; McFarlane et al., 2005; Menning et al., 2008; Priebe et al., 2009; Schulden et al., 2006). However, with one exception (Kimerling et al., 2006), no study has examined the performance of the scale in an independent sample. Kimerling et al. (2006) evaluated the validity of the scale in a clinical sample (n = 134), using clinical interviews with the Clinician Administered PTSD Scale (CAPS) as the standard. The authors demonstrated that the screening scale had high reliability and validity in a primary care setting. While that study supports the utility of the scale in clinical samples, the findings do not extend to general population samples, given the high prevalence of PTSD and the greater severity of the disorder in clinical samples. In this report, we present data on the performance of the short screening scale in a large US national sample.

Methods

Sample

Data are from the second wave of the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). The NESARC is a two wave population‐based face‐to‐face survey. The first wave (Wave 1) was conducted from 2001 to 2002 and included 43,093 individuals aged 18 years and older, 81% of the eligible respondents (Grant et al., 2003). Of the 39,959 individuals who were eligible for the Wave 2 assessment conducted from 2004 to 2005, 34,653 (86.7%) completed the interviews (Grant and Kaplan, 2005). The target population for the NESARC was the adult civilian population of the United States residing in households and group quarters during the first wave of assessment. Further detail on the NESARC can be found elsewhere (Grant et al., 2003; Grant and Kaplan, 2005).

Measures

Assessment of PTSD

The NESARC employed the Alcohol Use Disorder and Associated Disabilities Interview Schedule – DSM‐IV (AUDADIS‐IV) version, a fully‐structured diagnostic interview for use by experienced interviewers without clinical training (Grant et al., 2004). The PTSD section of the interview began with an inventory of 33 traumatic events that operationalize the DSM‐IV stressor criterion. Respondents who had experienced multiple traumatic events were asked to select the worst (“the most distressing”) traumatic event from the list of events they endorsed. Dichotomous DSM‐IV criterion symptoms and other criteria that define the disorder, including the subjective response to the event, duration, and impairment, were asked in connection with the worst (or single) event. Computerized algorithms that applied the DSM‐IV diagnostic definition were used to define cases with PTSD. Based on the PTSD symptom data, we constructed scores on the short screening scale by summing the seven symptoms included in the scale (Breslau et al., 1999).

Statistical analysis

Percentages of endorsement of all 17 criterion symptoms were computed. Sensitivity, specificity, positive and negative predictive value, and percent correctly classified were calculated using the AUDADIS‐IV‐based DSM‐IV PTSD diagnosis as the standard. All analyses were conducted using Stata 10.1.

Results

31,650 respondents (13,390 male and 18,260 female) reported exposure to one or more traumatic events. The three most common worst (or single) events were: “unexpected death of someone close to you” (7151 [23%]), “you indirectly experienced 9/11” (7015 [22%]), and “serious or life‐threatening illness/accident/injury to someone close to you” (6699 [21%]). Taken together these three events constitute two‐thirds of worst (or single) events. Of the 31,650 respondents with one or more traumatic events, 2863 (9%) met DSM‐IV criteria for PTSD in connection with their worst (or only) traumatic event. The analysis was performed on 31,112 individuals with complete data on all criterion symptoms (2764 with PTSD and 28,348 with no PTSD). The percentage of endorsement for each of the 17 DSM‐IV symptoms is shown in Table 1 (the seven symptoms used in the screening scale are depicted in italics in Table 1). The prevalence of the 17 symptoms in this study differed somewhat from the original study, as did the rank order of the endorsement of the seven symptoms in the short scale. However, the two symptoms with the lowest endorsement were the same in both studies (C6 and C7).

Table 1.

DSM‐IV criterion symptoms of PTSD endorsed by 31,112 respondents who reported exposure to a traumatic eventa

Symptom	Percentage
B. Re‐experiencing
1. Intrusive and distressing recollections, thoughts	53.1
2. Distressing dreams of the trauma	24.6
3. Acting or feeling as if the trauma were recurring (“flashbacks”)	23.5
4. Psychological reactivity at exposure to cues that symbolize the trauma	29.9
5. Psychological reactivity at exposure to cues that symbolize the trauma	14.4
C. Avoidance and numbing
1. Efforts to avoid thoughts, feelings associated with the trauma	37.9
2. Efforts to avoid activities, places, people that arouse recollections of the trauma	12.6
3. Amnesia for the trauma	10.9
4. Diminished interest in activities	15.0
5. Feeling of detachment from others	13.5
6. Restricted affect	8.9
7. Sense of foreshortened future	5.3
D. Arousal
1. Sleep problems	21.3
2. Irritability, outbursts of anger	12.6
3. Concentration problems	19.7
4. Hypervigilance	22.8
5. Exaggerated startle response	11.9

^aThe seven symptoms used in the screening scale are depicted in italics.

Table 2 summarizes the characteristics of the short screening scale. These characteristics mirror those obtained in the original report on the scale's development. As in the original report, a score of four or more appears to be the best overall cutoff value, yielding sensitivity of 78%, specificity of 97%, positive predictive value of 75% and negative predictive value of 98%; 96% of respondents were correctly classified as cases and non‐cases of PTSD. The tradeoffs between sensitivity and specificity are displayed for each cutoff value in Table 2.

Table 2.

Characteristics of the cutoff values on the seven symptom scale for PTSD applied to 31,112 respondents who reported exposure to a traumatic event

Cutoff value	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)	Correctly classified (%)
≥1	100.0	73.0	26.5	100.0	75.4
≥2	99.6	86.8	42.5	100.0	88.0
≥3	94.9	94.1	61.2	99.5	94.2
≥4	78.3	97.4	74.7	97.9	95.7
≥5	56.1	98.9	82.8	95.6	95.1
≥6	34.8	99.5	86.9	94.0	93.7
7	15.5	99.8	89.3	92.4	92.3

Discussion

In this study, we examined the characteristics of the short screening scale for PTSD, developed by Breslau et al. (1999), in a large nationally‐representative epidemiologic survey. The results obtained in this sample closely replicate the performance characteristics of the scale observed in the original sample (Breslau et al., 1999). Using the interview‐based diagnosis as the standard, the percentage of true cases missed is about 22%. The chance of falsely classifying someone as having PTSD, when the interview‐based diagnosis does not, is about 3%.

Two limitations should be mentioned. First, a clinical assessment would be a stronger standard against which to test the utility of the short screening scale. Second, the clinical utility of the short screening scale for diagnosing lifetime or current PTSD cannot be inferred directly from this validation study alone. However, Kimerling et al. (2006) evaluated the utility of the scale in a sample of primary care patients (PTSD was diagnosed in 25% of patients) and demonstrated high reliability and validity for identifying current PTSD (Kimerling et al., 2006). The purpose of the scale is limited to measuring PTSD in persons who have experienced DSM‐IV traumatic events. History of exposure to traumatic events and ascertainment of an index event (in persons exposed to multiple events) are not covered by the scale.

It should be noted that, in contrast with structured diagnostic interviews, a screening scale does not have to represent all the defining criteria of a disorder. The glaring absence of re‐experiencing, a core aspect of the syndrome of PTSD, does not diminish the utility of the scale. A screening instrument is judged by its success in predicting the disorder. The replication of the scales' performance characteristics in the NESARC adds considerable support for the utility of the scale. In addition to providing a test in an independent sample, the replication indicates that the scale's performance as originally reported was not unduly influenced by the specific data set on which it was developed, given the different data gathering methods in the NESARC.

The Kimerling et al. (2006) study and this study, taken together, support the utility of the short screening scale for clinical practice and epidemiological research. In clinical settings, this scale may be used by primary care providers to screen patients with probable PTSD, who may then be referred to specialized mental health professionals. In the epidemiological and health surveillance contexts, where there is often the need to assess many conditions in a limited time, the scale is an efficient method for measuring PTSD.

Declaration of interest statement

The authors have no conflicts of interest.

Short Screening Scale for PTSD

C2 Did you avoid being reminded of this experience by staying away from certain places, people or activities? (Remind respondent of life event if necessary.)

1. YES

2. NO

C4 Did you lose interest in activities that were once important or enjoyable?

(Remind respondent of life event if necessary.)

1. YES

2. NO

C5 Did you begin to feel more isolated or distant from other people?

(Remind respondent of life event if necessary.)

1. YES

2. NO

C6 Did you find it hard to have love or affection for other people?

(Remind respondent of life event if necessary.)

1. YES

2. NO

C7 Did you begin to feel that there was no point in planning for the future?

(Remind respondent of life event if necessary.)

1. YES

2. NO

D1 After this experience were you having more trouble than usual falling asleep or staying asleep? (Remind respondent of life event if necessary.)

1. YES

2. NO

D5 Did you become jumpy or get easily startled by ordinary noises or movements?

(Remind respondent of life event if necessary.)

1. YES

2. NO

(Based on the Diagnostic Interview Schedule for DSM‐IV (DIS‐IV), Washington University, St Louis, 1995.)

Commentary

The seven‐item scale screens for DSM‐IV PTSD in persons exposed to traumatic events as defined in DSM‐IV. It is intended to be used only after establishing that the respondent has experienced a qualifying event. Please read the paper carefully. It contains all the information needed for using the scale. As we emphasize in the paper, the screening scale is not an adequate substitute for a psychiatric diagnosis.