Abstract
The Clinician-Administered PTSD Scale (CAPS) is used to measure posttraumatic stress symptoms (PTSS) and diagnose posttraumatic stress disorder (PTSD). However, its use, particularly in settings involving longitudinal assessment, has been complicated by changes in the diagnostic criteria between the fourth and fifth editions of the Diagnostic and Statistical Manual of Mental Disorders (i.e., DSM-IV and DSM-5, respectively). The current sample included trauma-exposed U.S. veterans who were deployed in support of military operations following the September 11, 2001, terrorist attacks (N = 371) and were enrolled in a longitudinal study focused on deployment-related stress and traumatic brain injury. A hybrid clinical interview using item wording from the CAPS for DSM-IV (CAPS-IV) with the addition of items unique to the CAPS for DSM-5 (CAPS-5) was used to assess both DSM-IV and DSM-5 PTSD diagnostic criteria, allowing for the calculation of separate total scores and diagnoses. Diagnostic agreement, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and interrater reliability between CAPS-IV and CAPS-5 were evaluated for the entire sample and stratified by gender. We found high diagnostic agreement (92.9%–95.4%), sensitivity (94.4%–98.2%), specificity (91.7%–92.8%), PPV (89.5%–93.0%), NPV (95.7%–98.1%), and interrater reliability (κ = 0.86–0.91) for both men and women. The current study supports the use of a hybrid PTSD diagnostic interview assessing both DSM-IV and DSM-5 diagnostic criteria, particularly in situations such as longitudinal studies that may require a feasible method of incorporating changes in diagnostic criteria from the DSM-IV to the DSM-5.
The assessment of posttraumatic stress symptoms (PTSS) and diagnosis of posttraumatic stress disorder (PTSD), particularly in settings involving longitudinal assessment, has been complicated by changes in the diagnostic criteria between the fourth edition (text revision) of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association [APA], 2000) and most recent fifth edition (i.e., DSM-5; APA, 2013). In addition to a shift in taxonomy from an “anxiety disorder” to a “trauma-and stressor-related disorder,” the diagnostic criteria for PTSD were also revised between the DSM-IV-TR and DSM-5. These revisions included refining DSM-IV Criterion A1 to specify that the death of a family member or friend must have been violent or accidental to qualify as traumatic, the inclusion of indirect trauma exposure as a Criterion A1 qualifier, eliminating DSM-IV Criterion A2 (i.e., response to an A1 traumatic event with “intense fear, helplessness, or horror”), and transitioning from a three-factor model (i.e., intrusion, avoidance/numbing, increased arousal) to a four-factor model (i.e., intrusion, avoidance, negative alterations in cognitions and mood [NACM], alterations in arousal and reactivity). Experts recognized the potential for these changes, particularly changes to Criterion A1 and modification to the symptom structure related to symptoms of avoidance and numbing, to impact diagnostic concordance between DSM-IV and DSM-5 (Friedman, 2013; Hoge et al., 2016). For example, individuals who may have met the DSM-IV diagnostic criteria for PTSD associated with the nonviolent or nonaccidental death of a loved one or those who did not report avoidance symptoms now no longer meet the diagnostic criteria according to the DSM-5.
The Clinician-Administered PTSD Scale (CAPS) is a structured diagnostic interview that assesses for the presence of DSM PTSD symptoms (i.e., “yes” or “no”), as well as symptom severity. Administration of the CAPS allows for the assessment of symptom frequency and intensity along with the generation of an overall score. The original CAPS (i.e., CAPS-IV; Blake et al., 1995) included the DSM-IV PTSD diagnostic criteria (APA, 1994) as well as associated symptoms, such as dissociation. The measure was revised to reflect diagnostic criteria changes outlined in the DSM-5 and include adjustments in administration and scoring (Weathers et al., 2018).
In a study comparing diagnostic agreement between the CAPS-IV and CAPS-5 in a sample of U.S. Military veterans (Mage = 52.5 years, SD = 10.9), Weathers et al. (2018) identified a moderate association between the CAPS-IV and CAPS-5 using standard scoring methods (κ = 0.51), with 23 out of 30 concordant classifications. Of the seven discordant classifications, two had a PTSD diagnosis on the CAPS-IV but not the CAPS-5, and five had a diagnosis on the CAPS-5 but not the CAPS-IV. Discordance characterized by diagnosis on the CAPS-IV but not CAPS-5 was associated with lack of endorsement of either NACM or avoidance symptoms. Increased endorsement of symptoms under the NACM criteria explained all five discordant pairs characterized by a CAPS-5 but not CAPS-IV diagnosis. Improvement in diagnostic agreement was noted when a minimum severity score for both CAPS versions was used. Although these findings support compatibility between the CAPS-IV and CAPS-5, particularly when optimized scoring criteria are used, administration of the CAPS assessments was completed at two different time points, and the participant sample was representative of veterans older than most who served in support of military operations following the September 11, 2001, terrorist attacks (9/11; Mage = 52.5) and was relatively small (N = 30).
The current study extended initial work evaluating diagnostic concordance between the CAPS-IV and CAPS-5 using a larger and younger, post-9/11 veteran sample (McGlinchey, et al., 2017). Using a prospective longitudinal cohort design, the Translational Research Center for Traumatic Brain Injury and Stress Disorders (TRACTS) has included a structured assessment of PTSD since the onset of data collection in 2010. The CAPS-IV (Blake et al., 1995) was used during initial data collection. However, given the longitudinal nature of data collection for this study, it was necessary to consider adjustments in PTSD assessment following the 2013 publication of the DSM-5 and associated changes in the PTSD diagnostic criteria. Because the CAPS-IV had been used in data collection for several years, discontinuing this measure entirely in favor of the CAPS-5 was not a viable option, as the longitudinal study required continuity for analyses of associations between PTSD severity and other constructs, such as cognition and neuroimaging variables. Administering two full, separate CAPS interviews (i.e., CAPS-IV and CAPS-5) was also not a viable option given time constraints related to the study procedures and what would have been perceived as excessive repetition by study participants. Ultimately, the decision was made to create a hybrid CAPS assessment to incorporate new CAPS-5 items and scoring procedures into the CAPS-IV interview. This allowed for the ability to calculate both CAPS-IV and CAPS-5 total and subscale scores and eliminated possible concordance errors related to inconsistent reporting across multiple time points. Establishing the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and interrater reliability has the potential to validate the utility of this hybrid interview assessing DSM-IV and DSM-5 PTSD symptoms for longitudinal research.
We hypothesized that similar to Weathers et al. (2018), the CAPS-IV and CAPS-5 PTSD diagnosis would demonstrate strong diagnostic agreement as well as high sensitivity, specificity, PPV, NPV, and interrater reliability. Given the evidence related to gender differences in the incidence and prevalence of PTSD diagnosis (Kelber et al., 2021), we were interested in conducting secondary analyses comparing diagnostic discordance between men and women. Given the limited available literature, we did not have specific hypotheses related to gender and diagnostic agreement.
METHOD
Participants and procedure
The current sample included veterans deployed to locations associated with the post-9/11 era of conflicts (e.g., Afghanistan, Iraq, Kuwait) who were participating in the TRACTS Longitudinal Cohort Study focused on deployment-related stress and traumatic brain injury (TBI; McGlinchey et al., 2017). Participants were trauma-exposed veterans with at least one post-9/11 deployment and were recruited by a full-time recruitment specialist who attended military events as well as via word-of-mouth, fliers posted in a Veterans Affairs (VA) hospital, and self-referrals resulting from national media attention. No specific diagnoses were targeted during recruitment; rather, deployment to a war zone post-9/11 was the only required criterion. However, veterans with a history of blast exposure or TBI may have self-referred at higher rates than those without based on study goals. Individuals were ineligible for participation in this study if the following criteria were identified during the clinical interview or review of the veteran’s electronic health record: (a) history of neurological illness other than TBI; (b) history of seizures; (c) current diagnosis of schizophrenia spectrum or other psychotic disorders not related to PTSD; (d) current active suicidal or homicidal ideation, intent, or plan requiring crisis intervention; or (e) cognitive disorder due to a general medical condition other than TBI. For the present analyses, we restricted our sample to the first 371 participants consecutively seen following the January 2015 implementation of the hybrid CAPS-IV and CAPS-5 clinical interview. For instances in which an individual completed more than one assessment, given the longitudinal study design, we selected data from the first assessment that used the hybrid CAPS-IV and CAPS-5 interview. Research procedures were approved by the Institutional Review Board at the VA Boston Healthcare System. All participants provided informed consent and were reimbursed for their time.
Measures
Demographic characteristics
Demographic variables, such as age, gender, and race/ethnicity, and information regarding service branch were collected via self-report at the time of assessment.
PTSS
Self-report.
The PTSD Checklist for DSM-5 (PCL-5; Weathers, Litz, et al., 2013) is a 20-item, self-report measure of DSM-5 PTSD symptoms administered to provide a rapid assessment of DSM-5 PTSS. The PCL-5 asks participants to identify how bothered they have been by specific symptoms in the past month in response to stressful life experiences, with responses rated on a scale of 0 (not at all) to 4 (extremely). PCL-5 total symptom severity scores range from 0 to 80, with higher scores reflecting more severe symptoms. The PCL-5 has demonstrated good internal consistency, test–retest reliability, and convergent and discriminant validity in veteran samples (Bovin et al., 2016). To classify PTSD diagnosis using the PCL-5, symptoms endorsed with a score of 2 (moderately) or higher were required for at least one item on the Intrusion subscale (Items 1–5), one item on the Avoidance subscale (Items 6–7), two items on the NACM subscale (Items 8–14), and two items on the Alterations in Arousal and Reactivity subscale (Items 15–20). In the present sample, the PCL-5 demonstrated excellent reliability, Cronbach’s α = .96.
Clinician-administered interview.
Doctoral-level psychologists assessed current PTSS using the CAPS-IV as the basis for the hybrid interview. Psychologists were trained to administer the CAPS to criterion standard following guidelines outlined by the National Center for PTSD (n.d.), and all evaluations were reviewed by at least three doctoral-level psychologists or psychiatrists to achieve a consensus diagnosis for PTSD (McGlinchey et al., 2016). Interviews were audio-recorded to assess interrater reliability in 5% of the cohort; interrater reliability was strong (i.e., κ > 0.68; McGlinchey et al., 2016). The CAPS-IV (Blake et al., 1995), reflecting criteria from the DSM-IV (APA, 1994), begins with the completion of a life events checklist from which the interviewer initiates inquiry for up to three events that meet DSM Criterion A1 (i.e., experiencing, witnessing, or confronting actual or threatened death or serious injury, or threat to the physical integrity of the self or others). We considered all participants to meet both DSM-IV and DSM-5 Criterion A1 due to their history of war zone deployment, although additional DSM-5 Criterion A1 events, including both military and civilian exposures, were also assessed. We did not require participants to meet DSM-IV Criterion A2 (i.e., response to a Criterion A1 event with intense fear, helplessness, or horror). All participants completed a full CAPS hybrid interview. For each event, the participant was asked to provide a brief description of the event and how they responded emotionally. The frequency and severity of DSM-IV PTSD symptoms were queried and rated according to standard criteria (Blake et al., 1995). When interview items assessed the same symptom on both the CAPS-IV and CAPS-5 (e.g., recurrent recollections), the CAPS-IV description and prompts were initially used, and additional language unique to the CAPS-5 was added. CAPS-5 intensity descriptions were also included for each item to allow for CAPS-5 scoring (Weathers et al., 2017). Items unique to the CAPS-5 (e.g., several items assessing NACM) were added in the order that they appeared on the CAPS-5 (i.e., CAPS-5 items assessing NACM were included after CAPS-IV items assessing avoidance).
To calculate a CAPS-IV total score, past-month symptom frequency was rated on a 5-point scale ranging from 0 (never) to 4 (daily or almost every day); symptom intensity was also rated on a 5-point scale, with response options ranging from 0 (none) to 4 (extreme). A frequency rating of at least 1 (occurring once or twice in the past month) and a severity rating of at least 2 (moderate) was required for a symptom to be included in the PTSD diagnostic calculation. A diagnosis of PTSD was made if the participant endorsed at least one reexperiencing symptom, three avoidance/numbing symptoms, and two arousal symptoms using the previously described scoring rule. In the present sample, the CAPS-IV total score demonstrated excellent reliability, Cronbach’s α = .96.
CAPS-5 symptom severity ratings, reflecting a combination of symptom frequency and intensity, were made for each symptom, with items scored on a scale of 0 (absent) to 4 (incapacitating). Symptom severity ratings of at least 2 (moderate/threshold) were required for at least one intrusion symptom, one avoidance symptom, two NACM symptoms, and two alterations in arousal and reactivity symptoms to meet the CAPS-5 diagnostic criteria for PTSD. In the present sample, the CAPS-5 total score demonstrated excellent reliability, Cronbach’s α = .94.
Data analysis
In addition to evaluating CAPS-IV and CAPS-5 diagnostic agreement, we also evaluated the CAPS-IV relative to the CAPS-5 using measures of validity and reliability (i.e., sensitivity, specificity, PPV, NPV, and kappa). Diagnostic agreement and measures of validity and reliability were similarly evaluated for the PCL-5 relative to the CAPS-IV and CAPS-5. Sensitivity and specificity were used to examine the validity of diagnostic instruments by evaluating diagnostic agreement (i.e., true positives and negatives) against the accepted gold-standard instrument. For all validity measures, the CAPS-5 served as the gold standard for PTSD diagnostic assessment. PPV and NPV are similarly used to evaluate agreement but are often most applicable to clinical practice, as such measures are contingent upon disease prevalence. Estimated values of sensitivity, specificity, and predictive values above 0.80 were considered good according to generally accepted standards. Cohen’s kappa coefficient (κ) was used to measure interrater reliability, as kappa values approximate the observed agreement across diagnostic instruments that is above what can be expected based on chance alone. An estimated kappa value of 0.81 or higher was considered to indicate “almost perfect” agreement across diagnostic instruments (Landis & Koch, 1977). Spearman’s correlation coefficient was used to compute the correlation between PCL-5 and CAPS-5 total scores due to the violation of normality assumptions. All p values refer to two-tailed tests. All statistical analyses were conducted in SAS (Version 9.4).
Results
Sample characteristics
The study sample was predominantly White (74.7%) and male (88.7%). The average participant age was 37.19 years (SD = 9.78), and veterans had completed an average of 14.60 years (SD = 2.38) of education (Table 1). CAPS-5 total scores derived from the hybrid interview and PCL-5 administered at the time of diagnostic interview were highly correlated, ρ = .83, p < .001. Of the 371 veterans comprising our analytic sample, 51.2% (n = 190) met the CAPS-IV diagnostic criteria for PTSD and 48.5% (n = 180) met the criteria according to the CAPS-5 at the time of assessment. Within the full sample, 95.1% (n = 353) of participants had concordant diagnoses (i.e., PTSD or no PTSD).
Table 1.
Participant demographic and clinical characteristics
| Covariates | M | SD | n |
|---|---|---|---|
| Age (years) | 37.19 | 9.78 | 371 |
| Educational attainment (years) | 14.60 | 2.38 | 371 |
| PTSD | |||
| CAPS-IV | 47.58 | 29.65 | 371 |
| Total scorea | |||
| Reexperiencing symptoms | 2.42 | 1.80 | 371 |
| Avoidance/numbing symptoms | 2.98 | 2.26 | 371 |
| Arousal symptoms | 2.98 | 1.69 | 371 |
| CAPS-5 | |||
| Total scorea | 25.86 | 16.84 | 371 |
| Intrusion symptoms | 2.27 | 1.78 | 371 |
| Avoidance symptoms | 1.07 | 0.88 | 371 |
| NACM symptoms | 2.92 | 2.37 | 371 |
| Arousal/reactivity symptoms | 3.03 | 1.87 | 371 |
| Functional impairment symptoms | 1.59 | 1.31 | 367 |
| PCL-5 Total (range: 0–80) | 28.09 | 18.99 | 337 |
| N | % | n | |
| Male sex | 329 | 88.7 | 371 |
| Ethnicity | 371 | ||
| Hispanic/Latino | 62 | 16.7 | |
| Unknown | 223 | 60.1 | |
| Race | |||
| White | 277 | 74.7 | 371 |
| Black | 37 | 10.0 | 371 |
| American Indian | 7 | 1.9 | 371 |
| Asian | 12 | 3.2 | 371 |
| Native Hawaiian/Pacific Islander | 0 | 0 | 371 |
| Other | 5 | 5.2 | 96 |
| Unknown | 50 | 13.5 | 371 |
| Marital status | 371 | ||
| Married | 160 | 43.1 | |
| Widowed | 3 | 0.8 | |
| Divorced | 60 | 16.2 | |
| Separated | 14 | 3.8 | |
| Never married/single | 100 | 27.0 | |
| Other | 34 | 9.2 | |
| Service branch | 371 | ||
| Army | 260 | 70.1 | |
| Navy | 26 | 7.0 | |
| Air Force | 29 | 7.8 | |
| Marines | 78 | 21.0 | |
| Coast Guard | 1 | 0.3 | |
| PTSD diagnosis | |||
| DSM-IV | 190 | 51.2 | 371 |
| DSM-5 | 180 | 48.5 | 371 |
Note. N = 371. PTSD = posttraumatic stress disorder; CAPS-IV = Clinician-Administered PTSD Scale for DSM-IV; CAPS-5 = CAPS for DSM-5; DSM = Diagnostic and Statistical Manual of Mental Disorders (fourth [IV] and fifth [5] editions); NACM = negative alterations in cognitions and mood.
CAPS-IV total score range: 0–136; CAPS-5 total score range: 0–80.
Among men (n = 329), 52.0% (n = 171) met the CAPS-IV diagnostic criteria for PTSD, whereas 49.2% (n = 162) met the CAPS-5 diagnostic criteria. Approximately 95.4% (n = 314) of male participants had a concordant diagnosis (i.e., PTSD or no PTSD). A similar pattern of strong diagnostic agreement was observed among women (n = 42), as 45.2% (n = 19) qualified for a PTSD diagnosis per the CAPS-IV diagnostic criteria, and 42.9% (n =18) qualified using the CAPS-5 criteria. Approximately 92.9% (n = 39) of female participants had a concordant diagnosis (i.e., PTSD or no PTSD).
Diagnostic concordance, validity, and reliability
There was PTSD diagnostic agreement for most participants in the sample (95.1%, n = 353) based on CAPS-IV and CAPS-5 criteria (Table 2). Discordant diagnoses were rare; only 3.8% (n = 14) of participants had a PTSD diagnosis according to CAPS-IV but not CAPS-5, and 1.1% (n = 4) qualified for a CAPS-5 but not CAPS-IV diagnosis. In the full sample, the hybrid diagnostic interview exhibited good diagnostic agreement, with high sensitivity (97.8%) and specificity (92.7%) across the entire sample, using CAPS-5 as the standard. The PPV and NPV were high (92.6% and 97.8%, respectively), and there was strong interrater reliability, κ = 0.90, p < .001.
TABLE 2.
CAPS-IV and CAPS-5 subscale summary measures, by posttraumatic stress disorder (PTSD) diagnostic agreement
| Covariate | Concordance | |||||||
|---|---|---|---|---|---|---|---|---|
| Discordant (n = 18) | Positive diagnosis (n = 176) | Negative diagnosis (n = 177) | ||||||
| M | SD | M | SD | r | M | SD | r | |
| CAPS-IV | ||||||||
| Current total scorea | 52.78 | 10.82 | 72.48 | 18.53 | 0.31* | 22.29 | 15.14 | 0.45* |
| Subscale symptom totals | ||||||||
| Reexperiencing | 2.39 | 1.46 | 3.68 | 1.19 | 0.26* | 1.17 | 1.42 | 0.25* |
| Avoidance/Numbing | 3.28 | 0.83 | 4.95 | 1.16 | 0.38* | 1.00 | 1.22 | 0.45* |
| Arousal | 3.50 | 1.20 | 4.19 | 0.93 | 0.18* | 1.72 | 1.39 | 0.33* |
| Subscale frequency and intensity totals | ||||||||
| Reexperiencing | 12.61 | 6.83 | 19.40 | 7.43 | 0.25* | 5.99 | 5.98 | 0.28* |
| Avoidance/Numbing | 19.28 | 5.87 | 28.44 | 8.65 | 0.30* | 6.16 | 6.51 | 0.43* |
| Arousal | 20.89 | 8.13 | 24.65 | 6.20 | 0.14* | 10.14 | 7.36 | 0.33* |
| Impairmentb | 5.39 | 1.65 | 7.00 | 2.08 | 0.24* | 1.62 | 1.90 | 0.43* |
| CAPS-5 | ||||||||
| Current total scorea | 28.33 | 7.22 | 40.03 | 10.83 | 0.31* | 11.51 | 8.16 | 0.44* |
| Subscale symptom totals | ||||||||
| Intrusion | 2.28 | 1.53 | 3.55 | 1.23 | 0.25* | 1.01 | 1.30 | 0.26* |
| Avoidance | 0.61 | 0.85 | 1.77 | 0.42 | 0.44* | 0.43 | 0.69 | 0.06 |
| NACM | 3.72 | 1.41 | 4.85 | 1.39 | 0.22* | 0.93 | 1.37 | 0.45* |
| Arousal/Reactivity | 3.67 | 1.61 | 4.36 | 1.15 | 0.13 | 1.64 | 1.43 | 0.33* |
| Distress/Impairmentb | 2.28 | 0.75 | 2.68 | 0.59 | 0.19* | 0.41 | 0.78 | 0.52* |
| Subscale severity totals | ||||||||
| Intrusion | 6.11 | 3.63 | 9.51 | 3.63 | 0.25* | 2.93 | 3.00 | 0.26* |
| Avoidance | 2.00 | 1.64 | 4.83 | 1.52 | 0.39* | 1.24 | 1.55 | 0.16* |
| NACM | 10.11 | 3.88 | 13.47 | 4.69 | 0.22* | 2.84 | 3.55 | 0.43* |
| Arousal/Reactivity | 10.11 | 4.54 | 12.23 | 3.47 | 0.15* | 4.50 | 3.39 | 0.34* |
| Distress/Impairmentb | 5.39 | 1.65 | 7.00 | 2.08 | 0.24* | 1.62 | 1.90 | 0.43* |
Note. Tests of significance were conducted for the discordant group versus each concordant group. Reported p values are false discovery rate–adjusted. CAPS-IV = Clinician-Administered PTSD Scale for DSM-IV; CAPS-5 = CAPS for DSM-5; DSM = Diagnostic and Statistical Manual of Mental Disorders (fourth [IV] and fifth [5] editions); NACM = negative alterations in cognitions and mood.
CAPS-IV total score range: 0–136; CAPS-5 total score range: 0–80.
n = 173 for concordant negative diagnosis group
p < .05.
Diagnostic concordance, validity, and reliability for PCL scores
Further examination of PTSD concordance using PCL-5 scoring showed moderate diagnostic agreement between measures, albeit lower than for the CAPS, for the subset of the sample who completed the PCL-5 (n = 337). Using the CAPS-IV as the standard, sensitivity was 65.5% and specificity was 85.2%. Positive and negative predictive values were 81.5% and 71.3%, respectively, and interrater reliability was moderate, κ = 0.51, p < .001. We observed similar levels of agreement using the CAPS-5 as the standard such that the sensitivity was 69.4%, specificity was 86.4%, and PPV and NPV were 82.2% and 75.7%, respectively. Interrater reliability was also moderate, κ = 0.56, p < .001.
Diagnostic concordance, validity, and reliability by gender
After stratifying the analyses by gender, sensitivity and specificity of the hybrid interview among men remained high (98.2% and 92.8%, respectively). Additionally, the PPV and NPV were high (93.0 % and 98.1%, respectively), and there was excellent interrater reliability, κ = 0.91, p < .001. Among men, 4.6% (n = 15) had discordant PTSD diagnoses (Table 3); of these participants, 12 had a positive diagnosis using the CAPS-IV criteria but a negative diagnosis using the CAPS-5.
TABLE 3.
CAPS-IV and CAPS-5 subscale summary measures for men, stratified by posttraumatic stress disorder (PTSD) diagnostic agreement
| Covariate | Concordance | |||||||
|---|---|---|---|---|---|---|---|---|
| Discordant (n = 15) | Positive diagnosis (n = 159) | Negative diagnosis (n = 155) | ||||||
| M | SD | M | SD | r | M | SD | r | |
| CAPS-IV | ||||||||
| Current total scorea | 52.47 | 11.26 | 72.36 | 18.88 | 0.30* | 22.96 | 15.26 | 0.43* |
| Subscale symptom totals | ||||||||
| Reexperiencing | 2.13 | 1.36 | 3.65 | 1.22 | 0.29* | 1.19 | 1.44 | 0.21* |
| Avoidance/Numbing | 3.20 | 0.68 | 4.93 | 1.18 | 0.38* | 1.04 | 1.24 | 0.43* |
| Arousal | 3.67 | 1.18 | 4.19 | 0.95 | 0.14 | 1.77 | 1.41 | 0.34* |
| Subscale frequency and intensity totals | ||||||||
| Reexperiencing | 11.47 | 6.62 | 19.30 | 7.58 | 0.28* | 6.05 | 6.05 | 0.24* |
| Avoidance/Numbing | 18.47 | 5.26 | 28.45 | 8.82 | 0.32* | 6.40 | 6.63 | 0.41* |
| Arousal | 22.53 | 7.31 | 24.61 | 6.34 | 0.09 | 10.52 | 7.41 | 0.37* |
| Impairmentb | 5.47 | 1.68 | 6.97 | 2.13 | 0.21* | 1.69 | 1.93 | 0.42* |
| CAPS-5 | ||||||||
| Current total scorea | 28.20 | 7.76 | 39.93 | 11.09 | 0.29* | 11.89 | 8.27 | 0.41* |
| Subscale symptom totals | ||||||||
| Intrusion | 2.00 | 1.41 | 3.51 | 1.25 | 0.28* | 1.03 | 1.31 | 0.22* |
| Avoidance | 0.47 | 0.74 | 1.76 | 0.43 | 0.48* | 0.43 | 0.70 | 0.01 |
| NACM | 3.73 | 1.53 | 4.79 | 1.41 | 0.19* | 0.95 | 1.39 | 0.43* |
| Arousal/Reactivity | 4.00 | 1.36 | 4.38 | 1.18 | 0.08 | 1.71 | 1.45 | 0.37* |
| Distress/Impairmentb | 2.27 | 0.70 | 2.66 | 0.60 | 0.20* | 0.44 | 0.80 | 0.50* |
| Subscale severity totals | ||||||||
| Intrusion | 5.47 | 3.46 | 9.45 | 3.73 | 0.28* | 2.98 | 3.04 | 0.22* |
| Avoidance | 1.73 | 1.28 | 4.84 | 1.54 | 0.42* | 1.27 | 1.58 | 0.14 |
| NACM | 10.07 | 4.20 | 13.36 | 4.77 | 0.20* | 2.91 | 3.61 | 0.41* |
| Arousal/Reactivity | 10.93 | 4.20 | 12.28 | 3.58 | 0.10 | 4.73 | 3.43 | 0.37* |
| Distress/Impairmentb | 5.47 | 1.68 | 6.97 | 2.13 | 0.21* | 1.69 | 1.93 | 0.42* |
Note. Tests of significance were conducted for the discordant group versus each concordant group. Reported p values are false discovery rate–adjusted. CAPS-IV = Clinician-Administered PTSD Scale for DSM-IV; CAPS-5 = CAPS for DSM-5; DSM = Diagnostic and Statistical Manual of Mental Disorders (fourth [IV] and fifth [5] editions); NACM = negative alterations in cognitions and mood.
CAPS-IV total score range: 0–136; CAPS-5 total score range: 0–80.
n = 151 for concordant negative diagnosis group.
p < .05.
For women, we found high sensitivity (94.4%) and specificity (91.7%), although these psychometric properties were slightly lower compared with those observed for men. The PPV and NPV were also high (89.5% and 95.7%, respectively), and there was excellent interrater reliability, κ = 0.86, p < .001. Of the 42 women in the sample, three had discordant PTSD diagnoses (Table 4). Among these women, two had a positive PTSD diagnosis using CAPS-IV criteria but a negative diagnosis using CAPS-5.
TABLE 4.
CAPS-IV and CAPS-5 subscale summary measures for women, stratified by posttraumatic stress disorder (PTSD) diagnostic agreement
| Covariates | Concordance | |||||||
|---|---|---|---|---|---|---|---|---|
| Discordant (n = 3) | Positive diagnoses (n = 17) | Negative diagnoses (n = 22) | ||||||
| M | SD | M | SD | r | M | SD | r | |
| CAPS-IV | ||||||||
| Current total scorea | 54.33 | 10.21 | 73.59 | 15.35 | 0.39 | 17.59 | 13.66 | 0.55* |
| Subscale symptom totals | ||||||||
| Reexperiencing | 3.67 | 1.53 | 3.94 | 0.90 | 0.05 | 1.00 | 1.27 | 0.47* |
| Avoidance/Numbing | 3.67 | 1.53 | 5.12 | 0.99 | 0.40 | 0.73 | 1.03 | 0.53* |
| Arousal | 2.67 | 1.15 | 4.18 | 0.73 | 0.45 | 1.41 | 1.26 | 0.32 |
| Subscale frequency and intensity totals | ||||||||
| Reexperiencing | 18.33 | 5.51 | 20.29 | 6.03 | 0.11 | 5.64 | 5.59 | 0.50* |
| Avoidance/Numbing | 23.33 | 8.33 | 28.29 | 7.12 | 0.26 | 4.50 | 5.45 | 0.52* |
| Arousal | 12.67 | 8.14 | 25.00 | 4.85 | 0.51 | 7.45 | 6.53 | 0.23 |
| Impairment | 5.00 | 1.73 | 7.24 | 1.60 | 0.38 | 1.18 | 1.62 | 0.51* |
| CAPS-5 | ||||||||
| Current total scorea | 29.00 | 4.58 | 40.94 | 8.16 | 0.50 | 8.86 | 6.94 | 0.55* |
| Subscale symptom totals | ||||||||
| Intrusion | 3.67 | 1.53 | 3.94 | 0.90 | 0.05 | 0.86 | 1.21 | 0.50* |
| Avoidance | 1.33 | 1.15 | 1.82 | 0.39 | 0.17 | 0.41 | 0.59 | 0.32 |
| NACM | 3.67 | 0.58 | 5.41 | 1.06 | 0.50 | 0.77 | 1.23 | 0.54* |
| Arousal/Reactivity | 2.00 | 2.00 | 4.18 | 0.73 | 0.45 | 1.18 | 1.22 | 0.15 |
| Distress/Impairment | 2.33 | 1.15 | 2.82 | 0.39 | 0.17 | 0.23 | 0.53 | 0.64* |
| Subscale severity totals | ||||||||
| Intrusion | 9.33 | 3.06 | 10.00 | 2.57 | 0.10 | 2.59 | 2.75 | 0.50* |
| Avoidance | 3.33 | 2.89 | 4.71 | 1.36 | 0.20 | 1.05 | 1.25 | 0.29 |
| NACM | 10.33 | 2.08 | 14.47 | 3.83 | 0.39 | 2.32 | 3.05 | 0.52* |
| Arousal/Reactivity | 6.00 | 4.58 | 11.76 | 2.17 | 0.44 | 2.91 | 2.62 | 0.25 |
| Distress/Impairment | 5.00 | 1.73 | 7.24 | 1.60 | 0.38 | 1.18 | 1.62 | 0.51* |
Note. Tests of significance were conducted for the discordant group versus each concordant group. Reported p values are false discovery rate–adjusted. CAPS-IV = Clinician-Administered PTSD Scale for DSM-IV; CAPS-5 = CAPS for DSM-5; DSM = Diagnostic and Statistical Manual of Mental Disorders (fourth [IV] and fifth [5] editions); NACM = negative alterations in cognitions and mood.
CAPS-IV total score range: 0–136; CAPS-5 total score range: 0–80.
p < .05.
Qualitative characterization of discordant diagnoses in the full sample
Among discordant pairs in the full sample, 14 individuals met the criteria for PTSD based on the CAPS-IV but not CAPS-5 diagnostic criteria; one of these participants did not qualify for a CAPS-5 diagnosis due to failure to meet the criteria for both the Avoidance and Intrusion subscales (i.e., 15 total cases of failure to the meet the criteria for a CAPS-5 subscale among 14 individuals). Eleven individuals did not meet the criteria on the Avoidance subscale by either endorsing avoidance symptoms of insufficient severity (n = 8) or endorsing no avoidance symptoms (n = 3). Two additional individuals did not meet the CAPS-5 criteria for PTSD due to endorsing intrusion symptoms of insufficient severity to meet the threshold of moderate. One individual did not meet the criteria because of the separation of avoidance and NACM symptoms on the CAPS-5, and another individual did not meet the criteria because they endorsed arousal symptoms with an insufficient severity level.
Only four individuals met the diagnostic criteria for PTSD based on CAPS-5 but not CAPS-IV criteria. Of these four diagnostic disagreements, three individuals did not meet CAPS-IV criteria because they endorsed too few avoidance/numbing symptoms (n = 1) or endorsed avoidance/numbing symptom frequency and intensity below the CAPS-IV threshold (n = 2). Interestingly, although these three individuals did not meet the CAPS-IV diagnostic criteria for PTSD based on their responses to items related to avoidance/numbing symptoms, they did meet the CAPS-5 diagnostic criteria based on their responses to three symptoms on the NACM subscale (i.e., “exaggerated negative beliefs or expectations,” “distorted cognitions leading to blame,” “persistent negative emotional state”) that are unique to the CAPS-5. Finally, one individual did not meet the CAPS-IV criteria because they endorsed reexperiencing symptoms at a level below the required intensity threshold despite reporting sufficient symptom frequency.
DISCUSSION
Changes in the PTSD diagnostic criteria from DSM-IV to DSM-5 have resulted in challenges to longitudinal assessment that are a significant issue facing the PTSS research community. The current study used a hybrid assessment interview, which incorporated questions from both the CAPS-IV and CAPS-5, to assess PTSS in a relatively young veteran sample. The findings demonstrate high diagnostic agreement between structured assessments of DSM-IV and DSM-5 PTSD diagnostic criteria, with just 4.9% of the total sample discordant. Consistent with expectations (Friedman, 2013; Hoge et al., 2016), most participants with discordant diagnoses met the criteria for PTSD based on the CAPS-IV but not the CAPS-5, and most of these participants were discordant based on their responses to items in the avoidance symptom cluster (i.e., insufficient symptom severity or no symptom endorsement). Interestingly, three out of four discordant cases in which participants met the CAPS-5 diagnostic criteria but not the CAPS-IV criteria were also related to avoidance and numbing symptoms. In these instances, participants failed to endorse sufficient avoidance symptoms or their endorsement of symptom severity was insufficient to meet the CAPS-IV criteria; however, these individuals met the CAPS-5 diagnostic criteria based on their endorsement of symptoms in the NACM cluster. It is critical that researchers and clinicians comparing symptom endorsement between the CAPS-IV and CAPS-5 consider the role that modified diagnostic criteria related to avoidance symptoms may play in the ultimate diagnosis of PTSD.
The high diagnostic agreement between the CAPS-IV and CAPS-5 identified in the current study is consistent with findings reported by Weathers et al. (2017); however, the inclusion of a larger and younger participant group extends previous findings. Shortly after the publication of the DSM-5, some researchers (e.g., Friedman, 2013) posited that a possible source for diagnostic discordance based on DSM-IV and DSM-5 criteria might be related to avoidance symptom endorsement; in line with this, participants in the present sample most commonly failed to meet the CAPS-5 criteria due to the endorsement of insufficiently severe avoidance symptoms or a lack of avoidance symptom endorsement. In contrast, Weathers and colleagues (2018) did not find that endorsement of avoidance symptoms explained discordant diagnoses in their sample. This difference may be due to potential differences in study populations (e.g., age, trauma type, time since trauma) as well sample size. The PCL-5 demonstrated moderate diagnostic agreement, validity, and reliability when compared to CAPS-IV and CAPS-5 diagnostic interviews, suggesting that it can be used as an acceptable, although imperfect, proxy for the CAPS clinical interview. Additional analyses comparing diagnostic agreement between men and women further extend a limited literature evaluating gender differences in PTSD diagnosis based on different diagnostic criteria (e.g., Carmassi et al., 2014). Sensitivity, specificity, PPV, NPV, and interrater reliability were high for both men and women, although all values were slightly lower for women compared to men.
The shift in the PTSD diagnostic criteria from the DSM-IV to DSM-5 has resulted in a variety of methodological approaches intended to allow for the assessment of symptoms and derivation of a total severity score or PTSD diagnosis according to both sets of criteria. The creation of a PCL-C- to PCL-5 crosswalk, which allowed for the prediction of PCL-5 scores based on PCL-C symptom endorsement, provided one method of translating scores between the two measures that did not require the administration of DSM-5 items (Moshier et al., 2019). Rosellini et al. (2015) administered a hybrid assessment that combined PCL-C and PCL-5 items, using the wording from the PCL-C for most items other than those unique to the PCL-5. The assessment of symptoms using this measure revealed good diagnostic agreement based on the endorsement of PCL-C and PCL-5 symptoms. The current study provides further evidence that a hybrid CAPS assessment interview that incorporates DSM-IV and DSM-5 diagnostic criteria also demonstrates good diagnostic agreement.
Changes from DSM-IV to DSM-5 diagnostic criteria have also raised concern about the diagnostic reliability for numerous other conditions (e.g., Peters et al., 2020; Uher et al., 2014). Yet, despite the challenges of assessing diagnostic reliability across different criteria, there do not appear to be crosswalk or hybrid assessment models for other conditions that are similar to those developed for PTSD. The development of such methods may have important implications for longitudinal studies as well as for studies that seek to evaluate diagnostic agreement between DSM-IV and DSM-5 criteria.
The present study had several limitations. The creation of a hybrid assessment interview resulted in a measure that is different from both the CAPS-IV and CAPS-5 and, therefore, has unique psychometric properties. However, the correlation between the CAPS-5 total score derived from the hybrid interview and the PCL-5, administered at the time of the diagnostic interview, was strong (i.e., ρ = .83, p < .001), and there was evidence for moderate diagnostic agreement. In addition, although this study allowed for exploratory analyses comparing diagnostic agreement between men and women, the overall number of women included in this study was small, and future investigations into gender differences that affect PTSD diagnostic agreement are needed. Finally, the current study included only post-9/11 U.S. Military veterans; therefore, findings may not apply to military service members from other eras given the evidence regarding differences in reported PTSD symptoms on standardized assessment measures between veterans of varying periods of service (Bhattarai et al., 2020; Brown et al., 2016) or those from outside of the United States.
Despite these limitations, the current study provides strong support for the use of a hybrid PTSD diagnostic interview to assess both DSM-IV and DSM-5 diagnostic criteria. We found that the use of a hybrid diagnostic interview for PTSD resulted in strong diagnostic agreement as well as high sensitivity, specificity, PPV, NPV, and interrater reliability for men and women. This type of hybrid interview has potential utility for longitudinal studies conducted by researchers looking to incorporate changes in diagnostic criteria from the DSM-IV to the DSM-5 without losing existing data based on the DSM-IV or CAPS-IV.
Acknowledgments
The authors report no conflicts of interest or financial disclosures. This research was supported by the VA through the Translational Research Center for TBI and Stress Disorders (TRACTS), a VA Rehabilitation Research and Development Traumatic Brain Injury National Research Center (B3001-C); a VA Clinical Science Research and Development Merit Review (CX01327) to Regina E. McGlinchey; VA Rehabilitation Research and Development Merit Review (RX002907) to Catherine B. Fortier; and a National Institutes of Health National Center for Complementary and Integrative Health NCCIH grant (R21 AT009430-01) to William P. Milberg. The contents within do not represent the views of the VA or the United States government.
Footnotes
Open practices statement
The study reported in this article was not formally preregistered. Neither the data nor the materials have been made available on a permanent third-party archive; requests for the data or materials should be sent via email to Dr. Regina McGlinchey at regina_mcglinchey@hms.harvard.edu
References
- American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Author. [Google Scholar]
- American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Author. [Google Scholar]
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Author. [Google Scholar]
- Bhattarai J, Oehlert ME, & Weber DK (2020). Psychometric properties of the Mississippi Scale for Combat-Related Posttraumatic Stress Disorder based on veteran’s period of service. Psychological Services, 17(1), 75–83. 10.1037/ser0000285 [DOI] [PubMed] [Google Scholar]
- Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, & Keane TM (1995). The development of a Clinician-Administered PTSD Scale. Journal of Traumatic Stress, 8(1), 75–90. 10.1007/BF02105408 [DOI] [PubMed] [Google Scholar]
- Bovin MJ, Marx BP, Weathers FW, Gallagher MW, Rodriguez P, Schnurr PP, & Keane TK (2016). Psychometric properties of the PTSD Checklist for the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (PCL-5) in veterans. Psychological Assessment, 28(11), 1379–1391. 10.1037/pas0000254 [DOI] [PubMed] [Google Scholar]
- Brown W, Grubaugh AL, Knapp RG, & Acierno RE (2016). Interplay between service era, PTSD symptom expression, and treatment completion among veterans. Military Psychology, 28(6), 418–428. 10.1037/mil0000133 [DOI] [Google Scholar]
- Carmassi C, Akiskal HS, Bessonv D, Massimetti G, Calderani E, Stratta P, Rossi A, & Dell’Osso L (2014). Gender differences in DSM-5 versus DSM-IV-TR PTSD prevalence and criteria comparison among 512 survivors to the L’Aquila earthquake. Journal of Affective Disorders, 160, 55–61. 10.1016/j.jad.2014.02.028 [DOI] [PubMed] [Google Scholar]
- Fortier CB, Amick MM, Grande L, McGlynn S, Kenna A, Morra L, Clark A, Milberg WP, & McGlinchey RE (2014). The Boston Assessment of Traumatic Brain Injury – Lifetime (BAT-L) semistructured interview: Evidence of research utility and validity. Journal of Head Trauma and Rehabilitation, 29(1), 89–98. 10.1097/HTR.0b013e3182865859 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Friedman MJ (2013). Finalizing PTSD in DSM-5: Getting here from there and where to go next. Journal of Traumatic Stress, 26(5), 548–556. 10.1002/jts.21840 [DOI] [PubMed] [Google Scholar]
- Hoge CW, Yuhuda R, Castro CA, McFarlane AC, Vermetten E, Jetly R, Koenen KC, Greenberg N, Shalev AY, Rauch SAM, Marmar CR, & Rothbaum BO (2016). Unintended consequences of changing the definition of posttraumatic stress disorder in DSM-5 critique and call for action. JAMA Psychiatry, 73(7), 750–752. 10.1001/jamapsychiatry.2016.0647 [DOI] [PubMed] [Google Scholar]
- Karstoft K-I, Andersen SB, Bertelsen M, & Madsen T (2014). Diagnostic accuracy of the Posttraumatic Stress Disorder Checklist–Civilian Version in a representative military sample. Psychological Assessment, 26(1), 321–325. 10.1037/a0034889 [DOI] [PubMed] [Google Scholar]
- Kelber MS, Liu X, O’Gallagher K, Steward LT, Belsher BE, Morgan MA, Workman DE, Skopp NA, McGraw K, & Evatt DP (2021). Women in combat: the effects of combat exposure and gender on the incidence and persistence of posttraumatic stress disorder diagnosis. Journal of Psychiatric Research, 133, 16–22. 10.1016/j.jpsychires.2020.12.010 [DOI] [PubMed] [Google Scholar]
- Landis JT, & Koch GG (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. 10.2307/2529310 [DOI] [PubMed] [Google Scholar]
- McGlinchey RE, Milberg WP, Fonda JR, & Fortier CB (2017). A methodology for assessing deployment trauma and its consequences in OEF/OIF/OND veterans: The TRACTS longitudinal prospective cohort study. International Journal of Methods in Psychiatric Research, 26(3), e1556. 10.1002/mpr.1556 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moshier SJ, Lee DJ, Bovin MJ, Gauthier G, Zak A, Rosen RC, Keane TM, & Marx BP (2019). An empirical crosswalk for the PTSD Checklist: translating DSM-IV to DSM-5 using a veteran sample. Journal of Traumatic Stress, 32(5), 799–805. 10.1002/jts.22438 [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Center for PTSD. (n.d.). Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) clinician training curriculum. https://www.ptsd.va.gov/professional/continuing_ed/caps5_clinician_training.asp
- Peters WJ, & Matson JL (2020). Comparing rates of diagnosis using DSM-IV-TR versus DSM-5 criteria for autism spectrum disorder. Journal of Autism and Developmental Disorders, 50(6), 1898–1906. 10.1007/s10803-019-03941-1 [DOI] [PubMed] [Google Scholar]
- Rosellini AJ, Stein MB, Colpe LJ, Heeringa SG, Petukhova MV, Sampson NA, Schoenbaum M, Ursano RJ, Kessler RC, & Army STARRS Collaborators. (2015). Approximating a DSM-5 diagnosis of PTSD using DSM-IV criteria. Depression and Anxiety, 32(7), 493–501. 10.1002/da.22364 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Uher R, Payne JL, Pavlova B, & Perlis RH (2014). Major depressive disorder in DSM-5: Implications for clinical practice and research of changes from DSM-IV. Depression and Anxiety, 31(6), 459–471. 10.1002/da.22217 [DOI] [PubMed] [Google Scholar]
- Weathers FW, Blake DD, Schnurr PP, Kaloupek DG, Marx BP, & Keane TM (2013). The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) https://www.ptsd.va.gov/professional/assessment/adult-int/caps.asp [DOI] [PMC free article] [PubMed]
- Weathers FW, Bovin MJ, Lee DJ, Sloan DM, Schnurr PP, Kaloupek DG, Keane TM, & Marx BP (2018). The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5): Development and initial psychometric evaluation in military veterans. Psychological Assessment, 30(3), 383–395. 10.1037/pas0000486 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weathers FW, Litz BT, Herman D, Huska J, & Keane TM (1993, October). The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility [Conference presentation]. Annual meeting of the International Society for Traumatic Stress Studies, San Antonio, TX, United States. [Google Scholar]
- Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, & Schnurr PP (2013). The PTSD Checklist for DSM-5 (PCL-5). https://www.ptsd.va.gov/professional/assessment/adult-sr/ptsd-checklist.asp
- Wilkins KC, Lang AJ, & Norman SB (2011). Synthesis of the psychometric properties of the PTSD Checklist (PCL) military, civilian, and specific versions. Depression and Anxiety, 28(7), 596–606. 10.1002/da.20837 [DOI] [PMC free article] [PubMed] [Google Scholar]