Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 6.
Published in final edited form as: Psychiatry. 2020 Aug 6;83(4):390–401. doi: 10.1080/00332747.2020.1762396

A Pragmatic Approach to Psychometric Comparisons between the DSM-IV and DSM-5 Posttraumatic Stress Disorder (PTSD) Checklists in Acutely Injured Trauma Patients

Rddhi Moodliar a,*, Joan Russo a, Michele Bedard-Gilligan a, Kathleen Moloney a, Peyton Johnson a, Sara Seo a, Natalie Vaziri a, Douglas Zatzick a,b
PMCID: PMC7858198  NIHMSID: NIHMS1593109  PMID: 32762319

Abstract

Objective:

Prior investigations suggest the relative equivalence of the DSM-IV and DSM-5 versions of the Posttraumatic Stress Disorders (PTSD) Checklist, yet no investigations have assessed the psychometric properties of the scales in pragmatic trials of acutely injured trauma survivors.

Method:

DSM-IV and DSM-5 versions of the PTSD Checklist were included in follow-up interviews of physically injured patients enrolled longitudinally in a pragmatic clinical trial; pragmatic trials aim to efficiently implement research procedures to inform healthcare system policy changes. Psychometric comparisons of the DSM-IV, DSM-5 and a 20-item blended version of the scale included evaluations of internal consistency, correlational assessments, evaluation of item level agreements, and estimation of DSM-5 cutoffs that optimize electronic health record screening protocols.

Results:

128 patients were included in the pragmatic psychometric study. Cronbach’s alphas for the 3 versions of the PTSD Checklist ranged from 0.93–0.95. Correlations between the 3 scales ranged from 0.79–0.99. All 3 measures demonstrated excellent convergent and discriminant properties. Item level agreement ranged from 70–80%. For the DSM-5 and blended versions of the scale, a score of 30 and 24, respectively, best approximated the DSM-IV cutoff of ≥ 35 that had previously optimized PTSD detection in conjunction with EHR screening.

Conclusions:

Among injured trauma survivors, the psychometric performance of the DSM-IV PTSD Checklist with the addition of the 4 new DSM-5 PTSD Checklist items is nearly equivalent to the DSM-5 PTSD Checklist. The investigation also suggests that pragmatic psychometric methods can catalyze the rapid translation of research findings into real world practice settings.

Keywords: Posttraumatic Stress Disorder, Physical Injury, DSM-5, Psychometrics, Pragmatic Clinical Trials

Introduction

With the advent of the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5) in 2013 came an updated diagnostic criteria for Posttraumatic Stress Disorder (PTSD) (American Psychiatric Association, 2013; American Psychiatric Association & American Psychiatric Association Task Force on DSM-IV, 2000). The changes included alterations to Criterion A: Stressor requirements, and changes to the symptomatic clusters that comprise PTSD. Specifically, the DSM-5 proposed four distinct symptom clusters instead of three, which includes intrusion, avoidance, arousal and reactivity, and, newly, negative alterations in cognitions and mood. One DSM-IV avoidance symptom was eliminated and four new items (3 negative alterations in cognitions and mood items and 1 arousal item) were added.

The reactions to the criterion changes from DSM-IV to DSM-5 were mixed (Friedman et al., 2011; Hoge et al., 2016; Kilpatrick, 2013; Price et al., 2016; Zoellner et al., 2013). Some groups applauded the elimination of the A2 criteria and the inclusion of a fourth symptom (Friedman et al., 2011; Kilpatrick, 2013).

In contrast, Hoge and colleagues critiqued the new criteria on a number of grounds (Hoge et al., 2016). These authors noted that the DSM-5 criteria were based on a limited and selective review of the literature; also, at the time emerging research demonstrated that the revised DSM-5 definition of PTSD offered little improvement in research or clinical utility. Hoge and colleagues noted that the DSM-5 revision undermined the ability to capitalize on the extensive research base created for the DSM-IV and established a circular requirement for an entirely new foundation of instrumentation coupled with new neurobiological, genetic, and clinical interventions in diagnostic research. It was also noted that at the time in 2016, validation studies remained absent for the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) compared with the original CAPS (Weathers et al., 2001) or other diagnostic instruments. Hoge and colleagues suggested that the use of the DSM-IV assessments, including the PTSD Checklist, should be continued ongoing while an evidence-base accrued for the novel DSM-5 instrumentation.

Literature review revealed two investigations that had systematically compared the psychometric characteristics of the DSM-IV and DSM-5 PTSD Checklist (Hoge et al., 2014; Rosellini et al., 2015). In a large-scale study using a military sample, Rosellini and colleagues (2015) used a blended version of the DSM-5 PTSD Checklist to approximate both DSM-5 and DSM-IV diagnostic criteria. The blended version had excellent psychometric characteristics including a sensitivity of 92.6%, specificity of 99.6%, classification accuracy of 99.4%, and ROC curve diagnostics when compared to the standard version of the DSM-5 PTSD Checklist. Rosellini and colleagues concluded that DSM-IV symptoms could be used to approximate DSM-5 diagnosis of PTSD in military samples. These authors recommended replication studies in non-military trauma exposed patient populations.

Hoge and colleagues (2014) conducted perhaps the largest and most comprehensive psychometric comparison of DSM-IV and DSM-5 PTSD Checklist criteria to date. Hoge et. al. undertook a head-to-head comparison of the DSM-5 PTSD Checklist with the DSM-IV PTSD Checklist in 1,822 US infantry soldiers. In this head-to-head comparison, Hoge and colleagues reported similar PTSD screen positive rates for the DSM-IV criteria (13%) compared to DSM-5 criteria (12%). Of note, a substantial subgroup of soldiers were misclassified. Hoge and colleagues concluded that the DSM-IV and DSM-5 versions of the PTSD Checklist were equivalent although some discrepancies were noted.

Co-incident with the transition from DSM-IV to DSM-5 PTSD criteria was the development and refinement of pragmatic clinical trial methodologies (Califf & Sugarman, 2015; Loudon et al., 2015; Palinkas & Zatzick, 2019; Thorpe et al., 2009). Pragmatic clinical trials may present yet another important example of the challenges raised by the premature roll-out of the DSM-5 diagnostic criteria. This genre of trials aims to efficiently implement research procedures in order to inform healthcare system research policy and practice changes (Califf & Sugarman, 2015; Loudon et al., 2015; Palinkas & Zatzick, 2019; Thorpe et al., 2009). Pragmatic clinical trials build on research infrastructure and findings that derive from, at times, years of efficacy and effectiveness investigations (Russo et al., 2013). For example, pragmatic clinical trials may use electronic health record (EHR) screens to assess risk for disorders like PTSD or Major Depressive Disorder (MDD) in general medical settings (Russo et al., 2013; Simon et al., 2016; Zatzick et al., 2016). The use of the EHR to screen for common psychiatric conditions such as PTSD or MDD can rely psychometrically on foundational studies that derive from structured clinical assessments such as the CAPS or Structured Clinical Interview for DSM (SCID) (Russo et al., 2013). A perhaps unintended consequence of the PTSD changes across the DSM-IV and the DSM-5 is the requisite psychometric recalibration for pragmatic trials conducted in real world general medical settings. Unfortunately, pragmatic trials often involve large-scale healthcare system rollouts and are rarely funded to include structured clinical assessments such as the CAPS. The PRagmatic Explanatory Continuum Indicator Summary (PRECIS) explicitly categorizes trials that use structured clinical interviews such as the CAPS or SCID at the lowest grade of the pragmatic-explanatory continuum (Loudon et al., 2015; Thorpe et al., 2009). Patient reported outcome assessments such as the PTSD Checklist occupy a middle point on the continuum, while administrative data that accrues on the intent-to-treat sample with the need for patient follow-up constitutes the highest outcome assessment grade (Zatzick et al., 2018). Simply put, including the CAPS-5 or other diagnostic assessment as part of a pragmatic trial is not pragmatic.

Each year approximately 2.5 million traumatic injury hospital admissions occur annually (Bonnie et al., 1999; Centers for Disease Control and Prevention, 2003). Psychometric studies in acutely injured trauma survivors differ from investigations in veteran samples as there is a need to assess symptoms longitudinally – rather than cross-sectionally – after the acute injury event (Blanchard et al., 1996). High levels of PTSD symptoms are endemic among injured patients admitted to US trauma centers and PTSD symptoms frequently co-occur with depressive symptoms, suicidal ideation, alcohol and drug use, and other life events such as firearm violence (Bryant et al., 2010; Bulger et al., 2019; Zatzick et al., 2008; Zatzick et al., 2014). PTSD and related comorbidity are associated with a broad spectrum of functional impairments (Gabbe et al., 2016; Holbrook et al., 1999; Zatzick et al., 2008). A series of psychometric investigations have explored the DSM-5 PTSD Checklist as an abbreviated screening tool or in conjunction with other measures for post-injury PTSD assessments (Hunt et al., 2017; Price et al., 2016). Literature review however revealed no psychometric studies that have focused exclusively on comparisons of the DSM-IV and DSM-5 PTSD Checklist versions in injured patient populations.

In summary, although prior investigations are useful in that they begin to suggest equivalence between DSM-IV and DSM-5 versions of the PTSD Checklist, literature review revealed no pragmatic psychometric studies in acute care medical settings with injured trauma survivors. The current investigation therefore was challenged to simultaneously roll out a large pragmatic early PTSD intervention trial, while addressing psychometric issues raised by the DSM-IV to DSM-5 PTSD diagnostic transition. The investigation hypothesized that DSM-IV and DSM-5 versions of the measure would be highly correlated and demonstrate good internal consistency. The investigation also hypothesized that the measures would demonstrate excellent convergent and discriminant properties when compared to standardized assessments of mental health symptoms and physical function.

Materials and Methods

Design Overview

The investigation was a secondary study and analysis of data that was embedded within a larger randomized comparative effectiveness trial, “A Comparative Effectiveness Trial of an Information Technology Enhanced Peer-Integrated Collaborative Care Intervention versus Enhanced Usual Care for US Trauma Care Systems: Clinical Study Protocol,” designed to assess the impact of a peer integrated stepped collaborative care intervention targeting improvements in patient-reported outcomes including PTSD over the course of the year after injury admission (Scheuer et al., 2020). The investigation was approved by the University of Washington’s Institutional Review Board prior to study initiation. Informed consent was obtained from each participant at bedside during the inpatient admission prior to the initiation of study procedures.

Patients eligible for the study were required to have been admitted to the University of Washington’s Harborview Level I Trauma Center inpatient surgical ward or emergency department for at least 24 hours. Patients included in the study were English and Spanish speaking, male and female physically injured trauma survivors, ≥ 18 years of age who were residents of Washington, Oregon, Alaska or California.

The population of patients at risk for PTSD underwent an EHR PTSD risk screen. The 10 domain PTSD risk factor items pulled from the EHR were: 1) Gender (Female), 2) Race (Non-White), 3) Insurance (Veteran insurance or no commercial insurance), 4) Treated in intensive care unit during the injury admission, 5), blood alcohol positive, or any substance disorder ICD positive from the EHR, 6) Any psychiatric disorder ICD positive from EHR, 7) PTSD ICD positive from EHR, 8) Tobacco use, 9) Injury inflicted by another individual (i.e., intentional injury), and 10) At least one prior hospitalization (Russo et al., 2013). Each of the 10 domains was scored either 0 or 1. Consenting patients with scores ≥ 3 on the EHR screen were next screened with the DSM-IV PTSD Checklist. To be included in the study patients were required to score ≥ 35 on the DSM-IV version of the PTSD Checklist (Russo et al., 2013), and were also required to have ≥ 1 severe posttraumatic psychosocial concern (Zatzick et al., 2018). Patients were excluded if they required immediate psychiatric intervention (i.e., self-inflicted injury, active psychosis), were cognitively impaired, or were currently incarcerated. Recruitment for the larger study began in August of 2018. Data for this investigation was collected between October 2019 and February 2020 from the larger study. Follow-up interviews were conducted over the telephone at 1-, 3-, 6-, 9-, and 12-months post-injury by blinded research assistants. Study participants were approximately equally divided into the control (n =66, 48%) and intervention (n = 72, 52%) conditions.

Measures

PTSD Symptoms

The DSM-IV PTSD Checklist was used to screen injured trauma survivors into the study and to follow PTSD symptoms longitudinally over the course of the year after injury (Weathers et al., 1993; Weathers et al., 2013). The DSM-IV PTSD Checklist has established reliability and validity across trauma-exposed patient populations. Prior study team investigations have documented Cronbach’s alpha ranging from 0.88 to 0.92 for the DSM-IV PTSD Checklist across follow-up time points (Zatzick et al., 2018).

In the current investigation each injured trauma survivor completed both the DSM-IV and DSM-5 versions of the PTSD Checklist at either the 1-, 3-, 6- 9- or 12-month follow-up time points. During all administrations, the DSM-IV version was given during the initial portion of the interview and the DSM-5 version was administered in the middle portion of the 30- to 60-minute assessment. Following the DSM-IV PTSD Checklist, the Patient Health Questionnaire-9 Item Depression Screen (PHQ-9), and the Medical Outcomes Study Short Form-36 Physical Component Summary (SF-36 PCS) & the Mental Component Summary (SF-36 MCS) Scales were administered.

Depressive Symptoms

The PHQ-9 was used as a continuous measure to assess depressive symptoms. The questionnaire has established reliability and validity in acute and primary care medical patients (Kroenke et al., 2001; Zatzick et al., 2018).

Physical and Mental Function

The SF-36 PCS and SF-36 MCS were used to assess physical health and mental health function (Ware et al., 1993). The questionnaire has established reliability in acute care medical patient populations (Zatzick et al., 2018).

Data Analysis

This investigation builds upon foundational pragmatic studies of the DSM-IV PTSD Checklist that has developed EHR automated screening criteria derived from DSM-IV PTSD Checklist scale scores (Russo et al., 2013). The current approach to the psychometric comparison of the DSM-IV and DSM-5 versions of the PTSD Checklist also derived from and expanded upon previously articulated psychometric approaches in other trauma exposed patient populations (Blevins et al., 2015; Hoge et al., 2014; Rosellini et al., 2015).

The complete DSM-IV and DSM-5 PTSD Checklists were administered. A “blended” version of the PTSD Checklist was then constructed (Table 1) (Rosellini et al., 2015). For the 20-item blended measure, the four new DSM-5 items were added to the DSM-IV PTSD Checklist, and the DSM-IV C7 item that assessed foreshortened future was eliminated. Items that appeared in both the DSM-IV version of the Checklist and the blended version were categorized as having minor wording changes, more substantial phrasing changes or as identical (no changes).

TABLE 1.

Categorization of changes made to the DSM-IV PCL items compared to DSM-5 PCL items1

Criterion DSM-IV PCL Item Category of Change Criterion DSM-5 PCL Item
B. Re-experiencing B. Re-experiencing
B1 Repeated memories Word Change Item B1 Repeated memories
B2 Disturbing dreams Identical Item B2 Disturbing dreams
B3 Flashbacks Word Change Item B3 Flashbacks
B4 Upset at reminders Identical Item B4 Upset at reminders
B5 Cued physical reactions Word Change Item B5 Cued physical reactions
C. Avoidance C. Avoidance
C1 Avoiding internal reminders Phrase Change Item C1 Avoiding internal reminders
C2 Avoiding activities/situations Phrase Change Item C2 Avoiding activities/situations
D. Negative alterations in cognition and mood
C3 Trouble remembering Identical Item, Criterion Change D1 Trouble remembering
C4 Loss of interest Identical Item, Criterion Change D5 Loss of interest
C5 Feeling distant Identical Item, Criterion Change D6 Feeling distant
C6 Emotional distance Phrase Change Item, Criterion Change D7 Emotional distance
C7 Foreshortened future New Item D2 Negative beliefs
Not reflected in the DSM-IV New Item D3 Blaming self or others
Not reflected in the DSM-IV New Item D4 Negative affects
D. Hyperarousal E. Hyperarousal
D1 Insomnia Word Change Item E6 Insomnia
D2 Irritability Word Change Item E1 Irritability
D3 Problems concentrating Identical Item E5 Problems concentrating
D4 Hypervigilance Identical Item E3 Hypervigilance
D5 Startle Identical Item E4 Startle
Not reflected in the DSM-IV New Item E2 Risk behaviors
Open in a new tab

Note: DSM-IV PCL criterion C7 underwent a significant revision and as such the DSM-5 PCL criterion D2 is considered a new item. DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders (5th ed.); PCL = PTSD Checklist.

1

This table was adapted from Rosellini et. al., 2015.

The study team began by assessing the internal consistency of the DSM-IV, DSM-5 and blended versions of the PTSD Checklist. Next a Pearson’s correlation matrix was created that included all 3 versions of the PTSD Checklist; the PHQ-9, SF-36 PCS and SF-36 MCS were included in the matrix to assess convergent and discriminant properties of the 3 measures. A comparison of individual item thresholds (score of “3” or moderate symptoms on the DSM-IV version versus score of “2” or moderate on the DSM-5 version) was also performed; percent agreement and the kappa statistic were calculated for each item comparison. Last, receiver operator characteristic (ROC) curve analysis was used to identify scores on the DSM-5 and blended versions of the PTSD Checklist that best corresponded to the optimal DSM-IV cutoff of ≥ 35 that the study team had identified in previous psychometric studies that incorporated both CAPS and EHR PTSD assessments (Blevins et al., 2015).

Results

The demographic, clinical, and injury characteristics of the sample are displayed in Table 2. Approximately 46% of patients were female, and approximately 19% of patients were intentionally injured.

TABLE 2.

Baseline Patient Characteristics

No. (%) of Patients
Characteristics All
(N = 138)
 Gender: %Women 63 (45.7)
 Age, mean (SD), Years 41 (15.4)
 White 69 (50.0)
 Married 34 (24.6)
 Intentional Injury 26 (18.8)
 Education
  Less than high school 20 (14.5)
  High school/GED 72 (52.2)
  Associates degree/trade school 25 (18.1)
  Bachelors/graduate degree or higher 21 (15.2)
Open in a new tab

Note: SD = standard deviation.

Cronbach’s alpha for the DSM-IV, DSM-5, and blended versions of the scale were 0.93, 0.95, 0.94 respectively. Pearson’s correlations between the DSM-IV, DSM-5 and blended versions were 0.79–0.99 (Table 3). All three PTSD Checklist scale scores were strongly correlated with PHQ-9 and SF-36 MCS scale scores, but not associated (i.e., orthogonal) to SF-36 PCS scales scores (Table 3).

TABLE 3.

Pearson’s correlations between DSM-IV PCL, DSM-5 PCL, and the Blended PCL

DSM-IV PCL DSM-5 PCL Blended PCL
Variable r p r p r p
DSM-IV PCL - - 0.79** .00** .99** .00**
DSM-5 PCL .79** .00** - - .84** .00**
Blended PCL .99** .00** .84** .00** - -
PHQ-9 .80** .00** .74** .00** .80** .00**
SF-36 MCS −.74** .00** −.66** .00** −.74** .00**
SF-36 PCS −.09 .25 −.09 .32 −.11 .22
Open in a new tab

Note: n = 138. DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders (5th ed.); PCL = PTSD Checklist; Blended PCL = 16 items from DSM-IV PCL and 4 new items from DSM-5 PCL; PHQ-9 = Patient Health Questionnaire – 9; SF-36 MCS = Short Form Health Survey Mental Component Score; SF-36 PCS = Short Form Health Survey Physical Component Score.

**

p<0.01.

Individual symptom comparisons from the DSM-IV and DSM-5 versions of the PTSD Checklist are described in Table 4. Item level agreement ranged from 70% to 80%. Corresponding kappa ranged from 0.42–0.58.

TABLE 4.

Comparison of item response and item agreement between DSM-IV PCL Items and DSM-5 PCL Items

Item stem DSM-IV PCL % Positive DSM-5 PCL % Positive % Item Agreement κ
Repeated Memories 42.8 59.4 71.7 0.45
Disturbing dreams 29.0 44.9 72.5 0.43
Flashbacks 29.0 41.3 80.4 0.58
Upset at reminders 44.2 60.9 70.3 0.42
Cued physical reactions 32.6 48.6 73.9 0.47
Avoiding internal reminders 41.3 54.3 73.9 0.49
Avoiding activities/situations 37.7 56.5 72.5 0.47
Trouble remembering 38.4 45.7 76.8 0.53
Loss of interest 43.5 57.2 77.5 0.56
Feeling distant 47.1 61.6 78.3 0.57
Emotional distance 26.8 41.3 75.4 0.46
Insomnia 58.7 63.0 79.7 0.58
Irritability 31.9 33.3 76.8 0.47
Problems concentrating 44.9 61.6 71.7 0.45
Hypervigilance 42.8 56.5 77.5 0.56
Startle 37.7 50.7 78.3 0.57
Open in a new tab

Note: n = 138. DSM-IV PCL item ‘Foreshortened future’ was not listed because the DSM-5 PCL scale does not include this item and as such this item cannot be compared. New DSM-5 PCL items ‘Negative beliefs,’ ‘Blaming self or others,’ ‘Negative affects,’ and ‘Risk behavior’ were not included because these items were significant revisions and were not reflected in the DSM-IV PCL. DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders (5th ed.); PCL = PTSD Checklist.

ROC curve analysis identified optimal cutoffs that best approximated a DSM-IV PTSD Checklist score of ≥ 35. For the DSM-5 PTSD Checklist, a cutoff score of 30 (sensitivity = 0.89, specificity = 0.78) best approximated a DSM-IV PTSD Checklist score of ≥ 35. For the blended version, a cutoff of 24 (sensitivity = 0.92, specificity = 0.98), best approximated the PTSD Checklist DSM-IV scale score of ≥ 35.

Discussion

The current investigation corroborates and extends prior reports comparing the psychometric properties of the DSM-IV and DSM-5 PTSD Checklists. Two prior investigations that included large samples of trauma exposed veteran patients assessed the psychometric properties of the two measures across multiple domains including internal consistency, differential item function and convergent validity (Hoge et al., 2014; Rosellini et al., 2015). Both prior papers ultimately concluded that the two versions of the measure were nearly equivalent, and that the DSM-IV PTSD Checklist symptoms could be used in clinical research studies to approximate the DSM-5 PTSD criteria.

The current investigation with acutely injured trauma survivors again found the measures to be essentially comparable, further justifying the interchangeable use of the assessments. This investigation used a blended version of the 20-item PTSD Checklist that included the 4 new items in conjunction with 16 DSM-IV PTSD Checklist items. The blended version of the PTSD Checklist demonstrated excellent internal consistency. Also, the blended version of the measure was highly correlated with both the DSM-IV and DSM-5 versions and had excellent convergent and discriminant properties when compared to measures of depressive and mental health symptoms and physical function. Excellent overall agreement was also observed for the 16-items shared by the DSM-IV and DSM-5 versions of the PTSD Checklist.

As a pragmatic trial, the current investigation strived to attain broad patient eligibility and utilized EHR data for screening and administrative data (i.e. automated emergency department health service utilization that accrues on the intent-to-treat sample) for ascertainment of the primary outcome. This study serves to extend the results of prior investigations by developing methods for scale properties in “real world” settings with reduced resource availability for intensive clinical assessments. Traditional psychometric criteria for the establishment of the properties of a new version of a measure rest upon comparative assessment with a clinical gold standard structured interview, and include assessments of construct, convergent and discriminant validity as well assessments of internal consistency and test-retest reliability (Weiss & Marmar, 1997). Pragmatic clinical contexts however drastically limit the extent to which clinical researchers can administer assessments that require extensive adjudication, including structured diagnostic assessments such as the CAPS (Loudon et al., 2015). These observations, when combined with prior reports that suggest further refinement is required before the CAPS-5 validation would be complete (Hoge et al., 2014), suggested the use of DSM-IV PTSD symptom criteria with inclusion of new DSM-5 items.

Prior psychometric investigations by the study team suggests that the PTSD Checklist can be used in conjunction with automated EHR screens to efficiently assess populations of injured trauma survivors admitted to trauma center hospitals (Russo et al., 2013; Zatzick et al., 2016). In these prior investigations, a DSM-IV PTSD Checklist cutoff score of ≥ 35 optimally identifies patients who may demonstrate persistent high PTSD symptom levels after a positive EHR screen. For the DSM-5 and blended versions of the scale, the current investigation suggests that a score of 30 and 24, respectively, best approximates the DSM-IV cutoff of ≥ 35. Of note, although slightly higher, these cutoffs derived from an injured patient population are consistent with the range of prior cutoff scores recommended in other trauma exposed patient populations (Blevins et al., 2015; Hoge et al., 2014; Rosellini et al., 2015).

This investigation has limitations. The investigation was conducted at a single Level 1 trauma center; subsequent investigations could productively replicate these findings across multiple Level 1 trauma center sites. Additionally, the current investigation focused almost exclusively on continuous multi-item scale scores rather than diagnostic cutoffs for PTSD symptomatic criteria. As with military populations (Hoge et al., 2014), future acute care medical investigations could compare PTSD diagnostic criteria derived from both the DSM-IV and the DSM-5. Prior investigation in military populations have also included a full spectrum of subject PTSD Checklist scores (Hoge et al., 2014; Rosellini et al., 2015). The current acute care investigation only included highly symptomatic patients at baseline. Prior investigations suggest that early PTSD screening interventions might result in false negatives (Zatzick et al., 2013). That is, some individuals who had a low PTSD score at baseline might develop severe PTSD symptoms months after the injury event. Future investigations could incorporate assessments that include a broader range of patients with PTSD symptoms.

Beyond these considerations, the investigation contributes to an evolving dialogue on the study of trauma exposed patient populations in complex “real-world” settings. The study documents that for injured trauma survivors, use of the DSM-IV PTSD Checklist with addition of the 4 new items, is an optimal pragmatic assessment approach. The investigation also suggests that the development of pragmatic assessment procedures is an essential component of trauma focused study in early intervention contexts such as acute care medical settings; these observations could also be seen to extend to other trauma focused contexts such as natural and man-made disasters and combat zones. The need to adapt novel assessments to real world early intervention contexts presents yet another important example of the challenges raised by the premature roll-out of the DSM-5 diagnostic criteria (Hoge et al., 2016). In summary, further development of pragmatic psychometric methodologies may help to catalyze the rapid translation of research findings for trauma exposed patient populations presenting acutely in complex real-world contexts.

Acknowledgments

This study was supported in part by the Patient-Centered Outcomes Research Institute (PCORI) Award (IHS-2017C1-6151). This research was also supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by cooperative agreement 1UH2MH106338-01/4UH3MH106338-02 from the National Institute of Mental Health. Support was also provided by the NIH Common Fund through cooperative agreement U24AT009676 from the Office of Strategic Coordination within the Office of the NIH Director. The content is solely the responsibility of the authors and does not necessarily represent the official views of PCORI, its Board of Governors or Methodology Committee, or the NIH.

Funding

This work was supported by the Patient-Centered Outcomes Research Institute under Grant IHS-2017C1-6151; the National Institute of Mental Health under Grants 1UH2MH106338-01 and 4UH3MH106338-02; and the National Institutes of Health Common Fund under Grant U24AT009676.

Biography

Rddhi Moodliar is a Research Coordinator at the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinates the regulatory procedures, patient recruitment, and care management for early intervention studies for PTSD and related comorbidities. She received her bachelor’s degree in Psychology from the University of Washington.

Joan Russo is an Associate Professor in the Department of Psychiatry and Behavioral Sciences at the University of Washington and also serves as a statistician and psychometrician for the department. She specializes in health services and outcomes research and is interested in outcomes assessments for traumatically injured patients.

Michele Bedard-Gilligan is an Associate Professor in the Department of Psychiatry and Behavioral Sciences at the University of Washington as well as a licensed clinical psychologist at the University of Washington Outpatient Psychiatry Clinic. Her research focuses on the mechanisms of PTSD recovery and the co-occurrence of PTSD and substance use.

Kathleen Moloney is a Research Coordinator in the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinates a national study examining current practices related to mental health, substance use, and other related services. She also assists with an early intervention study for PTSD in trauma survivors. She received her bachelor’s degree in Neuroscience from the University of Southern California.

Peyton Johnson is a Research Coordinator in the Department of Psychiatry and Behavioral Sciences at the University of Washington where she coordinates patient outcome assessments for an early intervention study for PTSD. She received her bachelor’s degree in Biology from the University of Virginia.

Sara Seo is a Research Assistant in the Department of Psychiatry and Behavioral Sciences at the University of Washington where she assists with patient recruitment and outcome assessment for an early intervention study for PTSD. She received her bachelor’s degree in Public Health from the University of Washington.

Natalie Vaziri is a Research Assistant in the Department of Psychiatry and Behavioral Sciences at the University of Washington where she assists with outcome assessments for an early intervention study for PTSD. She received her bachelor’s degree in Public Health from the University of Washington.

Douglas Zatzick is a Professor in the Department of Psychiatry and Behavioral Sciences at the University of Washington as well as a member of the Core Research Faculty at the Harborview Injury Prevention and Research Center. Dr. Zatzick’s research interests focus on pragmatically focused clinical trials targeting PTSD and related comorbidities in traumatically injured populations.

Footnotes

Declaration of Interests

The authors declare no conflicts of interest.

References

  1. American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Menta Disorders, Fifth Edition: DSM-5 (Fifth ed.). Arlington, VA: American Psychiatric Association. [Google Scholar]
  2. American Psychiatric Association, & American Psychiatric Association Task Force on DSM-IV. (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR (Fourth ed.). Washington, DC: American Psychiatric Association. [Google Scholar]
  3. Blanchard EB, Jones-Alexander J, Buckley TC, & Forneris CA (1996). Psychometric properties of the PTSD Checklist. Behav Res Ther, 34, 669–673. [DOI] [PubMed] [Google Scholar]
  4. Blevins CA, Weathers FW, Davis MT, Witte TK, & Domino JL (2015). The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): Development and Initial Psychometric Evaluation. Journal of Traumatic Stress, 28(6), 489–498. doi: 10.1002/jts.22059 [DOI] [PubMed] [Google Scholar]
  5. Bonnie RJ, Fulco CE, & Liverman CT (Eds.). (1999). Reducing the Burden of Injury: Advancing Prevention and Treatment. Washington, DC: National Academy Press. [PubMed] [Google Scholar]
  6. Bryant RA, O’Donnell ML, Creamer M, McFarlane AC, Clark CR, & Silove D (2010). The psychiatric sequelae of traumatic injury. Am J Psychiatry, 167(3), 312–320. 10.1176/appi.ajp.2009.09050617Adolescent [DOI] [PubMed] [Google Scholar]
  7. Bulger EM, Kuhls DA, Campbell BT, Bonne S, Cunningham RM, Betz M, Dicker R, Ranney ML, Barsotti C, Hargarten S, Sakran J, Rivara F, James T, Lamis D, Timmerman G, Rogers SO, Choucair B, & Stewart RM (2019). Proceedings from the Medical Summit on Firearm Injury Prevention: A public health approach to reduce death and disability in the US. Journal of the American College of Surgeons, 229(4), 415–430.e412. doi: 10.1016/j.jamcollsurg.2019.05.018 [DOI] [PubMed] [Google Scholar]
  8. Califf RM, & Sugarman J (2015). Exploring the ethical and regulatory issues in pragmatic clinical trials. Clinical trials, 12(5), 436–441. doi: 10.1177/1740774515598334 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Centers for Disease Control and Prevention. (2003). Web-based Injury Statistics Query and Reporting System (WISQARS): Nonfatal Injury Data [Data set]. National Electronic Injury Surveillance System-All Injury Program. Retrieved from www.cdc.gov/injury/wisqars on 30 Oct. 2019.
  10. Friedman MJ, Resick PA, Bryant RA, & Brewin CR (2011). Considering PTSD for DSM-5. Depress Anxiety, 28(9), 750–769. doi: 10.1002/da.20767 [DOI] [PubMed] [Google Scholar]
  11. Gabbe BJ, Simpson PM, Harrison JE, Lyons RA, Ameratunga S, Ponsford J, Fitzgerald M, Judson R, Collie A, & Cameron PA (2016). Return to work and functional outcomes after major trauma: Who recovers, when, and how well? Annals of Surgery, 263(4), 623–632. doi: 10.1097/sla.0000000000001564 [DOI] [PubMed] [Google Scholar]
  12. Hoge CW, Riviere LA, Wilk JE, Herrell RK, & Weathers FW (2014). The prevalence of post-traumatic stress disorder (PTSD) in US combat soldiers: a head-to-head comparison of DSM-5 versus DSM-IV-TR symptom criteria with the PTSD checklist. Lancet Psychiatry, 1(4), 269–277. doi: 10.1016/s2215-0366(14)70235-4 [DOI] [PubMed] [Google Scholar]
  13. Hoge CW, Yehuda R, Castro CA, McFarlane AC, Vermetten E, Jetly R, Koenen KC, Greenberg N, Shalev AY, Rauch SA, Marmar CR, & Rothbaum BO (2016). Unintended Consequences of Changing the Definition of Posttraumatic Stress Disorder in DSM-5: Critique and Call for Action. JAMA Psychiatry, 73(7), 750–752. doi: 10.1001/jamapsychiatry.2016.0647 [DOI] [PubMed] [Google Scholar]
  14. Holbrook TL, Anderson JP, Sieber WJ, Browner D, & Hoyt DB (1999). Outcome after major trauma: 12-month and 18-month follow-up results from the trauma recovery project. Journal of Trauma, 46(5), 765–773. [DOI] [PubMed] [Google Scholar]
  15. Hunt JC, Sapp M, Walker C, Warren AM, Brasel K, & deRoon-Cassini TA (2017). Utility of the injured trauma survivor screen to predict PTSD and depression during hospital admission. Journal of Trauma and Acute Care Surgery, 82(1), 93–101. doi: 10.1097/ta.0000000000001306 [DOI] [PubMed] [Google Scholar]
  16. Kilpatrick DG (2013). The DSM-5 got PTSD right: Comment on Friedman (2013). Journal of Traumatic Stress, 26(5), 563–566. doi: 10.1002/jts.21844 [DOI] [PubMed] [Google Scholar]
  17. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, & Zwarenstein M (2015). The PRECIS-2 Tool: Designing trials that are fit for purpose. British Medical Journal, 350, 11. doi: 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
  19. Palinkas LA, & Zatzick D (2019). Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: Methods and applied case study. Administration and Policy in Mental Health, 46(2), 255–270. doi: 10.1007/s10488-018-0909-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Price M, Szafranski DD, van Stolk-Cooke K, & Gros DF (2016). Investigation of abbreviated 4 and 8 item versions of the PTSD Checklist 5. Psychiatry Research, 239, 124–130. doi: 10.1016/j.psychres.2016.03.014 [DOI] [PubMed] [Google Scholar]
  21. Rosellini AJ, Stein MB, Colpe LJ, Heeringa SG, Petukhova MV, Sampson NA, Schoenbaum M, Ursano RJ, & Kessler RC (2015). Approximating a DSM-5 diagnosis of PTSD using DSM-IV criteria. Depression and Anxiety, 32(7), 493–501. doi: 10.1002/da.22364 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Russo J, Katon W, & Zatzick D (2013). The development of a population-based automated screening procedure for PTSD in acutely injured hospitalized trauma survivors. General Hospital Psychiatry, 35(5), 485–491. doi: 10.1016/j.genhosppsych.2013.04.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Scheuer H, Engstrom A, Thomas P, Moodliar R, Moloney K, Walen ML., Johnson P, Seo S, Vaziri N, Martinez A, Maier R, Russo J, Sieber S, Anziano P, Anderson K, Bulger E, Whiteside L, Heagerty P, Palinkas L, & Zatzick D (2020). A comparative effectiveness trial of an information technology enhanced peer-integrated collaborative care intervention versus enhanced usual care for US trauma care systems: Clinical study protocol. Contemporary Clinical Trials, 19, 1–13. 10.1016/j.cct.2020.105970 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Simon GE, Beck A, Rossom R, Richards J, Kirlin B, King D, Shulman L, Ludman EJ, Penfold R, Shortreed SM, & Whiteside U (2016). Population-based outreach versus care as usual to prevent suicide attempt: Study protocol for a randomized controlled trial. Trials, 17(1), 452. doi: 10.1186/s13063-016-1566-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, & Chalkidou K (2009). A pragmatic-explanatory continuum indicator summary (PRECIS): A tool to help trial designers. Journal of Clinical Epidemiology, 62(5), 464–475. 10.1016/j.jclinepi.2008.12.011 [DOI] [PubMed] [Google Scholar]
  26. Ware JE, Snow KK, & Kosinski M (1993). SF-36 health survey: manual and interpretation guide. Boston, MA: The Health Institute, New England Medical Center. [Google Scholar]
  27. Weathers F, Keane TM, & Davidson JR (2001). Clinician-administered PTSD scale: A review of the first ten years of research. Depression and Anxiety, 13(3), 132–156. doi: 10.1002/da.1029 [DOI] [PubMed] [Google Scholar]
  28. Weathers F, Litz B, Herman D, Huska J, & Keane T (1993). The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. Paper Presented at the 9th Annual Conference of the International Society for Traumatic Stress Studies, San Antonio, TX. [Google Scholar]
  29. Weathers FW, Litz BT, Keane TM, Palmieri PA, Marx BP, & Schnurr PP (2013). The PTSD Checklist for DSM-5 (PCL-5). Scale available from the National Center for PTSD at www.ptsd.va.gov. [Google Scholar]
  30. Weiss DS, & Marmar CR (1997). The Impact of Event Scale - Revised In Wilson JP & Keane TM (Eds.), Assessing Psychological Trauma and PTSD (pp. 399–411). New York, NY: The Guilford Press. [Google Scholar]
  31. Zatzick D, Jurkovich G, Rivara FP, Russo J, Wagner A, Wang J, Dunn C, Lord SP, Petrie M, O’Connor SS, & Katon W (2013). A randomized stepped care intervention trial targeting posttraumatic stress disorder for surgically hospitalized injury survivors. Annals of Surgery, 257(3), 390. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Zatzick D, Jurkovich GJ, Rivara FP, Wang J, Fan MY, Joesch J, & Mackenzie E (2008). A national US study of posttraumatic stress disorder, depression, and work and functional outcomes after hospitalization for traumatic injury. Annals of Surgery, 248(3), 429–437. doi: 10.1097/SLA.0b013e318185a6b8 [DOI] [PubMed] [Google Scholar]
  33. Zatzick D, Russo J, Darnell D, Chambers DA, Palinkas L, Van Eaton E, Wang J, Ingraham LM, Guiney R, Heagerty P, Comstock B, Whiteside LK, & Jurkovich G (2016). An effectiveness-implementation hybrid trial study protocol targeting posttraumatic stress disorder and comorbidity. Implementation Science, 11, 58. doi: 10.1186/s13012-016-0424-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Zatzick D, Russo J, Lord SP, Varley C, Wang J, & Berliner L (2014). Collaborative care intervention targeting violence risk behaviors, substance use, and posttraumatic stress and depressive symptoms in injured adolescents: a randomized clinical trial. JAMA Pediatrics, 168. [DOI] [PubMed] [Google Scholar]
  35. Zatzick D, Russo J, Thomas P, Darnell D, Teter H, Ingraham L, Whiteside LK, Wang J, Guiney R, Parker L, Sandgren K, Hedrick MK, Van Eaton EG, & Jurkovich G (2018). Patient-Centered Care Transitions After Injury Hospitalization: A Comparative Effectiveness Trial. Psychiatry, 1–17. doi: 10.1080/00332747.2017.1354621 [DOI] [PubMed] [Google Scholar]
  36. Zoellner LA, Bedard-Gilligan MA, Jun JJ, Marks LH, & Garcia NM (2013). The evolving construct of posttraumatic stress disorder (PTSD): DSM-5 criteria changes and legal implications. Psychological Injury and Law, 6(4), 277–289. doi: 10.1007/s12207-013-9175-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES