Prediction of Posttraumatic Stress and Depression One-Month Post-Injury: A Comparison of Two Screening Instruments

Abstract

Objective:

To examine the combined and individual utility of 2 screening tools in prediction of depression and PTSD one-month post traumatic injury.

Method:

484 Level I Trauma Center patients were administered the Peritraumatic Distress Inventory (PDI) and Injured Trauma Survivor Screen (ITSS). Approximately 30 days post-injury, patients completed the Posttraumatic Stress Disorder Checklist for DSM–5 (PCL-5) and Patient Health Questionnaire (PHQ-8).

Results:

Receiver operating characteristic curve (ROC) curves for the PDI suggested a cutoff score of 17.5 predicting PTSD (Sensitivity = 70%; Specificity = 62%) and depression (Sensitivity = 74%; Specificity = 64%). For the ITSS, ROC curves suggested a cutoff score of 1.5 to predict PTSD (Sensitivity = 72%; Specificity = 60%) and depression (Sensitivity = 67%; Specificity = 62%). Inclusion of both instruments in regression analyses accounted for 2.4%-6.8% greater variance than 1 measure alone in predicting PCL-5 and PHQ-8 scores.

Conclusions:

The ITSS and PDI each demonstrated significant clinical utility in practice. Use of both measures, versus either alone, likely does not produce sufficient added clinical benefit. Follow-up screening and/or ongoing symptom monitoring is recommended as an adjunct to brief bedside screening.

Approximately 3 million adults in the United States are hospitalized in trauma centers annually (National Center for Injury Prevention & Control, 2020). Immediately following a traumatic event, most individuals display psychological distress that abates without intervention, but a subset will continue to suffer symptoms beyond the one-month required for a diagnosis of posttraumatic stress disorder (PTSD; e.g., Bonanno et al., 2012). Left untreated, symptoms tend to be chronic and debilitating (Kessler et al., 2017). Between 20%-40% of hospitalized traumatic injury survivors develop PTSD and/or depression within one-year of injury (Shih et al., 2010). Fortunately, early trauma-focused cognitive behavior therapy can prevent chronic PTSD among those with acute symptoms (Bryant et al., 2008) and symptoms at hospital admission can indicate who may go on to have persistent PTSD (Bonanno et al., 2012). Thus, the American College of Surgeons Committee on Trauma (2020) recommends routine screening for PTSD and depression via bedside screening tools to identify those who may benefit from intervention or follow-up services. Routine bedside rather than postdischarge screening is ideal due to the high risk of loss to follow-up in this population (Bryant et al., 2015). Current screening approaches across trauma centers are highly variable and poorly validated (Guess et al., 2019). Research is needed to validate efficient screening tools that can be used at bedside to identify patients at risk of posttraumatic psychopathology so that brief interventions can be provided. The Injured Trauma Survivor Screen (ITSS; Hunt et al., 2017) and Peritraumatic Distress Inventory (PDI; Brunet et al., 2001) have emerged as validated instruments that are viable candidates for integration into embedded mental health programs (Bunnell et al., 2018; Hunt et al., 2017; Nishi et al., 2010). This paper compared the ability of these tools to predict PTSD and depressive symptoms 30 days post-injury and whether administration of both measures added clinically meaningful predictive value over either measure alone.

The ITSS was developed based on a review of risk factors for PTSD and depression (Hunt et al., 2017). Items were determined by a panel of expert reviewers based on pre-, peri-, and posttrauma risk factors for PTSD and depression. They were later pilot tested with 139 trauma survivors on inpatient units (Hunt et al., 2017). Two items assess mental health prior to the traumatic event, two measure peritraumatic experiences, and five assess reactions to the traumatic event (e.g., “Since your injury do you find yourself crying and are unsure why?”).

The PDI is a self-report measure of physiological and emotional distress during and immediately after a traumatic event (Brunet et al., 2001), which are found to later predict PTSD (Ozer et al., 2003). The measure was initially validated with police officers and nonpolice peers (Brunet et al., 2001) but has since been used in injury samples in acute trauma care settings (Nishi et al., 2010) to predict PTSD and depression (Bunnell et al., 2018). The PDI exclusively focuses on acute peritraumatic experiences and, in contrast with the ITSS, the PDI does not include items measuring mental health prior to or following injury.

Method

We assessed 479 traumatic injury patients enrolled in the Trauma Resilience and Recovery Program (TRRP) between May 2018 and July 2020. Patients averaged 46.73 years old (SD = 19.67). Racial/ethnic composition was 38.4% African American, 54.5% White, 1.9% Hispanic/Latinx, .6% Asian and 1% “other.” The majority (81%) experienced nonassaultive injury (e.g., motor vehicle accident) compared to assaultive injuries (19%; e.g., gunshot wounds, stabbings). TRRP is a technology-enhanced stepped-care program (Ruggiero et al., 2020) that includes in-hospital completion of the ITSS and PDI as well as education about emotional and behavioral health recovery for patients exhibiting distress. Patients complete a 30-day follow-up telephone screen to assess PTSD and depressive symptoms; and, if indicated, receive referral to mental health treatment. The 30-day timeframe was chosen because a diagnosis of PTSD cannot be made until one-month following trauma (Kessler et al., 2017). According to the Medical University of South Carolina Institutional Review Board (IRB), this project was considered a quality improvement initiative, not “research” because it aims to improve quality of care in our treatment of traumatic injury patients. Therefore, the current project was not under IRB purview.

TRRP staff administered the PDI and ITSS in hospital. The PDI is a 13-item self-report measure. Items are rated on a 5-point Likert scale (0–4). Total scores range from 0–52. A total score ≥ 23 has been determined as the optimal cutoff for predicting PTSD and depression in hospital samples (Bunnell et al., 2018; Nishi et al., 2010). The ITSS is a 9-item self-report measure of risk for PTSD and depression post-injury. PTSD and depression are each assessed with 5 items. One item assesses risk for both. Items are rated as yes (1) or no (0). Scores of ≥ 2 on either scale indicate high risk. Staff contacted patients by telephone approximately 30 days after administration of the ITSS and PDI to conduct a mental health screen that included the Patient Health Questionnaire (PHQ-8; Kroenke et al., 2001) and PTSD Checklist for DSM–5 (PCL-5; Weathers et al., 2013). The PHQ-8 is a self-report measure with scores of ≥ 10 indicating depression. The PCL-5 is a 20-item self-report measure of DSM–5 PTSD symptoms. Scores of ≥ 30 indicate probable PTSD in hospitalized injury samples (Geier et al., 2019).

Data Analytic Plan

Receiver operating characteristic curve (ROC) analyses were conducted to determine the cut scores yielding the highest sensitivity and specificity for the PDI and ITSS predicting PTSD and depression scores. We balanced risk for false positives with false negatives due to the low resources available to conduct follow-up in most health care settings but danger of missing those at-risk who need additional care. Positive predictive value (PPV) and negative predictive value (NPV) were also calculated. Stepwise hierarchical linear regressions were run to assess the additive value of the PDI and ITSS in predicting PHQ-8 and PCL-5 total scores (continuous scores). Logistic regressions were conducted to examine the additive utility of the PDI and ITSS in predicting PTSD and depression diagnostic status.

Results

Descriptive Statistics

We screened 479 patients in hospital. On the ITSS, 218 (45.5%) exceeded the cutoff for PTSD risk (M = 1.57, SD = 1.40) and 209 (43.6%) for depression risk (M = 1.44, SD = 1.26). On the PDI, 166 (34.7%) exceeded the clinical cutoff (M = 17.98, SD = 11.82; range 0–52). Roughly half (n = 235) of the patients completed the 30-day follow-up (many were unreachable or did not respond to our contact attempts). Completers and noncompleters did not differ on baseline PDI (t[473] = .62, p = .54), and ITSS depression (t[474] = .05, p = .96) and PTSD (t[474] = .35, p = .73)scores or age (t[443.47] = 1.57, p = .12). At follow-up, according to the PCL-5 and PHQ-8, 47 patients (20%) had probable PTSD (M = 17.45, SD = 17.31) and 58 (24.6%) had significant symptoms of depression (M = 6.22, SD = 6.27).

ROC Curves

The ROC curve for the PDI predicting elevated 30-day PCL-5 scores was significant (AUC = .71, SE = .04, 95% CI [.63, .79], p < .001, PPV = 35.6%, NPV = 86.0%) and suggested a cutoff score of 17.5 (Sensitivity = 70%; Specificity = 62%). The ROC curve for the PDI predicting a significant 30-day PHQ-8 score was significant (AUC = .70 SE = .04, 95% CI [.62, .78], p < .001, PPV = 43.6%, PNV = 84.4%) and suggested a cutoff score of 17.5 (Sensitivity = 74%; Specificity = 64%). The ROC curve for the ITSS PTSD score predicting elevated 30-day PCL-5 scores was significant (AUC = .72, SE = .05, 95% CI [.63, .81], p < .001, PPV = 32.7%, PNV = 89.1%) and suggest a cutoff score of 1.5 (Sensitivity = 72%; Specificity = 60%). The ROC curve for the ITSS Depression score predicting a significant 30-day PHQ-8 score was significant (AUC = .68, SE = .04, 95% CI [.60, .76], p < .001, PPV = 37.1%, PNV = 85.0%). Examination of the coordinates suggest a cutoff score of 1.5 (Sensitivity = 67%; Specificity = 62%) predicting clinically significant PHQ-8 scores.

Hierarchical Linear Regressions

Four hierarchical linear regressions were conducted to estimate the relative utility of the PDI and ITSS and determine whether inclusion of both measures accounted for significant variance beyond one measure in predicting PCL-5 and PHQ-8 scores (See Supplementary Table 1). Two regressions were conducted for each outcome to ensure the order of variable entry (entering PDI measures first or ITSS measures first) did not change the outcome. Injury type (assaultive vs. nonassaultive) was included in each regression because assaultive injuries are well-documented predictors of posttraumatic psychopathology (Kessler et al., 2017).

The model with ITSS PTSD scores entered first to predict PCL-5 scores was significant, F(2, 222) = 17.67, p < .001, R² = .13 and ITSS PTSD was a significant predictor, b = .34, p < .001. Including PDI scores accounted for 5.5% of additional variance, a statistically significant change F(1, 219) = 14.81, p < .001. The reverse model whereby PDI scores were entered first to predict PCL-5 scores was also significant F(2, 220) = 22.40, R² = .16 and the PDI was a significant predictor b = .38, p < .001. The addition of ITSS PTSD scores accounted for an additional 2.4% of the model, a statistically significant change F(1, 219) = 6.45, p = .01.

The model where ITSS Depression scores were entered first to predict PHQ-8 was significant, F(2, 229) = 17.84, R² = .13 and the ITSS Depression was a significant predictor b = .36, p < .001. Including PDI scores explained an additional 6.8% of the variance, a statistically significant change, F(1, 228) = 19.47, p < .001. The regression where PDI scores were entered first in the model to predict PHQ-8 scores was also significant, F(2, 229) = 21.91, p < .001 R² = .15 and PDI scores were a significant predictor b = .39, p < .001. Including ITSS Depression scores to the model accounted for an additional 4.2% of the variance, a statistically significant change, F(2, 229) = 21.91, p < .001.

Logistic Regressions

Two logistic regressions were conducted to predict 30-day PTSD and depression statuses (present/absent) according to the PCL-5 and PHQ-8, respectively, while accounting for injury type (assaultive vs. nonassaultive). In the model to predict elevated PCL-5 scores, ITSS PTSD (β = .42, SE = .15, Wald = 7.66, p = .006, OR = 1.52, 95% CI [1.13, 2.05]) and PDI scores (β = .05, SE = .02, Wald = 7.37, p = .002, OR = 1.05, 95% CI [1.01, 1.09]) were significant predictors and injury type was not (β = .5−, SE = .44, Wald = 1.28, p = .26, OR = 1.65, 95% CI [.69, 3.94]). In the model predicting elevated PHQ-8 scores, PDI scores (β = .05, SE = .02, Wald = 10.19, p = .001, OR = 1.06, 95% CI [1.02, 1.08]) and ITSS Depression scores (β = .38, SE = .14, Wald = 6.77, p = .009, OR = 1.46, 95% CI [1.01, 1.93]) were significant whereas injury type was not (β = .57, SE = .41, Wald = 1.96, p = .16, OR = 1.77, 95% CI [.80, 3.91]).

Discussion

This is the first study to examine the predictive utility of two validated brief bedside screening tools for PTSD and depression 30 days post-injury. Both measures demonstrated adequate predictive and clinical utility according to ROC and neither emerged as superior to the other. Additional variance accounted for when both measures were included suggested marginal clinical benefit. Therefore use of one of these measures at bedside is likely most practical and efficient. As sensitivity and specificity estimates were modest, we suggest supplementing bedside screening with follow-up screening and/or ongoing patient symptom monitoring.

Neither measure was comparatively stronger than the other and both yielded adequate predictive utility. Decisions relating to bedside risk screening must balance efficiency, specificity, and sensitivity to optimize detection and minimize patient burden. Structurally, it is notable that both measures are brief and take roughly 2–4 min to administer and score. The PDI requires slightly more time to score and administer due to the additional items and use of a Likert scale compared to the yes/no format of the ITSS.

Neither the PDI nor ITSS demonstrated especially high predictive utility for 30-day PTSD and depressive symptoms. The sensitivity and specificity estimates of each measure according to ROC curves were adequate but not strong enough to suggest that either measure is sufficient to drive the full course of patient mental health care. These findings are consistent with research identifying a wide range of risk and protective factors for PTSD and depression (e.g., social support, additional life stress, course and speed of physical recovery, occupational and financial stressors), some of which cannot be accurately assessed or predicted at bedside (Ozer et al., 2003). These data highlight the critical importance of (1) supplemental postdischarge screening and/or symptom monitoring of trauma patients to minimize loss of patients with false negative risk scores to follow-up care and (2) additional research that leverages bedside screening measures to establish best practices in early intervention and ongoing risk assessment. Research consistently identifies peri- and posttraumatic risk factors to be the most potent predictors of posttraumatic depression and PTSD (Ozer et al., 2003). Therefore, screening for distress at bedside assists in identifying good candidates for early intervention. By administering the PDI or ITSS, clinicians have an opportunity to directly address and intervene to address acute distress, potentially mitigating development and exacerbation of depressive and trauma-related symptoms.

This study has several strengths, including the provision of suggested cutoff scores for the PDI and ITSS and data on their combined utility in a hospitalized trauma sample. The study was limited by use of self-report measures. The use of clinician-administered structured diagnostic interviews would have strengthened the design (Hunt et al., 2017). Second, there was high attrition due to data collection in the context of standard clinical operations, potentially contributing to bias in those who agreed to complete the follow-up. However, analyses of the 30-day screen data comparing completers and noncompleters on baseline variables indicated that patients were statistically comparable on a number of known sociodemographic risk factors. Third, suggested cutoff scores balanced consideration of sensitivity and specificity in recognition of the importance of minimizing loss to follow-up care among patients with false-negative screening results and the limited resources available in most trauma centers to provide follow-up care to patients with relatively low levels of risk. However, some trauma centers with adequate resources may want to shift cutoff scores downward to give greater priority to sensitivity over specificity. Taken together, findings are an important step in addressing the unmet mental health needs of traumatic injury patients. Additional research is needed to validate and improve the predictive utility of bedside screening tools and leverage these tools to improve quality of early intervention and follow-up care.