Clinical Decision Making Following Disasters: Efficient Identification of PTSD Risk in Adolescents

Abstract

The present study aimed to utilize a Receiver Operating Characteristic (ROC) approach in order to improve clinical decision-making for adolescents at risk for the development of psychopathology in the aftermath of a natural disaster. Specifically we assessed theoretically-driven individual, interpersonal, and event-related vulnerability factors to determine which indices were most accurate in forecasting PTSD. Furthermore, we aimed to translate these etiological findings by identifying clinical cut-off recommendations for relevant vulnerability factors. Our study consisted of structured phone-based clinical interviews with 2,000 adolescent-parent dyads living within a 5-mile radius of tornados that devastated Joplin, MO, and northern Alabama in Spring 2011. Demographics, tornado incident characteristics, prior trauma, mental health, and family support and conflict were assessed. A subset of youth completed two behavioral assessment tasks online to assess distress tolerance and risk taking behavior. ROC analyses indicated four variables that significantly improved PTSD diagnostic efficiency: Lifetime depression (AUC=.90), trauma history (AUC=.76), social support (AUC=.70), and family conflict (AUC=.72). Youth were 2–3 times more likely to have PTSD if they had elevated scores on any of these variables. Of note, event-related characteristics (e.g., property damage) were not related to PTSD diagnostic status. The present study adds to the literature by making specific recommendations for empirically-based, efficient disaster-related PTSD assessment for adolescents following a natural disaster. Implications for practice and future trauma-related developmental psychopathology research are discussed.

Youth exposed to disasters are vulnerable to trauma-related psychological problems (Udwin, Boyle, Yule, Bolton, & O'Ryan, 2000; Yule et al., 2000). Nearly half of disaster-exposed youth develop clinically elevated posttraumatic stress disorder (PTSD) symptoms (Adams et al., 2014; Furr, Comer, Edmunds, & Kendall, 2010; La Greca et al., 2013). Untreated PTSD in adolescence can persist into adulthood and result in pervasive functional impairment (Morgan, Scourfield, Williams, Jasper, & Lewis, 2003; Yule et al., 2000). It is crucial to identify youth at highest risk for developing PTSD following disasters and stem this deleterious trajectory via intervention. Similarly, given that many youth are also resilient following exposure to a traumatic event, such as a disaster, and may not warrant intervention, a parallel goal for post-disaster assessment is to identify those who should not be referred to unnecessary services. Many promising trauma-focused treatments exist to address youth psychological problems after disasters among those who are identified in need (Pfefferbaum, Sweeton, Newman, Varma, Nitiéma et al., 2014; Pfefferbaum, Sweeton, Newman, Varma, Noffsinger et al., 2014), but disruptions to community infrastructures can impede connecting these adolescents with mental health services. To meet youth mental health needs and guide resource allocation in the post-disaster context, the field needs clinical decision making tools that are efficient to administer and provide sound information for triage, diagnosis, case conceptualization, and treatment planning.

One approach to stratifying youth by PTSD risk is to assess well-established risk factors for each youth (Adams et al., 2014; Furr et al., 2010; La Greca et al., 2013; Norris et al., 2002). Guided by conceptual models rooted in a bioecological, developmental framework (La Greca, Silverman, Vernberg, & Prinstein, 1996; Weems & Overstreet, 2008), vulnerability factors identified in prior studies (Furr et al., 2010; Norris et al., 2002) can be organized across multiple levels of influence. Individual- or youth-level PTSD vulnerability factors include female gender; ethnic minority status; poverty; sustaining personal injury or severe threat to life; living in a highly disrupted community; high levels of secondary stress; prior trauma history; pre-disaster psychiatric problems (e.g., depression), and poor coping. Some youth-level PTSD risk factors can be assessed via computer-based performance tasks, such as distress tolerance (i.e., ability to persist through affective distress; Lejuez, Kahler, & Brown, 2003) and risk-taking propensity (i.e., tendency to engage in potentially harmful behavior with unpredictable rewards and punishments; Lejuez et al., 2002), which have been linked to PTSD symptoms in maltreated youth (Danielson, Ruggiero, Daughters, & Lejuez, 2010) and trauma-exposed adults (Vujanovic, Bonn-Miller, Potter, Marshall, & Zvolensky, 2011). Interpersonal factors include interpersonal conflict (Meiser-Stedman, Yule, Dalgleish, Smith, & Glucksman, 2005; Pelcovitz, Kaplan, DeRosa, Mandel, & Salzinger, 2000; Roussos et al., 2005; Udwin et al., 2000; Wickrama & Kaspar, 2007) and poor social resources (Stallard & Smith, 2007; Udwin et al., 2000; Vernberg, La Greca, Silverman, & Prinstein, 1996). Event-level factors included extreme, widespread damage (Goenjian et al., 1995; Goenjian et al., 2005); serious, ongoing financial hardship (Wickrama & Kaspar, 2007); and high injury and fatality rates (Bryant, Salmon, Sinclair, & Davidson, 2007; Landolt, Vollrath, Timm, Gnehm, & Sennhauser, 2005; Udwin et al., 2000). Prior findings highlight the importance of evaluating multiple types and sources of influence in predicting post-disaster psychopathology in adolescents.

Studies on post-disaster psychopathology typically report correlations summarizing associations between individual, interpersonal, and event level vulnerability factors and PTSD symptoms, or compare samples with versus without PTSD to find associated vulnerability factors. Such results identify candidate etiological factors, but are less helpful in providing post-disaster healthcare workers a risk-assessment framework. Moreover, a comprehensive assessment of all known vulnerability factors is typically not feasible in the post-disaster milieu. Healthcare providers must select variables and assessment tools that balance reliability and decision-making utility against time and resource burdens for patients and themselves. Thus, guidance is needed to develop efficient screening approaches by integrating multiple pieces of data to inform decisions about clinical care.

The use of receiver operating characteristic (ROC) analyses and related data-driven diagnostic efficiency statistics (Kraemer, 1992a; Youngstrom, 2014) represent a strategy for translating basic, post-disaster vulnerability studies into personalized, clinically useful information for decision-making. Rooted in signal detection theory, Bayesian statistics (Swets, Dawes, & Monahan, 2000) and evidence-based medicine (Chambless & Ollendick, 2001; Cohen, Sanborn, & Shiffrin, 2008; Jenkins, Youngstrom, Washburn, & Youngstrom, 2011), ROC analyses facilitate making empirically-derived algorithms to estimate probabilities of meeting diagnostic criteria for each person in light of his or her clinical profile (Straus, Glasziou, Richardson, & Haynes, 2011). This approach to prediction is more consistent, reliable, and efficient than clinical judgment, which often relies on arbitrary or idiosyncratic strategies for weighting multiple pieces of information to make clinical decisions (Meehl, 1954; Rettew, Lynch, Achenbach, Dumenci, & Ivanova, 2009).

Using ROC-based methods in post-disaster settings can reduce assessment-burden by prioritizing the most diagnostically predictive factors within the assessment protocol (Youngstrom & Frazier, 2013). Utilizing an assessment framework with optimized screening tools can address front-line psychiatrists', psychologists', and other disaster health workers' need to efficiently triage large groups of patients and establish clear plans of action for each person. Through the use of ROC procedures, improved assessment protocols have been developed for a wide range of pediatric mental health disorders including bipolar disorder (Youngstrom, Freeman, & Jenkins, 2009), mood disorders (Youngstrom, 2014), and anxiety disorders (Elkins, Carpenter, Pincus, & Comer, 2014; Van Meter et al., 2014). Most relevant to the present study, ROC approaches have also been applied to improving assessment protocols for PTSD in community (Suliman, Kaminer, Seedat, & Stein, 2005) and outpatient settings (You, Youngstrom, Feeny, Youngstrom, & Findling, in press), paving the way for more targeted referrals and interventions for individual who suffer from these symptoms. The present study seeks to build off these past studies by a) assessing PTSD specifically within a post-disaster context and b) including a wide-range of individual, interpersonal, and event-related vulnerabilities as part of the protocol. Broadening the assessment protocol beyond self-report PTSD symptoms may provide a more accurate assessment of post-trauma functioning (O'Donnell et al., 2008).

This study's overarching aim was to inform decision-making for clinicians and health workers following disasters to identify youth at highest risk for PTSD. The first goal was to identify the most potent predictive factors from a set of multi-method (caregiver report, self-report, behavioral tasks), theoretically and empirically justified candidates applied to disaster-exposed adolescents. The second goal was to generate clinical diagnostic likelihood ratios (DLRs; Jaeschke, Guyatt, & Sackett, 1994; Straus, Glasziou et al., 2011) for PTSD risk. As an extension of this latter goal, we aimed to provide a template for how the results can be used clinically. Specifically, we provide an example of how clinicians can apply DLRs by starting with the likelihood of having a diagnosis of PTSD in the general population (the base rate of PTSD in our sample, reported previously in Adams et al., 2014) and adjusting that estimate based on a vulnerable score on the factors found to be most predictive of PTSD risk. Given different base rates of PTSD for boys (6%), girls (8%), younger (ages 12-14; 6%), and older (ages 15-17; 8%) adolescents in our sample (Adams et al., 2014), we provide separate results for the increased likelihood of experiencing PTSD in each of these groups.

Method

Participants

Participants (n=2,000; 50.9% female) came from communities affected by the Spring 2011 tornadoes (Joplin, Missouri; Alabama). Participants were 12 to 17 years old (M=14.5, SD=1.7), 71% White/Caucasian, 26% Black/African-American, and 4% other per adolescent self-report. See (Adams et al., 2014) for further demographic and disaster details.

Procedure

Study design and procedures are detailed in Ruggiero et al. (2015). Selection criteria prioritized identification of families directly impacted by tornadoes. Latitude/longitude coordinates were used to obtain mailing addresses for households within a 5 mile radius of the impact areas (NOAA, 2011). Eligibility criteria included: 1) family residing at their address at the time of the tornado, 2) an adolescent aged 12 to 17 years and legal guardian, and 3) reliable home Internet access.¹ Reliable home Internet access was required because data were collected for a randomized controlled trial evaluating a web-based intervention for disaster-related mental health problems (Ruggiero et al., 2015). Addresses matched to a published phone number were contacted and screened by telephone. Addresses unmatched to a phone number were sent a letter explaining the study and a screening questionnaire that assessed inclusion criteria and requested telephone contacts. Adolescent- parent dyads independently completed structured, standardized computer-assisted telephone interviews conducted in a single interview session by well-trained professional interviewers. These interviews assessed PTSD and depression symptoms, substance use, trauma history, disaster impact, social support, and family conflict approximately 9 months (M=8.8, SD=2.6) following the tornado. The overall cooperation rate--(i.e., number screened/number screened + screen-outs + unknown eligibility)--was 61%. Adolescents received a $15 incentive for the interview. All adolescents were provided log-in information to a study website upon completion of the interview.

When adolescents signed in to the study website subsequent to the interview, they were invited to complete two brief game-like computer tasks (BART-Y and BIRD, described below). In total, 352 adolescents completed these tasks. Completers did not differ from noncompleters on any demographic or clinical variables (all ps>.10).

Measures

Main outcome: PTSD

The National Survey of Adolescents-Replication PTSD module (Resnick, Kilpatrick, Dansky, Saunders, & Best, 1993; Ruggiero et al., 2015) assessed the 17 DSM-IV (APA, 2004) PTSD symptom criteria in adolescents. Only symptoms that had been present at least 2 weeks and since the tornado were assessed. Total scores were then dichotomized as either having PTSD or not based on whether symptom counts in each diagnostic cluster were met and functional impairment was endorsed (see Kilpatrick et al., 2003 for psychometric details).

Event related vulnerability: Disaster impact

Caregivers reported on the family's experiences with the tornado, including property damage (to house, vehicle, furniture, personal item and pet), loss of services for over a week (water, electricity, clean clothing, food, shelter, transportation, and spending money), and interpersonal impact (present for the tornado, feared death, feared for safety of others, fear of injury to self, feared for pets). Responses were dichotomized (0=no/1=yes) and summed; higher scores meant worse severity (range = 0–17; M=3.71; SD=2.61).

Interpersonal-related vulnerability

Social support

The Social Support for Adolescents Scale (SSAS; Seidman et al., 1995) assessed the extent to which youth could turn to their mothers, fathers, siblings, close friends, and peers for 1) emotional social support; 2) instrumental social support; and 3) fun. Responses used a 3-point Likert scale ranging from Not at all to A great deal. Higher scores indicated elevated social support. Global support scores were summed across relationships and averaged across social support types. Total scores ranged from 1.67 to 10 (M=7.09; SD=1.64). Reliability (α=.80) was good, consistent with past research (Birman, Trickett, & Vinokurov, 2002).

Conflict

The Conflict Belief Questionnaire Short Form (CBQ; Prinz, Foster, Kent, & O'Leary, 1979; Robin & Foster, 1989). Adolescents completed the CBQ Short Form, a 20-item true/false measure of parent-adolescent conflict. A total score is computed; higher scores indicate greater conflict. Total scores ranged between 0 and 20 across administrations (M=2.78; SD=3.91). Similar to past research using the CBQ Short Form (Kane & Garber, 2009), this scale had excellent internal consistency in our sample (α=.90).

Individual-related vulnerability

Trauma history

Adolescents reported their experience of five different types of potentially traumatic events including physical assault, physical abuse, witnessing domestic violence, witnessing community violence, and serious accidents. Behaviorally specific prompts probed each trauma type, consistent with reliable and valid questions used in the National Survey of Adolescents (NSA; Kilpatrick et al., 2000). A count of the potentially traumatic event types endorsed by each adolescent in his or her lifetime indexed trauma history severity (range =0–5; M=1.03; SD=1.13).

Distress tolerance

The Behavioral Indicator of Resiliency to Distress (BIRD; Danielson et al., 2010; Lejuez, Daughters, Danielson, & Ruggiero, 2006) is a behavioral task that measures distress tolerance (DT). Adolescents selected a series of moving targets on a computer screen, getting points (when successful) or loud, unpleasant noises (when unsuccessful). Adolescents could exchange their points for a prize at the end of the task. Trials became faster, more challenging, and more aversive as the task progressed. During the final level of testing, adolescents could elect to stop at any time, but quitting hypothetically influenced the magnitude of their prizes. DT was measured by persistence (i.e., total time in minutes to quit) on the final level (range=0.03–7.30; M=2.51; SD=1.93). Higher times indicated higher levels of DT (Lejuez et al., 2006).

Risk taking

The Balloon Analog Risk Task-Youth (BART-Y; Lejuez et al., 2007) assessed risk-taking propensity by accounting for Participants' inflated virtual balloons via mouse click. Each pump earned the participant one point. If a balloon was pumped past its pre-set explosion point, the participant lost all that balloon's points. Each pump increased the likelihood of popping. Adolescents could stop pumping and bank each trial's points. The total number of pumps across 30 balloon trials quantified risk-taking propensity (range=1–203; M=101.17; SD = 45.05).

Substance Use

The CRAFFT Substance Abuse Screening Tool (Knight, Sherritt, Shrier, Harris, & Chang, 2002) is a six-item (yes/no) measure of risky substance use behavior. Higher summed scores indicate greater risk. The present study assessed for lifetime substance abuse. CRAFFT total scores ranged between 0 and 5 (M=0.18; SD=0.65). The CRAFFT reliability (α=0.54) was consistent with prior work using the CRAFFT (Dhalla, Zumbo, & Poole, 2011).

Depression

The National Survey of Adolescents-Replication MDD module (Resnick et al., 1993) assessed the 9 DSM-IV (APA, 2004) depression symptom criteria in adolescents. Symptoms were scored dichotomously (1=present/0=absent). Analyses used lifetime MDD symptom count as a dimensional variable (Range 0–9; M=1.35; SD=1.94). Reliability (α=.80) was comparable to previous research with this measure (Boscarino, Galea, Ahern, Resnick, & Vlahov, 2002).

Data Analytic Approach

Receiver Operating Characteristics (ROC) analyses followed recommendations by Youngstrom (2014). Back-to-back histograms investigated the degree of degeneracy across indicators (i.e., high scores on an indicator variable corresponding to no diagnosis; Van Meter et al., 2014). Areas under the curve (AUC) analyses quantified how well each predictor classified PTSD diagnoses (i.e., sensitivity) and identified cases that did not have PTSD (i.e., specificity). Per Swets' (1988) recommendations, AUC values above 0.70 were deemed adequate predictors of PTSD status. Hanley and McNeil's (1983) test of dependent AUCs tested which indicator had the best predictive accuracy for PTSD diagnoses. For the most discriminating predictors (AUC>.70), we created multilevel diagnostic likelihood ratios based on tertiles (i.e., balanced groups of high, medium, and low scores; Straus, Tetroe, & Graham, 2011). DLRs less than 1 indicate that corresponding predictor scores are less likely to lead to a diagnosis of PTSD. For DLR scores between 0.10 and 0.25, one can be moderately certain that the person does not have a PTSD diagnosis in most settings (Straus, Tetroe, & Graham, 2011). Scores above 1 indicate an increased risk for PTSD diagnosis. In cases, where age and sex correlate to the outcome, logistic regressions were used to examine if the influence of predictors varied as a function of these demographic variables. Similarly, as PTSD-risk following a natural disaster is multifaceted (Furr et al., 2010), and indicators for psychological distress should be studied in combination rather than isolation (Beauchaine & Gatzke-Kopp, 2012), interaction terms between predictor variables were created and logistic regression analyses tested if sets of predictors showed incremental value in identifying PTSD (i.e., interactions between predictor variables were tested). Finally, posterior probabilities were calculated for each vulnerable score range on significant indicators for PTSD. Posterior probabilities represent the increased risk for presenting with PTSD once accounting for a vulnerable score on a given indicator. Posterior probabilities were then used to complete standardized nomograms to demonstrate how clinicians could translate ROC analyses into real time decision making regarding patient-risk for PTSD especially for patients who present with more than one indicator of risk.

Results

Descriptive data is provided in Table 1 and Figure 1. Table 1 presents bivariate correlations between all of the study variables. Of note, females and older adolescents were more likely to report a diagnosis of PTSD since the natural disaster (p < .05). Figure 1 illustrates a series of population pyramids comparing the distribution of different vulnerability scores for cases without PTSD versus those with a PTSD diagnosis. Very few examples of degeneracy (i.e., a high vulnerability score corresponding to no diagnosis) were identified across the indicators.

Table 1.

Correlations between study variables

Variables		1	2	3	4	5	6	7	8	9	10
1)	PTSD Dx
2)	Depression	.48**
3)	Trauma	.27**	.43**
4)	Conflict	.24**	.38**	.28**
5)	Support	−.19**	−.35**	−.21**	−.43**
6)	Substance Use	.29**	.34**	.32**	.30**	−.19**
7)	Disaster	.04	.04	.04	.01	−.03	.01
8)	Risk Taking	.08	.08	.10*	.05	.00	.04	−.07
9)	Distress	.04	−.01	.03	.02	.00	.06	−.01	.13**
10)	Sex	−.05*	−.09**	.13**	−.10**	−.02	.02	−.01	.04	−.01
11)	Age	.06*	.11**	.11**	.11**	.00	.20**	.00	.02	.10	−.05*

Note: PTSD Dx = National Survey for Adolescents-Replication (NSA-R), PTSD Module since tornado Depression = NSA-R, Lifetime Depression Module; Trauma History = NSA-R, Trauma Module; Conflict = Conflict Belief Questionnaire Short Form (CBQ); Support = Social Support for Adolescents Scale; Substance Use = Substance Abuse Screening Test (CRAFFT) scores; Disaster = Counting variable consisting of post-disaster loss of services, property damage, and interpersonal impact; Risk Taking = Number of pumps on the BART; Distress Tolerance = Time on the BIRD. Sex = Adolescent participant sex (Female = 0; Male = 1); Age = Chronological age of adolescent participants.

^*p < .05,

^**p < .01

Figure 1.

Population pyramids for predictor variable distributions for those with a diagnosis of PTSD (green bars, right of center axis) versus no diagnosis of PTSD (blue bars, left of center axis). Predictors represented in each cell are as follows: (A) Number of prior potentially traumatic events, PTEs. (B) Total scores on the Social Support for Adolescents questionnaire. (C) Lifetime total score on the CRAFFT substance abuse screener. (D) Total time to quit on the BIRD task. (E) Depression symptoms, count. (F) Disaster impact severity score. (G) Number of pumps on non-explosion trials of the BART task. (H) Total scores on the Conflict Beliefs Questionnaire. Figures generated in SPSS.

Of the 2,000 adolescents completing baseline assessments, 126 met diagnostic criteria for PTSD, well exceeding the recommended minimum of 20 cases for estimating diagnostic efficiency parameters (Kraemer, 1992b). Table 2 presents AUC statistics and corresponding Cohen's d scores. Distress tolerance and risk taking propensity did not discriminate PTSD significantly better than chance (p > .05). Furthermore, disaster impact (AUC=.62) and lifetime substance abuse (AUC=.68) were deemed poor predictors (i.e., AUC <.70) of PTSD. The remaining four indicators (lifetime depression, trauma history, social support, family conflict) performed fair or better (Swets, 1988), with effect sizes in the medium (d >.50) to large (d >.80) range (Cohen, 1988). Analyses suggested that none of the indicators varied by sex (p > .05) or age (p > .20). Thus, the potential consequences for each significant indicator were consistent for boys, girls, and all adolescents within our population.

Table 2.

Diagnostic efficiency of predictors of PTSD diagnosis

Predictors	Area Under The ROC Curve	95% CI	Cohen's d
Event-Related Vulnerability
Disaster Impact	.62**	[.58, .67]	0.43
Interpersonal-Related Vulnerability
Social Support	.70 **	[.66, .75]	0.74
Family Conflict	.72 **	[.66, .78]	0.82
Individual-Related Vulnerability
Trauma History	.76 **	[.71, .80]	0.99
Substance Use	.68**	[.63, .77]	0.66
Lifetime Depression	.90 **	[.87, .93]	1.81
Risk Taking	.60	[.49, .72]	0.36
Distress Tolerance	.56	[.46, .66]	0.21

Note: Lifetime Depression = National Survey for Adolescents-Replication, Lifetime Depression Module; Trauma History = NSA-R, Trauma Module; Family Conflict = Conflict Belief Questionnaire Short Form (CBQ); Social Support = Social Support for Adolescents Scale; Substance Use = Substance Abuse Screening Test (CRAFFT) scores; Disaster Impact = Counting variable consisting of post-disaster loss of services, property damage, and interpersonal impact; Risk Taking = Number of pumps on the BART; Distress Tolerance = Time on the BIRD.

^**p < .01. Bold = >.70, a recommended cutoff point for when a variable may serve as an adequate predictor (Swets, 1998).

It was next tested which of these four constructs (lifetime depression, trauma history, social support, and family conflict) forecasted the greatest risk for PTSD within our sample (Hanley & McNeil, 1983). Lifetime depression predicted PTSD diagnosis significantly better than trauma history (z=7.07, p<.001), social support (z=10.00, p<.001), and family conflict (z=6.04, p<.001). Trauma history outperformed social support (z=2.34, p<.05); however, post hoc correction rendered this finding nonsignificant. Thus, lifetime depression performed best, and all other variables were functionally equivalent in predicting PTSD diagnosis.

We then examined how different score ranges on each significant indicator corresponded to the likelihood of presenting with a diagnosis of PTSD. Specifically, diagnostic likelihood ratios (DLRs) across three equal cutpoints (tertiles) were calculated (Straus et al., 2011). As shown in Table 3, there is a linear pattern of increasing DLRs (i.e., heightened risk for PTSD) with more vulnerable scores. Scores in the low and medium range across variables indicated that adolescents with these scores were less likely to experience PTSD compared to the base rate in the population (DLRs < 1). Meanwhile, adolescents who scored in the high range were at greater risk for presenting with a diagnosis of PTSD (DLRs > 1; Straus et al., 2011).²

Table 3.

Diagnostic Likelihood Ratios Predicting PTSD

		Score Range
		Low	Middle	High
Lifetime Depression	Score	0 Symptoms	1–2 Symptoms	Greater than 2 Symptoms
	DLR	0.10	0.22	3.52
Trauma History	Score	No History	One Past Event	Greater than 1 Traumatic Events
	DLR	0.28	0.65	2.71
Social Support	Score	8.01+	6.34–8.00	0–6.33
	DLR	0.31	0.77	2.20
Family Conflict	Score	0	1–2	3+
	DLR	0.44	0.56	2.04

Note: Lifetime Depression = National Survey for Adolescents-Replication, Lifetime Depression Module; Trauma History = NSA-R, Trauma Module; Social Support = Social Support for Adolescents Scale (scores for social support reversed in the table so that low corresponds to high social support to represent low risk; Family Conflict = Conflict Belief Questionnaire Short Form (CBQ). DLR = Diagnostic Likelihood Ratio. + = All scores greater than 8.01.

To better understand the level of risk associated with a vulnerable score, DLRs from the most vulnerable group were taken and inserted into Table 4. Table 4 demonstrates how the DLRs can be interpreted clinically by transforming the likelihood of having a diagnosis of PTSD in the general population (i.e., prior probability) to the percentage chance of having PTSD with a vulnerable score (posterior probability; Youngstrom, 2014). Because of different base rates for boys, girls, younger (ages 12-14), and older (ages 15-17) adolescents in our sample (Adams et al., 2014), separate results are reported for the increased likelihood of experiencing PTSD in each of these groups in light of vulnerable scores on each indictor. As seen in the posterior probabilities in Table 4, findings were both statistically and clinically significant. Youth were approximately 2 to 3 times more likely to have PTSD if they had elevated scores on any of the four predictor variables.

Table 4.

Pretest Probability and Posttest Probability for PTSD Diagnosis based on Vulnerable Scores

Measures	Prior Probability	DLR	Posterior Probability	Level (Test Positive Rate)
Lifetime Depression	7%	3.52	20%	0.41
Girls	8%	3.52	23%
Boys	6%	3.52	17%
Younger Adolescents	6%	3.52	17%
Older Adolescents	8%	3.52	23%
Trauma History	7%	2.71	17%	0.35
Girls	8%	2.71	19%
Boys	6%	2.71	15%
Younger Adolescents	6%	2.71	15%
Older Adolescents	8%	2.71	19%
Social Support	7%	2.20	14%	0.56
Girls	8%	2.20	16%
Boys	6%	2.20	12%
Younger Adolescents	6%	2.20	12%
Older Adolescents	8%	2.20	16%
Family Conflict	7%	2.04	13%	0.31
Girls	8%	2.04	15%
Boys	6%	2.04	12%
Younger Adolescents	6%	2.04	12%
Older Adolescents	8%	2.04	15%

Note: Lifetime Depression = National Survey for Adolescents-Replication, Lifetime Depression Module; Trauma History = NSA-R, Trauma Module; Social Support = Social Support for Adolescents Scale; Family Conflict = Conflict Belief Questionnaire Short Form (CBQ). DLR = Diagnostic Likelihood Ratio. Level (Test Positive Rate) = Percentage of cases scoring at or above a given diagnostic threshold.

Finally, two-way interactions between predictors were examined to test whether having a vulnerable score on more than one indicator for PTSD conferred additional risk. The interaction between social support and family conflict (B=.03, SE=.02; W=4.59, p=.03) was the only significant interaction between the indicator variables (all other ps>.10). Results suggested youth who endorsed low social support (score <6.33) and high levels of conflict with their parents (scores ≥3) were at increased risk for PTSD following natural disasters. Similar to Table 4, Figure 2 displays two probability nomograms that can also be used to highlight how DLRs can be used to interpret increased risk for a given diagnosis. In cases where there are multiple vulnerabilities that explain risk above and beyond one of the vulnerabilities independently (i.e., a significant interaction) clinicians can line probability nomograms side by side (Youngstrom, 2014). As seen in Figure 2, the left side of the left nomogram represents the general population's prior probability (i.e., base rate) for PTSD (7% in our study; Adams et al., 2014). The middle line of the nomogram represents the calculated DLR for a given predictor of PTSD. For instance, in the case of family conflict, the highest tertile of family conflict corresponded to a DLR of 2.04. With this information, clinicians would then draw a straight line from the prior probability through the DLR to calculate the posterior probability. In this case, individuals with high family conflict have a 13% of presenting with PTSD (also shown in Table 4). This posterior probability now becomes the prior probability in the second nomogram (displayed on the right). The DLR for this second nomogram comes from the second significant indicator in the interaction, in this case low social support (DLR = 2.20). We conclude by then repeating the step of drawing a line from the prior probability through the DLR to understand the posterior probability of having both vulnerability scores. As shown in Figure 2, an adolescent high in parental conflict goes from having a 13% probability of having a diagnosis of PTSD to a 25% probability if they also report low levels of social support (nearly a four-fold increase).³

Figure 2.

Nomograms for adolescents with high conflict and low social support

Discussion

Given the deleterious effects of PTSD in adolescence, there is great public health utility in reducing risk for its onset in the aftermath of a disaster. Mental health service resources are scarce in disaster-stricken communities, underscoring the value of empirically-based, efficient clinical-decision making tools. Present data show that, in the wake of a natural disaster, lifetime depression and trauma history, as well as social support and family conflict, effectively predict PTSD status among disaster-exposed youth. Depression emerged as the most diagnostically powerful, increasing the odds of PTSD 3.5-fold among youth reporting more than two depression symptoms. Assessing depression history may be highly informative for post-disaster PTSD risk screening. Results also underscore the importance of assessing other psychosocial factors such as interactions with family members, and peer social support. The prominent roles low social support and high family conflict showed for PTSD risk prediction are consistent with a recent meta-analysis of pediatric PTSD risk factors (Trickey, Siddaway, Meiser-Stedman, Serpell, & Field, 2012), supporting potential protective effects of social connectedness in the aftermath of trauma (Allen, Marcelin, Schmitz, Hausman, & Shultz, 2012; Landau, 2013; Norris & Stevens, 2007). The current study extends research by providing specific cut score ranges and DLRs to promote individualized risk probability estimation for disaster-exposed youth.

Although it is not surprising that trauma and mental health history are clinically useful in predicting post-disaster PTSD onset, it was surprising that individual-related factors would have greater diagnostic utility than impact of the disaster itself (i.e., event-related vulnerability). These findings highlight the utility of the ROC approach. A plethora of research has shown that traumatic event incident characteristics, such as degree of property damage (Goenjian et al., 1995; Goenjian et al., 2005; van den Berg, Wong, van der Velden, Boshuizen, & Grievink, 2012), are significantly related to post-disaster mental health symptoms (Goenjian et al., 2001; Parson, Fussell, Rhodes, & Waters, 2012; Zhang, Shi, Wang, & Liu, 2011). The current study suggests that while such disaster incident characteristics may contribute to the etiology and/or maintenance of PTSD per prior research, assessing these variables in post-disaster screenings may not improve prediction of PTSD beyond chance—and other variables are more effective for identifying those at highest risk.

It was also somewhat surprising that other individual characteristics (distress tolerance and risk-taking propensity) that have been previously empirically linked with PTSD risk did not improve diagnostic accuracy in this post-disaster sample. The extant literature suggests these factors may still be relevant in post-disaster mental health. Assessment of these characteristics may help in intervention tailoring (Zatzick et al., 2011) and/or detecting other mental health outcomes (e.g., externalizing problems; Daughters et al., 2009). There has been a recent emphasis on the utility of behavioral assessment tasks, including those that incorporate physiological measures, to potentially help inform empirically based assessment in mental health (Youngstrom & de Los Reyes, 2015). Additional research is needed to ultimately determine the utility of behavioral tasks and physiological measures in PTSD risk prediction.

Study strengths include the behaviorally-specific and comprehensive assessment of trauma history and the calculation of PTSD rates by gender and age group in our sample. The ROC approach also reminds clinicians to consider base rates of target problems, which vary by setting and context (Meehl, 1954; Straus, Tetroe, & Graham, 2011). For example, the base rate of PTSD in a clinical setting (e.g., 7.2–14.4%; Zimmerman & Mattia, 1999) is higher than community base rates (5%; Kilpatrick et al., 2003; McLaughlin et al., 2013), including among disaster-exposed community populations (6.7% in this sample; Adams et al., 2014). The DLRs provide locally contextualized interpretation in the form of a Bayesian posterior probability.

Despite these strengths, there are also noteworthy limitations. First, the length of time since the tornado for the initial post-disaster assessment in the current study (average of 8 months) prevented assessment of pre-existing vulnerability factors and symptoms immediately following the disaster. It is possible that certain factors such as social support or family dynamics may have been strong before the storm or in its immediate aftermath, then deteriorated by the time the assessments took place. Alternatively, families may have experienced temporary increases or breakdowns in support and conflict immediately following the storm that resolved by the assessments in this study. Thus the utility of these factors in predicting PTSD may vary by timing of the assessment—a hypothesis that should be tested in future studies. Second, a relatively lower number of youth who completed the behavioral assessment tasks in comparison to those who completed the interviews, which may prevent definitive conclusions from being drawn about the utility of behavioral assessment tasks in PTSD risk assessment, as noted above. Relatedly, while the multi-method approach of the study was a strength, findings indicated that only self-report measures predicted self-reported PTSD diagnostic status. Thus, shared method variance between the measures may have contributed to the strong effects seen for many of the self-report measures, and the null findings concerning distress tolerance and risk-taking propensity in predicting PTSD status. Because most assessment batteries rely on self-reported data, understanding which self-report data is most valuable in forecasting PTSD within a post-disaster context carries significant clinical value.

Post-Disaster PTSD Risk Assessment Practice Recommendations

One of the most effective strategies for achieving accurate and efficient mental health risk prediction is the use of psychological measures that are low burden in their cost and administration time and high yield in their output (i.e., low burden:high yield, LB:HY). These considerations are particularly important following disasters, when efficient resource allocation is key during recovery. Empirically-based clinical decision making—ROC analyses in particular—help achieve LB:HY goal by winnowing down the potential list of variables to screen to those most likely to inform PTSD risk status. Our ROC analyses here show that youth with high (problematic) scores across four key risk factors (lifetime depression, trauma history, social support, and family conflict) can be identified via brief sets of questions. Consideration of these four factors also will allow public health workers to triage a large group of people into a low risk category correctly, providing more fine-grained probability estimates for higher risk individuals based on presence or absence of specific risk factors.

Practical considerations for how to assess the four key post-disaster PTSD risk factors identified here (depression, trauma history, social support, family conflict) will depend somewhat on the resources (e.g., time, personnel, cost, computer and/or reliable internet access) available in the particular post-disaster context in which the recommendations are being applied. These four variables were assessed in the current study via phone-based interview. Each of the key risk factors identified here also may be feasibly and validly assessed using paper-and-pencil questionnaires or tablet-based surveys. Once these values are obtained, scores can be compared to the vulnerability scores reported in Tables 3 and 4 to yield a DLR and a Posttest Probability, as described earlier, to guide clinical decisions about intervention and referrals. Future studies should evaluate the correspondence between other publically available standardized measures for assessing these constructs and the measures used in the current study in predicting PTSD risk. Additional work may involve development of tools that can automate the process of calculating risk scores for high volume screening events in the post-disaster context.

Implications for Trauma-Related Developmental Psychopathology Research

Current findings also inform future trauma-related developmental psychopathology research directions to further evaluate and improve PTSD risk assessment among youth. Given the relevance of developmentally sensitive windows with regard to trauma exposure, the four identified potent predictors in this study, and their respective cut-scores, should be evaluated in prospective investigations across different age groups of youth with future disasters and mass traumatic events. Future studies can similarly inform the ideal timing of post-disaster PTSD risk assessment, which is a commonly posed question in practice (Pfefferbaum et al., 2014ab). In addition, given that changes to the PTSD diagnostic criteria have occurred since the execution of the current study (e.g., DSM-5 PTSD includes four clusters versus three clusters of symptoms), it will be important to determine if the results presented here replicate in a post-disaster youth sample assessed with the DSM-5 PTSD criteria. Finally, ROC analyses may be applied to other research data sets and other types of traumatic events to similarly derive clinically applicable information for a variety of problems and populations (Youngstrom, Choukas-Bradley, Calhoun, & Jensen-Doss, 2014).