Diagnostic validity of the PC-PTSD screen in college students

Abstract

Objective:

The purpose of this study was to test the diagnostic validity of the Primary Care PTSD screen (PC-PTSD) in a generalizable college sample and to examine potential differences in its predictive efficacy according to sex and racial/ethnic identity. An exploratory aim was to determine whether PC-PTSD symptom items differentially predicted PTSD diagnostic status.

Participants:

Data from 475 undergraduates were analyzed.

Methods:

Logistic regressions were conducted to examine the relationship between different PC-PTSD endorsement thresholds and probable PTSD among various subsamples. Follow-up tests of diagnostic accuracy were performed.

Results:

Results of this study indicated that the PC-PTSD identified PTSD among college students with poor accuracy. Furthermore, the PC-PTSD did not demonstrate equal predictive validity across neither sex nor racial/ethnic identity. Endorsement of reexperiencing symptoms appeared to be the strongest predictor of PTSD.

Conclusions:

Results highlight the clear need for a validated PTSD screener effective for a diverse college population.

Trauma & PTSD among college students

College is a period of increased vulnerability to potentially traumatic events, with exposure estimates ranging from 67% to 85%.^1–3 The college years represent the age group at greatest risk not only for exposure to traumatic events, but also for the development of posttraumatic stress disorder (PTSD).⁴ Extant research suggests that approximately 9% of college students have PTSD,⁵ estimates which are similar to and, in some cases, higher than those found among the general population (4%-9%).^4,6 Efforts to screen for PTSD on college campuses are imperative, given notable prevalence estimates and associated detrimental outcomes (e.g., risky alcohol use,⁷ educational attainment difficulties,⁸ and college dropout⁹) In order to properly screen for PTSD among college students, consideration of important demographic factors is warranted, as it is well known that PTSD estimates vary according to sex^4,6 and ethnic/racial identity.^10,11

Recent media attention spotlighting “the college student mental health crisis”^12,13 has highlighted drastic increases in anxiety and depression among college students over recent years, but has largely excluded trauma and PTSD from the discussion. This exclusion is particularly notable in light of the 205% increase in reported on-campus forcible sex crimes over the past couple of decades.¹⁴ Given high rates of sexual and other forms of interpersonal violence¹⁵ on college campuses, coupled with high comorbidity between PTSD, anxiety, and depression,¹⁶ as well as college being a known vulnerability period for trauma exposure and development of PTSD, the need to study PTSD among college populations specifically is highly warranted. Moreover, there has been a major increase in willingness to seek mental health treatment among college students,¹³ suggesting that the college years offer a timely window in which to deliver validated brief screens for PTSD to ensure that those impacted by trauma exposure receive critical intervention efforts.

PC-PTSD

One of the most commonly implemented screeners for PTSD is the Primary Care PTSD Screen (PC-PTSD).¹⁷ The PC-PTSD is a four-item measure developed, broadly, to screen for PTSD in primary care settings. Despite its popularity in both clinical and research settings, the existing literature to date has not established the diagnostic validity of the PC-PTSD amongst a college sample. Instead, the majority of PC-PTSD validation samples have been comprised of Veterans.^17–20 This is problematic given that Veteran samples endorse higher levels of trauma exposure and PTSD^21,22 compared to non-Veteran samples, tend to be overwhelmingly male,²⁰ and are exposed to types of traumas that are not common among the general population (i.e., combat exposure). The limited focus of the PC-PTSD validity among Veterans is insufficient to presume validity amongst other populations.

A smaller subset of the literature has examined the diagnostic validity of the PC-PTSD amongst substance use treatment-seeking samples.^23–25 Additionally, one study compared the efficacy of the PC-PTSD to that of the Posttraumatic Stress Disorder Checklist (PCL-C)²⁶ in screening for PTSD among inpatients receiving treatment at a Level I trauma center.²⁷ Although these studies extend beyond military literature, the use of treatment-seeking and acute samples continues to limit generalizability.

To the best of our knowledge, no study to date has investigated the diagnostic validity of the PC-PTSD within a non-Veteran or non-treatment seeking/inpatient sample, let alone within a college sample. The one exception to this that we could find was a brief report on the psychometric properties of the Korean version of the PC-PTSD for the fifth edition of the Diagnostic and Statistical Manual (K-PC-PTSD-5),²⁸ which was conducted among a sample of Korean civilians. Thus, further research is needed to validate the use of the PC-PTSD in a population that is more generalizable (i.e., non-Veteran, non-clinical, ethnic/racially diverse, and more evenly distributed with regard to sex) and particularly one characterized by greatest risk for trauma exposure and PTSD (i.e., college-aged individuals).

PC-PTSD predictive validity to date

Taken together, the majority of the extant literature has demonstrated that a cutoff score of 3 on the PC-PTSD maximized either predictive efficiency^{17,19,23,25,28} and/or sensitivity.^18–20,23 Test sensitivity refers to the ability of a test or screener to correctly identify individuals with a disease or condition (e.g., correctly identify an individual who meets criteria for PTSD), whereas test specificity refers to the ability of a test or screener to correctly identify individuals without a disease (e.g., correctly identify an individual who does not meet diagnosis for PTSD). Predictive efficiency, sometimes referred to as “optimal efficiency”, refers to the percentage of times that a test produces the correct answer relative to the total number of tests. Although the majority of studies have indicated an optimal cutoff score of 3, van Dam²⁴ found that, while a cutoff score of 3 provided moderate sensitivity (.67) and specificity (.72), a cutoff score of 2 was optimal for increasing sensitivity (.86) but at the expense of specificity (.57). Similarly, Prins et al.¹⁷ showed that, although the PC-PTSD had an optimally efficient cutoff score of 3, a cutoff score of 2 was recommended when optimizing sensitivity rather than efficiency. Lastly, Hanley and colleagues²⁷ did not compare cutoff thresholds but instead used a cutoff score of 3 or more in all of their analyses.

Notably, the majority of the literature has not considered potential sex and ethnic/racial differences in PC-PTSD diagnostic validity (e.g.,^{18,20,23–25,27}) This is particularly surprising given the evidenced differences in PTSD with regard to both sex and ethnic/racial identity. In fact, only two studies have investigated potential sex differences in PC-PTSD diagnostic validity^17,19 and only one study has investigated potential ethnic/racial differences in PC-PTSD diagnostic validity.¹⁹ Calhoun and colleagues¹⁹ plotted receiver operating characteristic (ROC) curve analyses for sex and race and found that the corresponding area under the curves (AUC) for the PC-PTSD did not differ as a function of either. Similarly, Prins et al.¹⁷ identified an optimally efficient cutoff score of 3 for both men and women. Notably, however, sensitivity and specificity were higher among males (.94 and .92, respectively) compared to females (.70 and .84, respectively), as was the optimal efficiency of the PC-PTSD (.92 vs. .81). Because the purpose of the PC-PTSD is to screen for PTSD, a psychiatric disorder associated with serious adverse outcomes, it is critical that the diagnostic validity be established across sex and ethnic/racial identities.

PC-PTSD item level predictive validity

Lastly, to the best of our knowledge, there is no existing research examining differences in the diagnostic validity across PTSD symptom clusters, as represented by each of the PC-PTSD items. However, this is an important, clinically relevant empirical question, as certain symptom clusters have been shown to be differentially predictive of overall PTSD symptomatology (e.g.,²⁹) For example, Harvey and Bryant³⁰ found that certain acute stress disorder (ASD) symptom clusters (i.e., a sense of reliving the trauma, acute numbing, depersonalization, and motor restlessness), in comparison to other symptoms, were more predictive of PTSD. Further, Brewin and colleagues²⁹ found evidence that similar predictive power could be attained by classifying individuals based only on the presence or absence of at least three reexperiencing or arousal symptoms. Thus, perhaps the overall diagnostic validity of the PC-PTSD is impacted by the diagnostic validity relative to each symptom cluster; further research is needed to assess potential predictive validity cutoffs among college-aged populations.

Study aims

Given trauma exposure and PTSD are high among college students,¹ it is important to validate quick and cost-effective methods of identifying trauma-exposed college students who are at increased risk for PTSD and, thus, a plethora of adverse academic, physical, and mental health outcomes. Thus, the aims of the current study were three-fold: (1) to provide a preliminary examination of the diagnostic validity of the PC-PTSD in a generalizable college sample; (2) to examine potential sex and racial differences in the diagnostic validity of the PC-PTSD; and (3) to examine the potential differential diagnostic validity of the PC-PTSD items (i.e., PTSD symptom clusters) in predicting subsequent PTSD.

Method

Recruitment

The present sample is derived from a large, ongoing cohort parent study of college students in an urban university started in 2011 Spit for Science [S4S]; for detailed methods see³¹ and a related spin-off study (Life Events and Alcohol Use [LEAU], P50AA022537, PI: Amstadter, F31AA025820, PI: Hawn; for detailed methods see Hawn et al., 2020³²). To be eligible for LEAU, participants needed to have endorsed at least one lifetime traumatic event during a prior S4S survey and reported any lifetime alcohol use on a prior S4S survey. The intention of the LEAU survey was to gather more indepth information about participant PTSD symptoms, trauma history, and trauma-related drinking to cope motives. The LEAU survey, which was administered online using REDCap electronic data capture tools,³³ took approximately 20 minutes to complete, after which participants were compensated $20 for their time. All study procedures were approved by the University’s Institutional Review Board (IRB).

Measures

Demographics–parent study (S4S)

Participant demographics (e.g., gender, race, ethnicity) were assessed as part of the S4S parent study.

Primary care PTSD screen (PC-PTSD)–parent study (S4S)

If a participant endorsed at least one item on the abbreviated version of the Life Events Checklist,³⁴ which asked participants to report on the occurrence of four different stressful events (i.e., natural disasters, physical assaults, sexual assaults, and transportation accidents), or at least one item from another instrument assessing additional stressful events (e.g., broken engagement, housing difficulties),³⁵ they were administered the Primary Care PTSD Screen (PC-PTSD).¹⁷ The PC-PTSD is a screening instrument with four “Yes/No” items, each representing one of the four PTSD symptom clusters, per the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV-TR).³⁶ PC-PTSD data assessed in the Spring of 2016 were used in the present analyses, given this was the most proximal time point to the external validating measure, the PTSD Symptom Checklist-5 (PCL-5),³⁷ which was assessed in Fall 2016. Four dichotomous variables were created to represent endorsement of at least one, two, three, and four items of the PC-PTSD, respectively.

PTSD checklist-5 (PCL-5)–spin-off study (LEAU)

Presence of PTSD symptoms in the past 30 days was assessed using the PTSD Checklist-5 (PCL-5)³⁷ in the Fall of 2016. The PCL-5 is a 20-item questionnaire, corresponding to the DSM-5 symptom criteria for PTSD. The self-report rating scale is 0–4 for each symptom, ranging from “Not at all” to “Extremely”. The PCL-5 has demonstrated good test-retest reliability (r = .82), and convergent (r’s = .74 to .85) and discriminant (r’s = .31 to .60) validity.³⁸ Cronbach’s alpha calculated from the LEAU sample suggested high internal consistency (.96). Because the newest version of the PC-PTSD (PC-PTSD-5)²⁰, which was created to reflect the new Diagnostic and Statistical Manual of Mental Disorders (DSM-5)³⁹ criteria for PTSD, was not yet available when the S4S study battery was compiled, data was collected using the original PC-PTSD, which is based off of the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR)⁴⁰ criteria for PTSD. Therefore, scores from the PCL-5 were recoded (0-4 Likert scale to 1–5) to reflect the DSM-IV version of the PCL-C.⁴¹ Notably, one item from the PCL-C was not included in the PCL-5 (“Item 12. Feeling as if your future will somehow be cut short”) and was therefore not included in the PCL-C composite score used in the present analyses. Following, using a PCL-C cutoff score of 50, which is consistent with previous work (e.g.,⁴²) a dichotomous threshold variable was created to represent individuals with greatest likelihood of meeting diagnostic criteria for PTSD.

Data analytic plan

Data checking

Relevant variables were examined for statistical assumptions and violations of normality before analysis. There were no violations in skewness or kurtosis. The data did not violate any assumptions of linear or logistic regression (i.e., homoscedasticity, linearity, multicollinearity, independence of errors).

Aim 1: Examine the diagnostic validity of the PC-PTSD in a generalizable college sample

Analyses were conducted in R Version 3.4.1.⁴³ A series of logistic regressions were conducted to examine the relationship between PC-PTSD at various endorsement thresholds and probable PTSD using the PCL diagnostic cutoff. Models were compared by taking the difference between their −2 log likelihoods (−2LLs) and using this as the test statistic in a Chi-square test with 1 df.

To establish the diagnostic properties of the screener, the following were analyzed for each of the PC-PTSD thresholds: (1) sensitivity: the chance of correctly screening a true positive; (2) specificity: the chance of correctly screening a true negative; (3) positive predictive value (PPV): the chance of having a positive diagnosis following a positive screen; (4) negative predictive value (NPV): chance of having a negative diagnosis following a negative screen; and (5) optimal efficiency (OE): chance of being classified appropriately. Receiver operating characteristic (ROC) analyses were also conducted considering different PC-PTSD thresholds weighing sensitivity versus specificity and were used to determine the area under the curve (AUC). AUC specifies the area under the receiver operating curve, mapping the screener’s true positive rate to its false positive rate. For instance, AUC values of 1.0 indicate a perfect fit, whereas values of .5 indicate that the screener is not more efficiently predictive than 50–50 chance.

Aim 2: Examine potential differences in the diagnostic validity of the PC-PTSD according to sex and racial/ethnic identity

In order to examine potential differences in the diagnostic validity of the PC-PTSD according to sex, analyses conducted for the purposes of Aim 1 were repeated in subsamples consisting of all females and all males. Similarly, analyses were repeated in subsamples consisting of all White participants, all Black American participants, and all “Other” participants.

Aim 3: Examine the potential differential diagnostic validity of the PC-PTSD items in predicting subsequent PTSD

To determine whether individual PC-PTSD items, each representing a PTSD symptom cluster, differentially predicted PTSD, a series of logistic regression, sensitivity/specificity, ROC, and AUC analyses were conducted once more among the total sample to compare the predictive validity of item endorsement for each of the four PC-PTSD items.

Results

Sample characteristics

Data was available from the measures and time points of interest for 475 LEAU participants. The sample was primarily female (75.79%) and, consistent with the greater LEAU and S4S samples, was generally representative of the overall university population from which it was drawn with regard to race (51.59% White, 19.75% Black/African American, 15.07% Asian American, 4.03% Hispanic/Latino, and 9.55% “Other” [.85% Pacific Islander, .42% American Indian/Native Alaskan, .42% unknown, 7.86% multiracial). The mean PC-PTSD score in the present sample was 1.36 (SD = 1.5). Over half (55.58%) of the sample endorsed at least 1 item on the PC-PTSD screen, 38.32% endorsed at least 2 items, 26.74% endorsed three or more items, and 15.27% endorsed all four items. The mean PCL score in the present sample was 31.04 (SD = 14.41) and 11.52% of the sample exceeded the PCL’s diagnostic cutoff for PTSD. This was relatively consistent among the analyzed subsamples (11.78% of females, 10.00% of males, 13.14% of White participants, and 11.45% of “Other” participants), with the exception of the subsample comprised of Black participants, among whom only 6.74% exceeded the PCL’s diagnostic cutoff for PTSD.

Aim 1 results: diagnostic validity of the PC-PTSD

A series of binary logistic regressions were conducted and compared to determine the predictive validity of the PC-PTSD at various thresholds onto PCL diagnostic cutoff. Results from the logistic regressions are presented in Table 1. Endorsement of at least one, two, three, and four items on the PC-PTSD were all positively significantly associated with PTSD diagnosis, per the PCL diagnostic cut off. Not surprisingly, endorsement of all four PC-PTSD items accounted for the highest total variance (6.7%) in PTSD compared to the other models and was associated with a 3.9 times increased likelihood of meeting PTSD diagnostic criteria. The two item threshold was a significant improvement compared to the one item threshold (Δ −2LL = 5.629***), but the three item threshold was not a significant improvement compared to the two item threshold. Notably, the same models were also run including sex and racial/ethnic identity as covariates; neither sex nor racial/ethnic identity were significantly associated with PTSD diagnosis in any of the four models, and the same pattern of results was demonstrated for each of the four PC-PTSD diagnostic cut offs.

Table 1.

Logistic regression results comparing PC-PTSD item thresholds.

Cutoff	B	SE B	OR	95% CI OR	χ2	Psuedo R2	−2LL	Δ−2LL
(> =1)	0.882	0.327	2.415	1.301 – 4.731	7.988**	0.034	320.715
(> =2)	1.092	0.302	2.981	1.664 – 5.464	13.617***	0.057	315.086	5.629***
(> =3)	1.097	0.298	3.000	1.666 – 5.387	13.129***	0.055	315.575	0.488
(> =4)	1.361	0.323	3.901	2.044 – 7.302	15.998***	0.067	312.706	2.869***

Note: “Cutoff” indicates PC-PTSD item threshold;

^***p < .001

^**p < .01

^*p < .05.

Table 2 summarizes the PC-PTSD psychometric properties at each item threshold in the total sample. To determine whether model performance could be improved with an alternative decision threshold, the predicted probabilities of PC-PTSD diagnostic cutoffs were subjected to ROC analyses. The ROC curves for each cutoff are presented in the top row of Figure 1. The two item PC-PTSD threshold represented the largest AUC (0.633).

Table 2.

Diagnostic efficiency of the PC-PTSD detecting PCL diagnostic cutoff: total sample.

Cutoff	Sensitivity	Specificity	PPV	NPV	OE
Full Sample
(> =1)	.736	.464	.152	.931	.496
(> =2)	.623	.644	.185	.929	.641
(> =3)	.491	.757	.208	.919	.726
(> =4)	.358	.875	.271	.913	.815

Note: “Cutoff” indicates PC-PTSD item threshold; PPV = positive predictive value (PPV); NPV = negative predictive value; OE = optimal efficiency.

Figure 1.

ROC curves for Aims 1 & 2.

Aim 2 results: sex and racial differences in the diagnostic validity of the PC-PTSD

Sex

First, in order to examine potential differences in the diagnostic validity of the PC-PTSD according to sex, analyses conducted for the purposes of Aim 1 were repeated in subsamples consisting of all females (N = 360) and all males (N = 115). Results from the logistic regressions are presented in Table 3. Each PC-PTSD item threshold was positively significantly associated with PTSD diagnosis, per the PCL diagnostic cut off, for both sexes.

Table 3.

Logistic regression results for female and male subsamples.

Cutoff	B	SE B	OR	95% CI OR	χ2	Psuedo R2	−2LL	Δ−2LL
Female Subsample
(> =1)	0.614	0.371	1.848	0.917 – 3.982	2.928	0.016	249.407
(> =2)	0.952	0.349	2.591	1.327 – 5.265	7.867**	0.043	244.468	4.939***
(> =3)	0.898	0.336	2.454	1.267 – 4.771	7.023**	0.039	245.312	−0.844
(> =4)	1.264	0.357	3.540	1.734 – 7.092	11.517***	0.063	240.819	4.493***
Male Subsample
(> =1)	1.584	0.709	4.876	1.318 – 23.353	5.700**	0.106	65.818
(> =2)	1.621	0.671	5.060	1.306 – 19.133	5.393*	0.100	66.125	−0.307
(> =3)	2.016	0.739	7.510	1.656 – 32.098	6.516**	0.120	65.002	1.123***
(> =4)	2.187	0.848	8.906	1.542 – 47.963	5.711*	0.106	65.807	−0.805

Note: “Cutoff” indicates PC-PTSD item threshold;

^***p < .001

^**p < .01

^*p < .05.

Interestingly, the pattern of results differed between the sexes. First, the pattern of results among the female subsample mirrored those found in the overall sample, such that the four item threshold accounted for the highest total variance (6.3%) in PTSD compared to the other models and the three item threshold was not a significant improvement compared to the two item threshold, which was a significant improvement compared to the one item threshold (Δ −2LL = 4.939***). Notably, however, an inverse pattern of results was seen among the all male subsample compared to the all female subsample and the overall sample. Among males, the three item threshold accounted for the highest total variance (12.0%) in PTSD compared to the other models, which is almost double that seen in the overall and all female samples. Additionally, the three item threshold was a significant improvement compared to the two item (Δ −2LL = 1.123***) threshold. Consistent with previous literature, the four item threshold did not significantly predict PTSD diagnosis more efficiently than the three item threshold among the all male sample.

Table 4 summarizes the psychometric properties in the female and male subsamples, respectively. The ROC curves for females and males are presented in the second row of Figure 1. For females, the four item PC-PTSD threshold represented the largest AUC (0.619). Males demonstrated an inverse pattern, such that the one item PC-PTSD threshold represented the largest AUC (0.687).

Table 4.

Diagnostic efficiency of the PC-PTSD detecting PCL diagnostic cutoff: sex differences.

Cutoff	Sensitivity	Specificity	PPV	NPV	OE
Female Subsample
1	.731	.404	.141	.919	.443
2	.659	.573	.171	.926	.583
3	.512	.700	.186	.915	.678
4	.390	.847	.254	.912	.891
Male Subsample
1	.727	.646	.186	.955	.655
2	.455	.859	.461	.934	.818
3	.364	.929	.364	.929	.873
4	.273	.960	.429	.922	.891

Note: “Cutoff” indicates PC-PTSD item threshold; PPV = positive predictive value (PPV); NPV=negative predictive value; OE = optimal efficiency.

Racial/ethnic identity

As with sex, a series of binary logistic regressions were conducted within separate ethnic/racial subsamples to determine potential differences in PC-PTSD diagnostic validity across racial groups. Examined racial subgroups consisted of all White participants (N = 243), all Black participants (N = 93), and all “Other” participants (N = 135). The Other subsample consisted of individuals who identified as American Indian/Native Alaskan (n = 2), Asian American (n = 71), Hispanic/Latino (n = 19), more than 1 racial identity (n = 37), Native/Pacific Islander (n = 4), and Unknown (n = 2). Results from the logistic regressions are presented in Table 5. Each PC-PTSD item threshold was positively significantly associated with PTSD diagnosis, per the PCL diagnostic cut off, for the White subsample only. None of the item thresholds were significantly associated with PTSD diagnosis in either the Black or the Other subsamples.

Table 5.

Logistic regression results for ethnic/racial subsamples.

Cutoff	B	SE B	OR	95% CI OR	χ2	Psuedo R2	−2LL	Δ−2LL
White Subsample
(> =1)	1.202	0.476	3.326	1.390–9.262	7.584**	0.058	176.003
(> =2)	1.627	0.436	5.090	2.250 – 12.666	16.117***	0.122	167.471	8.533***
(> =3)	1.299	0.395	3.667	1.693 – 8.071	10.733**	0.082	172.855	−5.384
(> =4)	1.519	0.417	4.565	1.992 – 10.333	12.307***	0.094	171.280	1.574***
Black Subsample
(> =1)	0.814	0.893	2.256	0.417 – 16.920	0.880	0.025	43.069
(> =2)	0.785	0.850	2.192	0.384 – 12.551	0.836	0.024	43.112	–0.044
(> =3)	1.214	0.857	3.368	0.583 – 19.543	1.923	0.055	42.026	1.087***
(> =4)	1.414	0.935	4.111	0.519 – 24.465	1.963	0.056	41.986	0.040***
Other Subsample
(> =1)	0.198	0.559	1.219	0.412 – 3.843	0.126	0.002	93.101
(> =2)	0.087	0.561	1.091	0.345 – 3.235	0.024	0.000	93.203	–0.102
(> =3)	0.406	0.588	1.500	0.438 – 4.598	0.499	0.007	92.769	0.434***
(> =4)	0.600	0.706	1.821	0.382 – 6.637	0.662	0.010	92.565	0.204***

Note: “Cutoff” indicates PC-PTSD item threshold;

^***p < .001

^**p < .01

^*p < .05.

Although the pattern of results among the White subsample somewhat mirrored those found in the overall and all female samples (i.e., significant improvements between the one and two (Δ −2LL = 8.533***) but not two and three item thresholds), unlike in the overall and all female samples, in which the four item threshold accounted for the highest total variance in PTSD, the two item threshold accounted for the highest total variance (12.20%) in PTSD in the White subsample compared to the other models and the four item threshold was not a significant improvement over the two item threshold (Δ −2LL = −3.810). This finding also differed from the all male subsample, which demonstrated a three item threshold accounted for the highest total variance.

Table 6 summarizes the psychometric properties in the White, Black, and Other subsamples, respectively. Once again, the predicted probabilities of PTSD diagnostic cutoffs were subjected to ROC analyses among each subsample separately. The ROC curves for each racial subgroup are presented in Figure 1. For White participants, the two item PC-PTSD threshold represented the largest AUC (0.691). The ROC pattern of results were consistent across Black and Other participants, such that the three item PC-PTSD threshold represented the largest AUC (0.639 & 0.542, respectively).

Table 6.

Diagnostic efficiency of the PC-PTSD detecting PCL diagnostic cutoff: ethnic/racial differences.

Cutoff	Sensitivity	Specificity	PPV	NPV	OE
White Subsample
1	.806	.444	.180	.938	.492
2	.742	.639	.237	.942	.653
3	.548	.751	.250	.917	.725
4	.419	.863	.317	.908	.805
Black Subsample
1	.667	.530	.093	.957	.539
2	.500	.687	.103	.950	.674
3	.500	.771	.136	.955	.753
4	.333	.892	.182	.949	.854
Other Subsample
1	.600	.448	.123	.897	.466
2	.400	.621	.120	.889	.595
3	.333	.750	.147	.897	.702
4	.200	.879	.176	.895	.802

Note: “Cutoff” indicates PC-PTSD item threshold; PPV = positive predictive value (PPV); NPV = negative predictive value; OE = optimal efficiency.

Aim 3 results: PC-PTSD items predicting PTSD

To determine whether individual PC-PTSD items, each representing a PTSD symptom cluster, differentially predicted PTSD, analyses conducted for the purposes of Aims 1 and 2 were conducted once more among the total sample comparing the predictive validity of item endorsement status of each of the four PC-PTSD items. First, a series of binary logistic regressions were conducted and compared to determine the predictive validity of each PC-PTSD item (endorsed vs. not endorsed) onto PCL diagnostic cutoff. Results from the logistic regressions are presented in Table 7. Endorsement of each of the four items was positively significantly associated with PTSD diagnosis, per the PCL diagnostic cut off. Endorsement of symptoms related to reexperiencing (item 1) accounted for the highest total variance (5.6%) in PTSD compared to the other models and was associated with a 2.9 times increased likelihood of meeting PTSD diagnostic criteria. Negative alteration in cognition and mood (item 3; Δ −2LL = 8.185*) and arousal (Δ −2LL = 5.364*) both evidenced a significant improvement compared to avoidance (item 2) in predicting PTSD. No other models significantly differed from one another.

Table 7.

Logistic regression results comparing PC-PTSD item endorsement.

Item	B	SE B	OR	95% CI OR	χ2	Psuedo R2	−2LL	Δ-2LL
1	1.078	0.302	2.938	1.640 – 5.385	13.258***	0.056	315.200
2	0.697	0.297	2.008	1.129 – 3.629	5.628*	0.024	322.584	*7.384
3	1.070	0.298	2.915	1.632 – 5.287	13.073***	0.055	314.399	8.185*
4	0.973	0.304	2.646	1.474 – 4.881	10.746**	0.045	317.221	−2.822

Note:

^***p < .001

^**p < .01

^*p < .05.

Table 8 summarizes the psychometric properties of each item endorsement in the total sample. ROC curves for each item endorsement are presented in Figure 2. Item 1, representing reexperiencing symptoms, demonstrated the largest AUC (0.632).

Table 8.

Diagnostic efficiency of PC-PTSD items in detecting PCL diagnostic cutoff: total sample.

Item	Sensitivity	Specificity	PPV	NPV	OE
1	.623	.640	.184	.929	.638
2	.585	.588	.157	.915	.587
3	.585	.674	.191	.925	.664
4	.642	.597	.173	.927	.602

Note: “Cutoff” indicates PC-PTSD item threshold; PPV = positive predictive value (PPV); NPV = negative predictive value; OE = optimal efficiency.

Figure 2.

ROC curves for Aim 3: PC-PTSD symptom cluster endorsement.

Discussion

The purpose of this study was to test the diagnostic validity of the PC-PTSD in a generalizable college sample, as well as to examine potential differences in the predictive efficacy of the PC-PTSD according to sex and racial/ethnic identity, filling three major gaps in the current literature. To the best of our knowledge, the diagnostic validity of the PC-PTSD has never been examined within a college sample and important sex and ethnic/racial differences in PC-PTSD diagnostic validity have been markedly understudied to date. An additional aim was to determine whether PC-PTSD symptom items differentially predicted PTSD diagnostic status. Broadly, results of this study indicated that the PC-PTSD identified PTSD among college students with poor accuracy. Furthermore, the PC-PTSD did not demonstrate equal predictive validity across neither sex nor racial/ethnic identity, suggesting the need for further refinement. Endorsement of reexperiencing symptoms appeared to be the strongest predictor of PTSD diagnostic status compared to the other three PC-PTSD items. Each of these findings is discussed in greater detail below.

The first aim of this study sought to examine the diagnostic validity of the PC-PTSD in a representative college sample. Findings indicated that a two item cutoff on the PC-PTSD represented the largest AUC, thereby providing the maximum test accuracy. Notably, the largest AUC represented in the overall sample (0.633) was considerably smaller than AUC values derived from other studies investigating the diagnostic validity of the PC-PTSD among other populations (e.g., 0.94²⁰). In addition to being much smaller than AUCs demonstrated in previous studies, the present findings were inconsistent with previous literature, which has generally found a three item cutoff to best maximize predictive validity.^{17,19,23,25,28}

These inconsistencies may be explained by sample differences. Most PC-PTSD research has been characterized by primarily older, white, predominantly male, Veteran samples.^17–20 The current research, however, extends the literature to include college students representative with regard to sex and racial/ethnic identity. Importantly, consistent with the low AUC values, the two item threshold found to maximize predictive validity in the present sample only contributed to moderate sensitivity (.62) and specificity (.64), while existing work has shown substantially higher scores for specificity and sensitivity when using cutoff scores of 3 (e.g. specificity of .85, sensitivity of .83).¹⁹ These discrepancies suggest that the PC-PTSD well serve adult, primary care populations but may not be the best screening tool for PTSD among college populations. Findings warrant further research to identify effective screeners for PTSD among college students.

The second aim of this study was to examine potential sex and ethnic/racial differences in the diagnostic validity of the PC-PTSD. To our knowledge, this study is the first of its kind to examine sex and ethnic/racial differences in PC-PTSD validity in a college student sample and findings highlight the importance of validating measures that are useful across sex, race, and ethnicity. With regard to sex, different cutoffs were observed for men and women in this sample. This is inconsistent with the two studies that have investigated sex differences in PC-PTSD diagnostic validity,^17,19 both of which found that optimal cutoff scores did not differ according to sex. For females in the present sample, a three item threshold was not a significant improvement over two items, consistent with results from the overall sample. While it is noted that the four item threshold demonstrated the highest AUC in females, sensitivity and specificity estimates were more equally balanced for the two item, as compared to the four item, threshold. This differs from findings presented in Prins and colleagues’¹⁷ original paper, in which they reported an optimally efficient cutoff score of three for both males and females. Among males, a three item threshold demonstrated significant improvement over a two item threshold in this sample, consistent with what has been demonstrated in the literature more broadly in a range of samples (e.g., Veterans,^17–20 substance use disorder patients^23–25) Study findings reflect the importance of examining sex differences, given they may influence overall results. Additionally, differences in predictive thresholds across men and women should be acknowledged in clinical settings.

Findings suggested that the PC-PTSD may not be effective at identifying risk for PTSD among people of color. Specifically, results indicated PC-PTSD item thresholds were significantly associated with PTSD diagnosis among White participants while none of the item thresholds were significantly associated with PTSD diagnosis in Black or Other participants. This notable discrepancy is contradictory to findings put forth by the one study that has examined ethnic/racial differences in the diagnostic efficiency of the PC-PTSD,¹⁹ which found that the AUC for the PC-PTSD did not differ as a function of race among a sample of Veterans. The inconsistency between the present findings and those presented by Calhoun and colleagues may be once more explained by sample differences (e.g., Veterans compared to college students). The poor predictive validity of the PC-PTSD among nonwhite individuals in the present sample poses a major issue, as failure to identify PTSD risk in people of color could lead to less treatment and longer chronicity. Further research and action are needed to identify a PTSD screener that can effectively identifies risk for PTSD among people of color. Overall, findings from Aim 2 reflect the importance of examining the diagnostic efficacy of a screener as a function of sociodemographic factors, such as sex and racial/ethnic identity. Indeed, after specifically examining both sex and ethnic/racial differences, it was determined that white females were mostly influencing results from the overall sample.

The final aim of this study was to examine the potential differential diagnostic validity of the PC-PTSD items in predicting subsequent PTSD. While all items (symptom clusters) were found to significantly predict PTSD independently, participants endorsing symptoms related to reexperiencing (item 1) were most likely to meet PTSD diagnostic criteria by 2.9 times and demonstrated the largest AUC. Both negative alteration in cognition and mood and arousal (items 3 and 4, respectively) were significantly better at predicting PTSD diagnosis than avoidance (item 2). To our knowledge, this study is the first of its kind to investigate differences in diagnostic validity across PC-PTSD items. Limited studies, however, have examined the differential predictive validity of ASD symptoms, in predicting PTSD, and existing literature on this topic is mixed. Current study findings are similar to a previous study examining the differential validity of ASD symptoms in predicting PTSD in a sample of violent crime victims.²⁹ These findings provide preliminary evidence that PTSD symptom clusters, as determined by the PC-PTSD items, may differentially predict PTSD; however, this research is exploratory and results from Aim 2 suggest that follow-up research should examine potential demographic differences in the predictive efficacy of PTSD symptom clusters.

Limitations

Results of this study, which sought to serve as a preliminary examination of the predictive validity of the PC-PTSD, a brief and easy to use PTSD screening measure, should be considered in the context of several limitations. First, it is noted that the predictive validity of the PC-PTSD in this college student sample was measured using the PCL, as compared to a clinical diagnosis or structured diagnostic interview. Previous studies have examined the predictive validity of the PC-PTSD using clinical interviews [e.g. Posttraumatic Diagnostic Scale;²⁴ computer-assisted structured diagnostic interview;²⁵ Clinician Administered PTSD Scale;¹⁷ MINI-International Neuropsychiatric Interview²⁰] in both veterans and civilian substance use disorder patients. Clinician administered interviews may capture diagnosis better than symptom-based self-report assessments and therefore may be better for examining predictive validity.⁴⁴ However, the PCL has shown a strong correlation (0.93) with the CAPS,⁴⁵ and has been used by previous studies to evaluate predictive validity of the PC-PTSD.^17,18,25,27

Second, while the present analyses used each individual’s most recent PC-PTSD assessment, there still exists a 6-month delay between the PC-PTSD assessment (collected in the Spring of 2016) and the PCL assessment (collected in the Fall of 2016). PTSD symptoms are dynamic in that they can improve and worsen over time.⁴⁶ The present study does not account for treatment being sought, new additional trauma exposures, or overall changes in symptoms. Because the PC-PTSD and PCL were not administered concurrently, there may be variability in PTSD symptom severity across the two data collection points. Therefore, the results of this paper are presented as a preliminary examination of the diagnostic validity of the PC-PTSD screen in college students to be followed up with more rigorous methodology.

Third, this study does not account for potentially confounding variables, such as additional psychiatric symptoms, despite known comorbidity among PTSD and various psychiatric outcomes (e.g., depression, substance use^47,48) These potential preexisting psychiatric symptoms could impact findings. Further, trauma load, timing, and type was not considered in the present analyses, although these factors are known to have differential effects on increased risk for PTSD in college students.⁴⁹ Further details on trauma and comorbid disorders can be beneficial when determining the diagnostic validity of the PC-PTSD among college students.

Fourth, the proportion of individuals exceeding the suggestive clinical cutoff for PTSD in the Black subsample (6.74%) was lower compared to proportions found in the total (11.52%) and other subsamples (10.00%–13.14%). This discrepancy could be contributing to the results demonstrating differences in predictive validity between White and Black participants. However, it should be noted that differences in predictive validity were still found between the White and “Other” subsamples, which showed similar rates of PTSD. Further research into the predictive validity of the PC-PTSD among college students of diverse racial and ethnic backgrounds is warranted.

Finally, it is noted that the PCL was adapted for the current study, as the PC-PTSD used in the present analyses reflected DSM-IV criteria for PTSD and the PCL-5 reflected DSM-5 criteria for PTSD. To mitigate this limitation, only DSM-IV items from the PCL-5 were included in the present study, and items were recoded to reflect the DSM-IV PCL. One item from the PCL-IV was not included in the newer PCL-5 (i.e. “Item 12. Feeling as if your future will somehow be cut short”), which was excluded from the PCL composite score in the present analysis.

Future directions

Given the increased vulnerability to trauma exposure and PTSD in college students, it is important to correctly and quickly screen and identify PTSD in college counseling centers. Due to cuts in education funding and increased prevalence of trauma and mental health conditions on college campuses, study findings reaffirm the clear need for a brief PTSD screen in college counseling centers and health centers. This is the first study of its kind to test the predictive validity of the PC-PTSD in a college student sample and it is noteworthy that results differ from previous studies involving other samples, thereby warranting further research aimed at identifying other effective PTSD screeners for the college population.

This study also fills a surprisingly understudied gap in the literature by incorporating sex and ethnic/racial considerations with regard to differential predictive validity. A great benefit of the PC-PTSD is its brevity and validity in some populations, warranting the usage of this measure on college campuses and in college counseling centers to quickly identify individuals in need of further follow-up. However, results, which indicated that the PC-PTSD is less effective when determining PTSD risk in female, Black, Hispanic, Asian American, and a mix of other ethnic/racial subgroups as compared to white individuals, highlight the clear need for a validated PTSD screener effective for, and generalizable to, a diverse college population. With growing knowledge and literature reporting that ethnic/racial minorities often experience greater degrees of polyvictimization (i.e. the sum of different types of traumatic events experienced)⁵⁰ than their white counterparts and women showing higher risk for PTSD than men,⁴ there is a need for validated measures that are specifically effective for women and ethnic minorities. Future research would greatly benefit from replication of this study’s findings with regards to racial/ethnic identity and sex in other samples (college and other), which should seek to either refine the PC-PTSD for varying populations or inform the development of a novel PTSD screener that effectively identifies risk for PTSD across multiple ethnic/racial groups and sex. Lastly, as stated in the limitations section, the current study does not consider comorbid psychiatric symptoms or trauma load, type, or count. As such, future research should account for comorbid psychiatric symptoms and additional trauma details.