Reliable and Clinically Significant Change in the Clinician-Administered PTSD Scale for DSM-5 and PTSD Checklist for DSM-5 Among Male Veterans

Abstract

We calculated the reliable change index (RCI) and clinically significant change (CSC) values for two widely used measures of posttraumatic stress disorder (PTSD): The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) and the PTSD Checklist for DSM-5 (PCL-5) and examined how symptom changes at these thresholds related to improvements in psychosocial functioning. We used data from three independent samples of male military veterans, including two randomized controlled trials for PTSD (N = 198 for Sample 1 and N = 102 for Sample 2) and a cross-sectional study of primary care patients (N = 228). For Sample 1, within-person change in CAPS-5 and PCL-5 scores of ≥13 and 15, respectively, was indicative of reliable change. For Sample 2, within-person change in CAPS-5 and PCL-5 scores of ≥12 and 18, respectively, was indicative of reliable change. Scores of ≤8 and 28 on the CAPS-5 and PCL-5, respectively, indicated a participant is more likely to belong to the non-PTSD population than the PTSD population (i.e., clinically significant change) in both Samples 1 and 2. Participants who exhibited reliable or CSC reported significantly better psychosocial functioning at all posttreatment assessments than those who did not. Results provide thresholds for identifying clinically meaningful PTSD symptom change using these measures. Care should be taken to interpret these values appropriately and relative to numerous other definitions for meaningful symptom change.

Although results from randomized controlled trials may demonstrate that an intervention reduces symptom severity, the inferential statistics used to demonstrate these group differences and their accompanying effect sizes provide no guidance for interpreting the magnitude of individual symptom change. Put simply, statistically significant results do not inform if symptom changes from pre- to posttreatment represent merely day-to-day symptom fluctuation or are clinically meaningful.

To address whether symptom change does indeed indicate reliable and clinically significant change, Jacobson et al. (1984) and Jacobson and Truax (1991) created the reliable change index (RCI) and clinically significant change (CSC) margin. The RCI is used to determine if the magnitude of observed change over time on a given measure is beyond what should be attributed to measurement error. The CSC is used to determine if an observed end score on a measure of symptomatology indicates that a respondent is more likely to belong to the nondisordered population than the disordered population.

In this article, we present results from our efforts to calculate the RCI and CSC for DSM, fifth edition (DSM-5; American Psychiatric Association, 2013) versions of two widely used posttraumatic stress disorder (PTSD) symptom measures: the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5; Weathers, Blake, et al., 2013) and the PTSD Checklist (PCL-5; Weathers, Litz, et al., 2013). We calculated these values using data from three large veteran samples, including two randomized controlled trials, and examined how symptom changes at these thresholds related to improvements in psychosocial functioning to benchmark the meaningfulness of these indicators of symptom change (Kazdin, 1999, 2001; Schnurr & Lunney, 2016).

Method

Participants and Procedure

We used data from three independent samples to calculate both values and replicate our findings. The first sample (Sample 1) was from a randomized controlled trial of two group interventions for PTSD (N = 198; Sloan et al., 2018). The second sample (Sample 2) was from a randomized controlled trial of two interventions for comorbid PTSD and alcohol use disorder (N = 119; Norman et al., 2019). The third sample (Sample 3) was from a cross-sectional study of primary care patients (N = 495; Bovin et al., 2021); we included this sample to calculate the CSC margin using CAPS-5 and PCL-5 values from a large sample of veterans who do not meet diagnostic criteria for PTSD. Written informed consent was obtained prior to participation for all three studies. Procedures were approved by the local institutional review boards. Studies including Samples 1 and 2 were registered on www.ClinicalTrials.gov (NCT01544088 and NCT01601067) prior to data collection. For access to data and study materials, please contact the first author.

We excluded participants from Study 2 who did not meet diagnostic criteria (n = 5) and participants from Study 3 who met subthreshold criteria (n = 241), defined as meeting DSM-5 Criterion A and all but one of Criteria B-D (McLaughlin et al., 2015), from analyses (see Data Analytic Strategy section for justification). Additionally, given that so few participants identified as female (0 participants in Sample 1, 12 participants [10.53%] in Sample 2, and 26 participants in Sample 3 [10.24%]; 38 of the 566 participants [6.71%] in this study), we excluded individuals who identified as female from our analyses. Table 1 displays demographic characteristics of all participants retained for analyses. All participants were United States military veterans.

Table 1.

Demographic Characteristics

Demographic	Sample 1		Sample 2		Sample 3
	N = 198		N = 114		N = 228
	M/n	SD/%	M/n	SD/%	M/n	SD/%
Age	55.82	12.05	41.22	12.20	65.07	14.65
Male	198	100.00%	102	89.47	224	88.19%
Race
White	147	74.24%	74	64.91%	204a	80.31%
African American	33	16.67%	16	14.04%	29a	11.42%
American Indian	3	1.52%	2	1.75%	6a	2.36%
Other	15	7.58%	21	18.42%	27a	10.63%
Did not respond	0	0.00%	1	0.88%	1a	0.39%
Ethnicity
Hispanic	13	6.57%	34	29.82%	11	4.33%
Non-Hispanic	180	90.91%	79	69.30%	209	82.28%
Did not respond	5	2.53%	1	0.88%	34	13.39%
Education
<HS graduate	19	9.60%	—	—	5	1.97%
HS graduate	29	14.65%	10	8.77%	33	12.99%
Some college/VT	103	52.02%	65	57.02%	77	30.31%
≥Graduated college	40	20.20%	33	28.95%	59	23.23%
Did not respond	0	0.00%	6	5.26%	80	31.50%
Household income
<$15,000	45	22.73%	33	28.95%	b	—
$15,000–$50,000 K	86	43.43%	74	64.91%	b	—
$50,001–$75,000 K	37	18.69%	5	4.39%	b	—
>$75,000	24	12.12%	2	1.75%	b	—
Did not report	6	3.03%	6	5.26%	b	—
Employment
Full time	59	29.80%	29	25.44%	80a	31.50%
Part time	26	13.13%	7	6.14%	b
Student	3	1.52%	12	10.53%	10a	3.94%
Retired/disability	80	40.40%	51	44.74%	176a	69.29%
Unemployed	22	11.11%	13	11.40%	4a	1.57%
Other	0	0.00%	2	1.75%	1a	0.39%
Marital status
Married/remarried	98	49.49%	31	27.19%	117a	46.06%
Widowed	6	3.03%	3	2.63%	18a	7.09%
Cohabitating	9	4.55%	—	—	6a	2.36%
Separated/divorced	63	31.82%	57	50.00%	69a	27.17%
Single	22	11.11%	23	20.18%	47a	18.50%

^aRespondents able to identify more than one category, numbers are not cumulative.

^bInformation not collected.

Measures

All three studies included the CAPS-5 administered at the first interview assessment session. The CAPS-5 is a 30-item structured interview used to determine PTSD diagnostic status and symptom severity during the past month. Internal consistency of total scores was adequate in all three studies and interrater reliability was high in all three studies (see Supplemental Material, for a detailed description of CAPS-5 rating, scoring, internal consistency, and interrater reliability). We examined changes in CAPS-5 scores from baseline to 12-month posttreatment in Sample 1 and from baseline to 6-month posttreatment in Sample 2.

All three studies administered the PCL-5, a 20-item self-rating scale that assesses the 20 DSM-5 Criteria B-E symptoms of PTSD. Respondents rate the degree to which each symptom bothered them during the past month on a 5-item scale ranging from not at all to extremely. Study 1 administered the PCL-5 on the same days as the CAPS-5; Study 2 administered the PCL-5 prior to each therapy session. In Study 3, the CAPS and PCL were administered on different days up to 30 days apart (mean = 11.6 days, SD = 7.1, range = 1–30; see Supplemental Material, for a detailed description of PCL-5 scoring and internal consistency). We examined changes in PCL-5 scores from baseline to 12-month posttreatment in Sample 1 and baseline to 6-month posttreatment in Sample 2. Because Sample 2 only administered the PCL-5 at treatment sessions and not during follow-up, we only used baseline PCL- 5 scores to replicate RCI and CSC estimates from Sample 1 but did not explore reliable or clinically significant change in PCL-5 scores over time.

Studies 1 and 2 administered the 36-Item Short-Form Health Survey Questionnaire (SF-36; Ware & Sherbourne, 1992). The SF-36 is a brief self-rating measure of psychosocial functioning. To provide construct validity evidence for RCI and CSC values, we used the role limitations due to physical health, role limitations due to emotional problems, and social functioning SF-36 subscales. Higher scores indicate greater psychosocial functioning. Respondents rate the degree to which physical health or emotional problems have interfered with social activities during the past month on a 5-point scale that varies in response options by item. Strong evidence of internal consistency, test–retest reliability, and construct validity has been established among multiple samples (see Ware et al., 1993, for a summary). We examined how individuals who exhibited reliable or clinically significant change improved in psychosocial functioning; we did this by comparing changes in SF-36 scale scores from baseline to 12-month posttreatment in Sample 1 and from baseline to 6-month posttreatment in Sample 2 between groups. Internal consistency for all three subscales was good in both studies: α ≥ .81 at all assessment points in Sample 1 and α ≥ .82 at all assessment points in Sample 2.

Data Analytic Strategy

We calculated the RCI and CSC values per guidance from Jacobson and Truax (1991; see Supplemental Materials, for the RCI formula). For CSC, Jacobson & Truax proposed three possible operationalizations. They cautioned against using the first definition (a), defined as a score falling ≥2 SDs below the mean score in the psychopathology group, when a control sample is available. Instead, when distributions for psychopathology and control groups do not overlap, they recommended the second definition (b), defined as a score falling within 2 SDs of the control group. Finally, when distributions do overlap, they recommended the third definition (c), defined as the midway point between the mean scores of the psychopathology and control groups.

We examined the association between novel change values and changes in functioning to explore construct validity of the novel change margins. We used between-samples t-tests to compare mean SF-36 scores in groups who did or did not exhibit reliable or clinically significant symptom change and used Cohen’s d (Cohen, 1988) to quantify between-group differences in SF-36 scores. We used standardized mean gain scores (ESsg; Lipsey & Wilson, 2001) to quantify change in functioning over time in groups who did or did not exhibit reliable or clinically significant symptom change.

We provide details of missing data in Supplementary Material. We handled missing data using pairwise deletion because it is only possible to identify participants who exhibited reliable or clinically significant change for participants with pretreatment and posttreatment data and limitations of using more advanced techniques in relatively small samples. We conducted all analyses using SPSS version 26.

Results

Reliable Change

Table 2 displays RCI results. We calculated the RCI separately for the CAPS-5 and PCL-5 using respective baseline standard deviation values in Sample 1—a sample in which all participants met diagnostic criteria for PTSD based on CAPS-5 results—and published test–retest reliability coefficients (Bovin et al., 2016; Weathers et al., 2018; see Table 1). Results suggest that within-person change in CAPS-5 and PCL-5 scores of ≥13 and 15, respectively, is indicative of change in PTSD symptom severity beyond what is attributable to measurement error (i.e., reliable change). We then replicated these analyses using the baseline standard deviation of both measures from participants in Sample 2—a sample in which all participants met diagnostic criteria for PTSD based on CAPS-5 results—and the same test–retest reliability coefficients. Results suggest that within-person change in CAPS-5 and PCL-5 scores of ≥12 and 18, respectively, is indicative of reliable change.

Table 2.

Reliable Change Values for the CAPS-5 and PCL-5

Measure/Sample	SD	Test–retest r	S E	S diff	RC > 1.96
CAPS-5
Sample 1	9.63	.78a	4.52	6.39	13
Sample 2	9.07	.78a	4.25	6.02	12
PCL-5
Sample 1	12.68	.84a	5.07	7.17	15
Sample 2	16.12	.84a	6.45	9.12	18

Note. RC = reliable change; S_diff = standard error of difference; SE = standard error of measurement.

^aValues obtained from published results cited in text, not from these studies.

Clinically Significant Change

Next, we calculated CSC definitions b (symptoms posttreatment fall within 2 SD of the non-PTSD population) and c (symptoms posttreatment fall closer to the non-PTSD population than the PTSD population) separately for the CAPS-5 and PCL-5 by comparing the baseline mean and standard deviation in Sample 1 with mean and standard deviation values in Sample 3 participants who did not meet full or subthreshold diagnostic criteria. CAPS-5 distributions for the PTSD and No PTSD groups did not overlap in either sample, indicating that we should use definition b to operationalize the CAPS-5 CSC; the PCL-5 distributions overlapped in both samples, indicating that we should use definition c to operationalize the PCL-5 CSC. Accordingly, male veterans with CAPS-5 and PCL-5 scores of ≤8 and 28, respectively, were more likely to belong to the non-PTSD population than the PTSD population (see Table 3). We then replicated these analyses, comparing Sample 2 with Sample 3. Results were identical for both measures.

Table 3.

Clinically Significant Change Values for the CAPS-5 and PCL-5

Measure/sample	PTSD M	PTSD SD	Non-PTSD M	Non-PTSD SD	CSC
CAPS-5
Sample 1	39.60	9.63	2.02	3.05	8
Sample 2	42.73	9.11	2.02	3.05	8
PCL-5
Sample 1	48.36	12.68	9.15	12.03	28
Sample 2	47.94	15.86	9.15	12.03	28

Note. CSC = clinically significant change.

Reliable and Clinically Significant Change and Psychosocial Functioning

To benchmark these RCI and CSC values, we compared change in psychosocial functioning between participants in Samples 1 and 2 who did and did not exhibit reliable change and those who did and did not exhibit clinically significant change. Using the more conservative 13-point RCI and ≤8 end score CSC margin for the CAPS-5, in Sample 1, 49 participants (34.03% of participants with pre- and posttreatment CAPS-5 data) exhibited reliable symptom reduction and 12 (6.06% of participants with pre- and posttreatment CAPS-5 data) exhibited clinically significant symptom reduction (see Table 4). In Sample 2, 33 participants (54.10% of participants with pre- and posttreatment CAPS-5 data) exhibited reliable symptom reduction and 13 (11.40% of participants with pre- and posttreatment CAPS-5 data) exhibited clinically significant symptom reduction. Using the more conservative 18-point RCI and ≤ 28 end score CSC margin for the PCL-5, in Sample 1, 36 participants (24.83% of participants with pre- and posttreatment PCL-5 data) exhibited reliable symptom reduction and 38 (26.21% of participants with pre- and posttreatment PCL-5 data) exhibited clinically significant symptom reduction.

Table 4.

SF-36 Psychosocial Functioning Scores by CAPS-5 Reliable and Clinically Significant PTSD Symptom Change

		Role limitations physical					Role limitations emotional					Social functioning
		Baseline		12 months			Baseline		12 months			Baseline		12 months
Sample/metric	n	Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg
Sample 1
13-pt RCI	49	57.99	25.27	62.23	27.27	0.20	53.23	27.63	66.84	26.89	0.50	43.37	26.28	64.10	26.40	0.74
No 13-pt RCI	95	54.23	29.58	53.71	29.75	0.11	51.75	25.20	54.17	28.77	0.14	43.16	27.58	45.92	27.05	0.14
Between-group d		0.14		0.30			0.06		0.46*			0.01		0.68*
CSC (end ≤ 8)	12	71.70	26.48	77.60	23.45	0.27	64.58	31.61	83.33	25.38	0.86	60.42	35.69	85.42	20.53	0.65
No CSC (end > 8)	132	54.04	27.93	54.61	28.89	0.09	51.14	25.24	56.10	27.94	0.21	41.67	25.74	48.92	26.68	0.29
Between-group d		0.65*		0.88*			0.47		1.02*			0.61*		1.55*
		Baseline		6 months			Baseline		6 months			Baseline		6 months
Sample/metric	n	Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg
Sample 2
13-pt RCI	33	57.41	42.64	68.00	41.16	0.72	35.80	42.29	62.67	44.43	1.04	44.91	24.58	70.00	30.19	1.64
No 13-pt RCI	28	40.74	37.44	31.48	42.51	0.25	8.64	14.89	19.75	32.37	0.24	33.33	19.61	33.33	20.51	0.24
Between-group d		0.42		0.87			0.95		1.12			0.52		1.45
CSC (end ≤ 8)	13	57.50	42.57	85.00	26.87	1.99	56.67	48.81	86.67	32.20	1.10	53.75	29.49	87.50	18.63	4.15
No CSC (end > 8)	48	47.16	40.43	40.48	44.84	0.22	14.39	24.27	29.37	39.10	0.80	35.80	19.92	42.26	27.32	0.41
Between-group d		0.25		1.24*			1.16*		1.61*			0.73		1.97 *

Note. Sample sizes reflect participants identified as exhibiting reliable or clinically significant change, see Supplemental Material for a description of missing data; SF-36 = 36-Item Short-Form Health Survey Questionnaire; CAPS-5 = Clinician-Administered PTSD Scale for DSM-5; CSC = clinically significant change margin; RCI = reliable change index; ESsg = standardized mean gain score quantifying within-group change over time.

^*p < .05.

Participants who exhibited reliable or clinically significant change on the CAPS-5 reported better functioning at all posttreatment assessments in role limitations due to physical, emotional, and social functioning with the single exception of physical functioning among those who exhibited reliable change; these results were consistent across samples (see Table 4). Between-group effect sizes varied by functioning domain and sample but were the largest for social functioning (range = 0.68–1.97). In addition to posttreatment differences, participants who experienced clinically significant change reported higher social functioning at baseline in Sample 1 and higher emotional functioning in Sample 2. Within-group effect sizes indicated that participants who exhibited reliable or clinically significant change experienced medium-to-large-magnitude improvements in emotional and social functioning whereas participants who did not exhibit reliable or clinically significant symptom change did not. Within-group effect sizes were small in Sample 1 but large in Sample 2.

Participants who exhibited reliable or clinically significant change on the PCL-5, only examined in Sample 1, reported better functioning at all posttreatment assessments in role limitations due to physical, emotional, and social functioning with the single exception of physical functioning among those who exhibited reliable change (see Table 5). Between-group effect sizes indicated medium-to-large differences for emotional and social functioning and small-to-medium magnitude differences for physical functioning. In addition to posttreatment differences, participants who experienced clinically significant change reported higher baseline social functioning. Within-group effect sizes indicated that participants who exhibited reliable or clinically significant change experienced large-magnitude improvements in role limitations due to emotional functioning and social functioning whereas participants who did not exhibit reliable or clinically significant symptom change did not. Within-group effect sizes for physical functioning indicated small-magnitude improvement.

Table 5.

SF-36 Psychosocial Functioning Scores by PCL-5 Reliable and Clinically Significant PTSD Symptom Change in Sample 1

Metric	n	Role limitations physical					Role limitations emotional					Social functioning
		Baseline		12 months			Baseline		12 months			Baseline		12 months
		Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg	Mean	SD	Mean	SD	ESsg
18-pt RCI	36	55.50	29.05	63.83	29.71	0.28	47.22	28.03	71.30	27.49	0.87	40.28	29.75	65.97	27.16	1.03
No 18-pt RCI	109	54.89	27.64	53.30	28.62	0.11	54.36	25.22	54.01	28.08	0.10	44.38	26.21	46.99	26.69	0.14
Between-group d		0.02		0.36			0.27		0.62*			0.15		0.70*
CSC (end ≤ 28)	38	61.29	25.40	69.74	26.81	0.33	59.43	27.41	79.39	23.63	0.87	51.64	30.09	70.72	26.36	0.79
No CSC (end > 28)	107	52.28	28.52	50.98	28.46	0.11	50.16	25.21	50.79	26.79	0.11	40.42	25.44	44.93	25.34	0.21
Between-group d		0.33		0.68*			0.35		1.13*			0.40*		1.00*

Note. SF-36 = 36-Item Short-Form Health Survey Questionnaire; PCL-5 = PTSD checklist for DSM-5; CSC = clinically significant change margin; RCI = reliable change index; ESsg = standardized mean gain score quantifying within-group change over time.

^*p < .05.

Discussion

This study calculated and validated RCI and CSC values for the CAPS-5 and the PCL-5, two widely used measures of DSM-5 PTSD, using multiple veteran samples. These RCI and CSC values offer important information about what constitutes meaningful PTSD symptom change when using these measures. Both values, used independently and in conjunction with other indices of meaningful change (e.g., loss of diagnosis, changes in functioning and quality of life; see Schnurr & Lunney, 2016), can provide valuable information about clinically meaningful symptom change.

These RCI values varied slightly across samples; RCI ranged from 12 to 13 for the CAPS-5 and 15 to 18 for the PCL-5. The CSC was consistent for the CAPS-5 at 8 and the PCL-5 at 28 in both samples. As with all other point estimates, these values are likely to vary by sample. In general, ranges in RCI and CSC values may suggest conservative and liberal thresholds that may be used accordingly. In other words, our results indicate that there is a likely a range of values for RCI and CSC for these measures, rather than a singular value, depending on the samples from which they are derived. Clinicians and researchers working with treatment-seeking male veterans with PTSD should use their best judgment when selecting values for these change indicators.

Previously, Kazdin (1999, 2001) described challenges associated with interpreting current indicators of clinical significance (such as the RCI and CSC), the reliance on symptom reduction as the sole or primary criterion of clinically meaningful change, and the need to match the criteria and measures of clinical significance to the clinical problems, treatment goals, and lives of the clients. These are all important considerations that deserve future research attention. We responded to Kazdin’s concerns here by benchmarking our derived CAPS-5 and PCL-5 RCI and CSC values against changes in psychosocial functioning. Doing so builds upon previous benchmarking efforts (Schnurr & Lunney, 2016) and conceptualizes change in terms of real-world impact of PTSD treatment for patients.

Of the domains of functioning we examined, observed differences between participants who did and did not experience reliable or clinically significant treatment gains were most pronounced for emotional and social functioning. This is not surprising given the focus of the interventions studied here on emotional and social functioning. That said, we did observe significant differences in physical functioning in participants who experienced clinically significant gains; these effects were the largest in Sample 2, indicating the physical functioning benefits of clinically significant improvement may be most relevant in those with a comorbid alcohol use disorder.

In multiple instances, baseline psychosocial functioning was significantly worse in participants that failed to demonstrate an RCI or CSC. In other words, those participants who were functioning better before treatment were more likely to experience meaningful PTSD symptom improvements from treatment. More work is needed to explore this prospect and how worse baseline functioning might influence future benchmarking efforts and explore if individuals with greater impairment may require different care.

It is important to interpret these values in the context of several limitations and nature of the intended function of the values. First, the test–retest coefficients used to calculate these values were derived from relatively small samples (n = 60 for the CAPS-5, n = 99 for the PCL-5). Similarly, Samples 1 and 2 are treatment-seeking samples, all of which were male. These values are point estimates and may vary as a function of the test–retest coefficient used and variability in the sample. They should be replicated in other large, more diverse trauma-exposed samples to increase confidence in their stability and generalizability. Second, care should be taken to interpret these values appropriately. Jacobson and Truax (1991) interpreted participants who exhibited reliable change as having improved and participants who exhibited clinically significant change as having recovered. Although it is appropriate to interpret these thresholds as reliable and clinically significant change, we caution against interpreting these values as indicating recovery. As detailed in Schnurr & Lunney (2016), numerous definitions have been proposed for recovery from the disorder (e.g., improvement in psychosocial functioning in addition to symptoms) that are important to consider. Finally, the samples we used to derive RCI and CSC values were convenience samples. Therefore, it is important not to reify these estimates, as they are guidelines and not representative of every sample for whom estimates of change are beneficial.

Results from this study provide thresholds for identifying meaningful PTSD symptom change when using the CAPS-5 and the PCL-5. Intraindividual change, and whether it is greater than would be expected by the precision of a measure, is a crucial consideration. These values offer a meaningful addition to conclusions that can be reached by reliance solely on group changes in average CAPS-5 and PCL-5 scores.

Public Significance Statement.

We calculated two values for determining if change in posttraumatic stress disorder (PTSD) symptoms were clinically meaningful using two common measures among male veterans. Results indicate these margins were associated with marked improvement in psychosocial functioning.