Measuring Use of Evidence-Based Psychotherapy for PTSD in VA Residential Treatment Settings with Clinician Survey and Electronic Medical Record Templates

Abstract

Objective

Available studies on implementation of evidence-based psychotherapy (EBP) for patients attending Department of Veterans Affairs (VA) residential post-traumatic stress disorder (PTSD) programs rely on therapist self-report of EBP delivery. Patient-level data on receipt of EBP are needed both to corroborate therapist self-report and to understand patient factors that predict receipt of EBPs for PTSD.

Materials and Methods

We identified 159 therapists from 38 VA residential PTSD programs who responded to a survey about EBP implementation during the 2015 fiscal year (FY15). Therapists self-reported their use of two EBPs, including prolonged exposure delivered in an individual format (PE-I) and cognitive processing therapy delivered in individual and group formats (CPT-I and CPT-G). Using electronic medical record (EMR) templates mandated for EBP documentation in FY15, we measured contemporaneous patient-level receipt of EBPs for PTSD. We assessed the degree of correlation between therapist self-reported EBP delivery and patient receipt of EBT as measured by EMR templates using polychoric correlation coefficients. We determined patient and therapist factors that predicted the receipt of EBPs with multivariable logistic regression, using random effects and robust standard error estimation, and controlling for site. The Veterans IRB of Northern New England provided a waiver of informed consent; as this was a retrospective review, no patients or therapists were contacted, and all data were stored, transmitted, and analyzed on secure VA servers. The VA Connecticut Health Care System Human Research Protection Program approved secondary use of therapist survey data for this project.

Results

When EMR template use became mandated in FY15, the proportion of patients in residential PTSD programs who received at least one EBP session that was recorded with an EMR template increased dramatically from 8.8% to 33.9%. There was adequate correlation and between survey-based and EMR-based measures of EBP receipt, with polychoric correlation values of 0.77 for PE-I, 0.69 for CPT-I, and 0.82 for CPT-G. Multiple patient factors were positive (e.g., female gender) and negative (e.g., depressive disorders) predictors of receipt of EBPs, even after controlling for site. Among therapist factors, only EBP consultant or trainer status was a positive predictor of EBP provision and only therapist race was a negative predictor of EBT provision after controlling for site.

Conclusion

Following a FY15 mandate, EMR templates documenting EBP delivery were widely used by therapists working in VA residential PTSD programs. EBP receipt measured using EMR templates was consistent with therapist self-report of EBT delivery. There were several patient-level predictors of EBP receipt and therapist-level predictors of EBP delivery. However, therapists most likely to deliver EBPs were clustered at a limited number of sites.

Introduction

Individual Department of Veterans Affairs (VA) hospitals began to develop specialized inpatient hospital wards for treatment of war-related psychological distress in the late 1970s.¹ Over time, these inpatient wards have been refined into residential post-traumatic stress disorder (PTSD) treatment programs, organized, and spread throughout the United States to assist those whose need for treatment exceeds what can be provided in the outpatient setting.² Patients admitted to residential PTSD programs often have more severe symptoms and more mental health comorbidities than those treated in outpatient settings.³

Over the past decade,⁴ VA has invested substantial resources in the national implementation of two evidence-based psychotherapies (EBPs) for PTSD, prolonged exposure (PE)⁵ and cognitive processing therapy (CPT).⁶ Reports from individual VA residential PTSD treatment programs have shown that PE and CPT can be delivered effectively to the high-need patients who receive care in those settings.^7–12 Early in the national implementation efforts, therapists working in PTSD residential settings reported low rates of EBP delivery even after attending VA-sponsored training.¹³ Relatively few programs reported using PE with any of their patients, and CPT adoption occurred on a discernable continuum ranging from no adoption, to use of only one aspect (e.g., specific worksheets), to strict manual adherence with all patients. Although many therapists expressed belief in the efficacy of EBPs during training, they also attributed perceived efficacy of the residential programs to other elements, such as the residential milieu, staff cohesion, varied programming, and individualized treatment.¹⁴

Although additional programs adopted PE or CPT over time, some de-adopted or limited their use of PE and CPT to select patients, and many sites adapted EBPs for their local settings.¹⁵ Factors such as organizational context (e.g., leadership, dedicated time and resources, and compatibility with existing practices) and positive view of the treatment (e.g., ability to notice patient improvements and perceived relative advantage over alternative approaches) were associated with higher implementation.^16,17 One limitation of available findings is a reliance on therapist self-report of EBP use and lack of a measure of patient-level receipt of the treatments. Thus, there is a need to corroborate therapist reports using an alternative data source in order to ensure the validity of the purported use of EBPs and to understand the patient-level factors that impact their implementation.

In the summer of 2013, the VA launched electronic medical record (EMR) templates to record EBP sessions,¹⁸ which VA therapists have been required to use since the 2015 fiscal year (FY15). Therapists record PE and CPT patient encounters using a combination of free-text fields and check boxes that indicate which treatment was delivered. Data on use of PE and CPT resulting from EMR template use are tracked on a VA intranet site, where national, regional, facility, and therapist-level reports can be generated. For example, in the second quarter of FY17, the site reports that 3.2% (26,614) of 824,476 VA patients with a PTSD diagnosis received at least one session of PE or CPT.¹⁹ However, the VA intranet site does not generate reports regarding specific program types, such as residential PTSD treatment programs. Rates of PE and CPT use in VA residential PTSD treatment programs as measured using EMR templates have not yet been reported in the literature.

Overall, reliance on therapist self-report has resulted in several gaps in our understanding of the implementation of PE and CPT in residential PTSD treatment programs, including how patient factors influence EBP implementation. Our goal was to narrow these gaps by leveraging EMR template data to provide a more complete understanding of PE and CPT use. We had three objectives to determine: (1) The overall use of PE and CPT in VA residential PTSD treatment programs as recorded in the EMR templates; (2) the relationship between therapist-reported use of PE and CPT via yearly survey and EMR template-measured use of PE and CPT; and (3) patient and therapist factors associated with use of PE and CPT, accounting for residential treatment site.

Method

Data Sources

We had two sources of data. First, we used therapist survey data from a large mixed-method longitudinal study examining the implementation of PE and CPT in 38 VA residential treatment programs across the United States.¹⁴ We used the most recent therapist survey data, which was collected in FY15. Second, we used the VA corporate data warehouse (CDW) to identify patients who were admitted to residential PTSD programs in FY14, FY15, and FY16. We obtained patient and therapist and demographic information as well as encounter, diagnostic, and template use data from the CDW. Appendix 1 describes our approach to codification of study variables using CDW data.

The Veterans IRB of Northern New England provided a waiver of informed consent; as this was a retrospective review, no patients or therapists were contacted, and all data were stored, transmitted, and analyzed on secure VA servers. The VA Connecticut Health Care System Human Research Protection Program approved secondary use of therapist survey data for this project.

Study Population

We developed our cohort in two ways. First, our overall cohort, used to understand the implementation of EBP templates over time, consisted of all patients who were admitted to residential PTSD programs during FY14, FY15, and FY16 and the therapists who treated them. Second, our analytic cohort, used to understand EBP implementation factors, consisted of therapists who participated in FY15 therapist survey and the patients that they treated.

Therapist Self-report of EBP Provision

We measured therapist-reported EBP provision using a series of three self-report items assessing two treatment modalities (PE and CPT) and two delivery formats (Individual and Group treatment) as follows: (1) use of PE administered individually (PE-I), (2) use of CPT administered individually (CPT-I), and (3) use of CPT administered on a group basis (CPT-G). Each item assessed the proportion of patients to whom the treatment was administered. For example, the item assessing PE read, “How often do you conduct PE on an individual basis?” and response options were on a 6-point scale, ranging from “less than 10%” to “over 90%” of clients. We categorized use as frequent (50%–100% of patients), occasional (10%–49% of patients), and rare (less than 10% of patients). These trichotomized single-item measures correlate with attitudes toward treatments,¹⁷ with prior intentions to use treatments, and with prior reported use of treatments.¹⁵

EMR Template Documentation of EBP Receipt

We identified psychotherapy encounters using current procedural technology codes for individual and group psychotherapy. For each encounter, we determined whether therapists completed EMR templates indicating EBP delivery. When therapists check boxes on the EMR templates, they create string variables called “health factors” that are linked to the encounter. Using health factor data, we divided EBP encounters into PE-I, CPT-I, and CPT-G.

Covariates

Two groups of covariates were included in these analyses. The first group consisted of therapist factors. Using the CDW, we were able to determine a limited number of therapist characteristics including gender, profession, most advanced degree, prescription privileges, and years of government service. From the therapist survey, we were able to determine gender, age, race, profession, most advanced degree, clinical experience, level of EBP training, and level of EBP use. The second group of covariates consisted of patient factors. Using the CDW, we were able to determine age, gender, ethnicity, race, fiscal year and quarter of admission, length of stay, and comorbid mental health diagnoses.

Analysis

This analysis consisted of five steps. First, we determined the percentage of all patients admitted to VA residential PTSD programs in each quarter of FY14–16 who received at least one session of each EBP for PTSD, as indicated by therapist template completion. Second, we categorized site-level use, as measured using EMR templates, using the frequent–occasional–rare rubric described above. Third, we compared characteristics of therapists who responded to the FY15 survey versus those who did not and characteristics of patients who saw therapist–respondents versus those who did not, using t-tests or Χ² analysis, as appropriate. Fourth, we compared therapists’ use of PE-I, CPT-I, and CPT-G as recalled by survey and as measured by template use during FY15. We compared therapist-reported EBP use and measured percentage of patients who received each EBP by examining dot plots and calculating polychoric correlations. Fifth, we developed a series of multivariate logistic regression models to predict patients’ odds of receiving at least one EBP session during their PTSD program admission. We began with a multivariate model that included patient and therapist characteristics and then added site nesting using random effects and robust standard error estimation. Logistic regression model output was described in terms of odds ratios and 95% confidence intervals. Model fit was described in terms of concordance statistic. All analyses were completed in SAS Version 9.4 (Carey, NC, USA).

Results

EBP Receipt Among All VA Residential PTSD Patients, Fiscal Years 2014–2016

There were 13,473 patients admitted to PTSD residential treatment programs from FY14 through FY16 (Table I). Of these, 28% (3,771) received at least one EBP session that was recorded with an EMR template. When template use became mandated in FY15, the proportion increased dramatically from 8.8% to 33.9%. Patient receipt of EBPs, as measured by EMR template use, stabilized by the second quarter of FY15, ranging between 36.4% and 46.2% of patients admitted each quarter through the end of FY16. Overall, according to the template data, 4.2% of patients received PE-I, whereas 12.7% received CPT-I and 15.3% received CPT-G.

Table I.

Delivery of Any PE or CPT by Fiscal Year and by Quarter, as Indicated by Template Use

Time	Patients, n	PE-I, % (n)	CPT-I, % (n)	CPT-G, % (n)	Any, % (n)
Overall	13,473	4.2 (564)	12.7 (1,715)	15.3 (2,055)	28.0 (3,771)
By fiscal year of admission
FY2014	4,441	1.3 (57)	3.2 (143)	6.1 (273)	8.8 (393)
FY2015	4,500	5.4 (242)	13.7 (618)	18.8 (847)	33.9 (1,527)
FY2016	4,532	5.8 (265)	21.1 (954)	20.6 (935)	40.8 (1,851)
By quarter of admission
FY14Q1	1,062	0.1 (1)	1.5 (16)	4.2 (45)	4.4 (47)
FY14Q2	1,151	0.9 (10)	2.3 (27)	4.3 (50)	6.5 (75)
FY14Q3	1,098	2.1 (23)	5.2 (27)	7.7 (85)	11.7 (129)
FY14Q4	1,130	2.0 (23)	3.8 (43)	8.2 (93)	12.6 (142)
FY15Q1	1,059	5.2 (55)	9.3 (98)	15.4 (163)	26.7 (283)
FY15Q2	1,132	5.7 (65)	13.7 (155)	21.9 (248)	36.9 (418)
FY15Q3	1,123	6.0 (67)	16.2 (182)	18.3 (205)	36.3 (408)
FY15Q4	1,186	4.6 (55)	15.4 (183)	19.5 (231)	35.2 (418)
FY16Q1	1,148	4.7 (54)	20.1 (231)	22.0 (253)	40.7 (467)
FY16Q2	1,159	7.2 (83)	23.4 (271)	23.6 (273)	46.2 (536)
FY16Q3	1,137	6.1 (69)	21.5 (244)	19.6 (223)	39.8 (452)
FY16Q4	1,088	5.4 (59)	19.1 (208)	17.1 (186)	36.4 (396)

PE, prolonged exposure; I, individual format; CPT, cognitive processing therapy; G, group format; FY, fiscal year; Q, quarter.

Accounting for sites where therapists did and did not participate in the survey, we were able to identify 46 active residential PTSD programs using FY15 CDW data. There was a variation in EBP use among sites (Table II). When both modalities and delivery formats were accounted for, 17 sites (37.0%) were frequent EBP users, 16 sites (34.8%) were occasional users, and 13 sites were rare users (28.3%).

Table II.

Site Characteristics, Fiscal Year 2015

PE-I Use
Frequent: 50–100% of patients, % (n)	4.3 (2)
Occasional: 10–49% of patients, % (n)	13.0 (6)
Rare: Less than 10% of patients, % (n)	82.6 (38)
CPT-I Use
Frequent: 50–100% of patients, % (n)	4.3 (2)
Occasional: 10–49% of patients, % (n)	43.5 (20)
Rare: Less than 10% of patients, % (n)	52.2 (24)
Any individual EBP
Frequent: 50–100% of patients, % (n)	21.7 (10)
Occasional: 10–49% of patients, % (n)	32.6 (15)
Rare: Less than 10% of patients, % (n)	45.7 (21)
CPT-G Use
Frequent: 50–100% of patients, % (n)	13.0 (6)
Occasional: 10–49% of patients, % (n)	26.1 (12)
Rare: Less than 10% of patients, % (n)	60.9 (28)
Any EBP for PTSD Use
Frequent: 50–100% of patients, % (n)	37.0 (17)
Occasional: 10–49% of patients, % (n)	34.8 (16)
Rare: Less than 10% of patients, % (n)	28.3 (13)

PE, prolonged exposure; I, individual format; CPT, cognitive processing therapy; G, group format; EBP, evidence-based psychotherapy; PTSD, post-traumatic stress disorder.

Representativeness of the FY15 Survey

There were 1,645 clinicians who coded at least one encounter with a residential PTSD patient as psychotherapy during FY15. The 159 therapists (10.0%) who participated in the FY15 survey did not significantly differ in gender or years of government service than the 1,486 clinicians who did not participate (Appendix 2). Survey participants were less likely to have a profession of nurse or psychiatrist, half as likely to have prescription privileges, and more likely to be a psychologist than non-participants. Thus, it appears that survey respondents were appropriately serving in primarily psychotherapist roles rather than medical roles within the residential PTSD treatment teams.

Among the 4,500 patients admitted to VA residential PTSD programs in FY15, 80.0% (n = 3,580) saw at least one therapist–respondent (Appendix 2). Compared with the 920 patients who saw only therapists that did not participate in the survey, these patients did not differ in gender or age. Patients who saw therapist–respondents were less likely to have psychiatric comorbidities, but slightly more likely to have a history of traumatic brain injury and to have a race of Hispanic/Latino or Asian/Pacific Islander.

Comparing Therapist-Reported Use and EMR Template Data

Therapist-reported frequent, occasional, and rare use of EBPs and corresponding measures of EBP use from EMR template data were adequately correlated. The polychoric correlation values were 0.77 for PE-I, 0.69 for CPT-I, and 0.82 for CPT-G. Dot plots (Appendix 3) show that EMR-measured EBP receipt was generally lower than therapist-reported EBP delivery, especially for CPT-I.

Characteristics of 2015 Fiscal Year Therapist–Respondents

Most therapists who completed the 2015 fiscal year survey were White women between the ages of 35 and 64 yr (Table III). They were most commonly psychologists or social workers with doctoral or master’s degrees. They had been in clinical practice for almost 15 yr, including 6.5 yr working in the residential program. They reported treating just over 14 residential PTSD patients per week. They achieved VA certification, consultant, or trainer status in CPT more frequently than in PE. Few reported using PE-I or CPT-I frequently, but a third reported using CPT-G frequently.

Table III.

Therapist Self-reported Characteristics, Fiscal Year 2015 Therapist Survey (n = 159)

Female gender	67.9 (108)
Age
Less than 35, % (n)	18.9 (30)
35–44, % (n)	30.2 (48)
45–54, % (n)	19.5 (31)
55–64, % (n)	23.9 (38)
65+, % (n)	3.8 (6)
Unknown, % (n)	3.8 (6)
Race
White, % (n)	83.6 (133)
Black/African American, % (n)	4.4 (7)
Spanish/Hispanic/Latino, % (n)	2.5 (4)
Other, % (n)	5.7 (9)
Unknown, % (n)	3.8 (6)
Profession
Psychologist, % (n)	50.9 (81)
Social Worker, % (n)	37.8 (60)
Other, % (n)	8.2 (13)
Unknown, % (n)	3.1 (5)
Most advanced degree
Doctoral, % (n)	54.1 (86)
Masters, % (n)	40.9 (65)
Other, % (n)	2.5 (4)
Unknown, % (n)	2.5 (4)
Experience
Years in clinical practice, M (SD)	14.7 (10.2)
Years in VA residential PTSD program, M (SD)	6.5 (5.9)
Number of residential PTSD patients per week, M (SD)	14.1 (9.8)
PE training and reported use (highest level)
Participated in training, % (n)	43.4 (69)
Achieved VA certification, % (n)	33.3 (53)
Consultant or trainer, % (n)	2.5 (4)
Self-rated skill level on 1–7 scale, median (IQR)	5.0 (4.0)
50–100% of patients in PE-I, % (n)	3.1 (5)
10–49% of patients in PE-I, % (n)	20.8 (33)
Less than 10% of patients in PE-I, % (n)	76.1 (121)
CPT training and reported use
Participated in training, % (n)	37.1 (59)
Achieved VA certification, % (n)	50.9 (81)
Consultant or trainer, % (n)	3.8 (6)
Self-rated skill level on 1–7 scale, median (IQR)	5.0 (2.0)
50–100% of patients in CPT-I, % (n)	2.5 (4)
10–49% of patients in CPT-I, % (n)	17.6 (28)
Less than 10% of patients in CPT-I, % (n)	57.2 (91)
50–100% of patients in CPT-G, % (n)	33.3 (53)
10–49% of patients in CPT-G, % (n)	8.2 (13)
Less than 10% of patients in CPT-G, % (n)	58.5 (93)

VA, Department of Veterans Affairs; CPT-I, individual cognitive processing therapy; CPT-G, group cognitive processing therapy; PE, prolonged exposure.

Characteristics of Patients Who Saw Therapist–Respondents

Patients who saw therapist–respondents were most commonly White men under the age of 35 yr (Table IV). Their most common mental health comorbidities were depressive disorders, alcohol use disorders, and anxiety disorders. Patients were admitted evenly over the four quarters of the 2015 fiscal year, with a mean length of stay of over 50 d. During that time, they saw a mean of 6.4 clinicians who coded at least one encounter as psychotherapy, of whom approximately 2.8 were therapist survey respondents. Less than 20% of patients received an individual EBP, but the percentage of patients receiving any EBP was raised to just over 35% when accounting for group therapy.

Table IV.

Characteristics of Patients Treated Whose PTSD Residential Program Therapists Participated in Fiscal Year 2015 Survey (n = 3,570)

Female gender, % (n)	11.2 (400)
Age
Less than 35, % (n)	27.9 (1,000)
35–44, % (n)	19.2 (687)
45–54, % (n)	23.0 (823)
55–64, % (n)	14.9 (534)
65+, % (n)	15.0 (536)
Race
White, % (n)	60.0 (2,148)
Black/African American, % (n)	24.9 (890)
Spanish/Hispanic/Latino, % (n)	8.2 (292)
American Indian/Alaskan Native, % (n)	1.9 (68)
Asian/Pacific Islander, % (n)	2.4 (86)
Comorbidities
Any listed below, including TBI, % (n)	85.5 (3,061)
Any listed below, excluding TBI, % (n)	84.9 (3,012)
Alcohol use disorders, % (n)	49.6 (1,776)
Anxiety disorders, % (n)	29.6 (1,059)
Bipolar disorders, % (n)	10.1 (360)
Depressive disorders, % (n)	61.4 (2,197)
Opioid use disorders, % (n)	11.7 (420)
Personality disorders, % (n)	5.8 (207)
Psychotic disorders, % (n)	5.1 (182)
Traumatic brain injury, % (n)	14.7 (525)
Residential treatment receipt
Admitted fiscal year 2015 quarter 1	24.1 (863)
Admitted fiscal year 2015 quarter 2	25.6 (915)
Admitted fiscal year 2015 quarter 3	25.2 (903)
Admitted fiscal year 2015 quarter 4	25.1 (899)
Length of stay (days), M (SD)	51.2 (35.5)
No. of clinicians who coding therapy encounters, M (SD)	6.4 (3.2)
No. of therapist survey participants, M (SD)	2.8 (1.5)
Received PE-I, % (n)	5.4 (192)
Median no. of sessions, received PE-I (IQR)	7 (5)
Received CPT-I, % (n)	13.6 (488)
Median no. of sessions, received CPT-I (IQR)	6 (8)
Received any individual EBP, % (n)	18.6 (665)
Median no. of sessions, received EBP-I (IQR)	7 (7)
Received CPT-G, % (n)	20.8 (744)
Median no. of sessions, received CPT-G (IQR)	8.0 (9)
Received any individual or group EBP, % (n)	35.8 (1,281)
Median no. of sessions, received EBP-I/G (IQR)	8.0 (8)

SD, standard deviation; IQR, interquartile range.

Predictors of EBP Use

A multivariate logistic regression model predicting receipt of any EBP had acceptable discrimination, with a concordance statistic of 0.77. Before controlling for site, almost all patient-level variables predicted EBP receipt. Far fewer patient factors predicted EBP receipt after controlling for site. These included female gender, alcohol use disorder, history of traumatic brain injury (TBI), admission later in FY15, length of stay greater than 6 weeks, and seeing more than 10 therapists during that stay. The sole patient factor that continued to predict not receiving an EBP after controlling for site was having a depressive disorder. Before controlling for site, the largest therapist predictor of EBP receipt was being an EBP consultant or trainer, followed by EBP certification, high self-rated skill level, and having 20 yr or more of clinical practice experience. Therapist factors that predicted not receiving EBP treatment included not having any EBP training, minority race, having worked in a VA residential PTSD program for 20 yr or more, and seeing 20 or more VA residential PTSD program patients per week. After controlling for site, only EBP consultant or trainer status remained as a positive therapist predictor of EBP receipt, whereas only minority race remained as a negative therapist predictor or EBP receipt. These results indicate a strong site effect on EBP receipt (Table V).

Table V.

Odds Ratio for EBP for PTSD Receipt (95% Confidence Interval), as Measured Using Templates

Predictor	Received EBP	Add Site
Female gender	0.87 (0.75–1.00)*	1.46 (1.03–2.08)*
Age less than 35	1.00 (reference)	1.00 (reference)
Age 35–44	1.18 (1.03–1.35)*	0.97 (0.68–1.40)
Age 45–54	1.19 (1.04–1.36)*	1.11 (0.79–1.56)
Age 55–64	1.12 (0.96–1.30)	1.13 (0.76–1.67)
Age 65+	0.82 (0.70–0.97)*	0.80 (0.49–1.29)
Race White	1.00 (reference)	1.00 (reference)
Race Black/African American	0.86 (0.77–0.97)*	0.80 (0.57–1.11)
Race Spanish/Hispanic/Latino	0.89 (0.75–1.06)	0.86 (0.61–1.22)
Race Other	0.91 (0.77–1.09)	0.66 (0.40–1.10)
Anxiety disorders	1.33 (1.20–1.48)***	1.04 (0.83–1.30)
Bipolar disorders	0.75 (0.64–0.88)**	0.81 (0.61–1.10)
Depressive disorders	0.82 (0.75–0.91)***	0.71 (0.54–0.92)**
Alcohol use disorder	1.23 (1.12–1.35)***	1.36 (1.06–1.75)*
Opioid Use disorders	0.99 (0.86–1.14)	0.98 (0.65–1.49)
Personality disorders	1.26 (1.03–1.53)*	0.64 (0.38–1.09)
Psychotic disorders	1.33 (1.09–1.64)**	0.80 (0.48–1.33)
Traumatic brain injury	1.05 (0.92–1.20)	1.53 (1.01–2.31)*
Admitted 2015 fiscal year, quarter 1	1.00 (reference)	1.00 (reference)
Admitted 2015 fiscal year, quarter 2	1.86 (1.63–2.12)***	3.25 (1.24–8.50)*
Admitted 2015 fiscal year, quarter 3	2.30 (2.02–2.63)***	5.47 (0.93–32.00)
Admitted 2015 fiscal year, quarter 4	2.61 (2.28–2.99)***	7.04 (1.20–41.30)*
Length of stay <6 wk	1.00 (reference)	1.00 (reference)
Length of stay 6–10 wk	4.40 (3.89–4.97)***	2.88 (1.75–4.74)***
Length of stay >10 wk	3.27 (2.74–3.90)***	7.11 (3.84–13.19)***
Total therapists seen <5	1.00 (reference)	1.00 (reference)
Total therapists seen 5–10	1.31 (1.15–1.50)***	1.56 (0.78–3.14)
Total therapists seen >10	2.35 (1.97–2.79)***	2.37 (1.11–5.07)*
Therapist race White	1.00 (reference)	1.00 (reference)
Therapist race Black/African American	0.59 (0.47–0.73)***	0.77 (0.65–0.92)**
Therapist race Spanish/Hispanic/Latino	0.10 (0.06–0.15)***	0.50 (0.33–0.75)***
Therapist race Other	0.08 (0.06–0.10)***	0.76 (0.58–1.00)*
Therapist race Unknown	0.19 (0.14–0.26)***	1.00 (0.74–1.36)
Therapist degree doctoral	0.62 (0.56–0.70)***	0.95 (0.75–1.19)
Therapist years in clinical practice, >20	1.30 (1.17–1.46)***	1.00 (0.84–1.20)
Therapist years in VA residential PTSD program, >20	0.63 (0.51–0.79)***	0.83 (0.57–1.20)
Therapist residential PTSD patients per week, >20	0.55 (0.49–0.61)***	1.09 (0.88–1.34)
Highest level of training, participated in EBP training	1.00 (reference)	1.00 (reference)
Highest level of training, achieved VA EBP certification	1.62 (1.44–1.81)***	1.10 (0.94–1.28)
Highest level of training, is EBP consultant or trainer	4.30 (3.41–5.43)***	1.38 (1.03–1.86)*
Highest level of training, no EBP training	0.09 (0.02–0.40)**	1.29 (0.66–2.54)
Therapist self-rated EBP skill level, PE + CPT score >8	1.55 (1.37–1.76)***	1.02 (0.80–1.30)

*p < 0.05; **p < 0.01; ***p < 0.001; EBP, evidence-based psychotherapy for post-traumatic stress disorder.

Discussion

Use of EMR templates to record the delivery of EBPs in VA residential PTSD treatment programs increased rapidly following a FY15 mandate. By FY16, over 40% of patients in VA residential treatment programs received an EBP for PTSD that was documented using an EMR template. EBP receipt in VA residential PTSD treatment programs as measured with EMR templates is relatively high compared with the most recently reported quarterly rate of 3.2% for all VA PTSD patients reported on the VA intranet.¹⁹

There was an adequate level of correlation between therapist self-report of EBT use and EMR template data in FY15, the last year for which therapist survey data are available. This suggests both that the mandate to use EBP templates was effective in driving use of the EMR templates in residential PTSD programs, and therapists working in these programs are reporting their EBP with EBP templates in a manner that is consistent with their own self-reports of EBP use.

Several patient factors predicted receipt of EBPs, even after controlling for therapist factors and clustering by site. Given prior data indicating greater therapy retention among women with PTSD,²⁰ it is not surprising that women were more likely to receive EBPs. As several programs have been specifically designed to treat PTSD in patients with comorbid alcohol use disorders^11,12 or history of TBI,^7,10 it is not surprising that these patient factors have become predictors of EBP receipt in the residential setting. Given that EBPs are delivered over the course of 8–12 wk in the outpatient setting, it is not surprising that patients receiving EBPs in the residential setting have longer lengths of stay. The only therapist factors that persisted as predictors of EBT receipt after controlling for site were EBP consultant or trainer status and race. It is encouraging that EBP consultants and trainers are also effective in delivering PE and CPT in their own practices. Therapist race data have not been available in prior work on patient–therapist matching in the VA.²⁰ Given that only 16.4% of the therapists were from minority populations, whereas 40.0% of patients were from minority populations, further exploration of racial disparities using this dataset may be warranted.

There are several limitations to this study. First, we used receipt of one or more EBP session as our outcome measure, while an adequate EBP dose is typically considered to be eight or more sessions,^21–23 as most patients receiving EBPs for PTSD typically achieve the bulk of their clinical gains by that point.^24,25 Because EMR template use was a new method to measure EBP receipt, we opted to use a measure that would avoid missing potential cases as much as possible. Future studies should examine not only whether patients received any EBP at all but also whether they received an adequate number of sessions. Second, we did not account for treatment fidelity. Although the EMR templates do have check boxes for session-specific skills, use of these additional elements varied widely between therapist and sites. Given that no patient-level data on EBP use in residential programs has previously been reported, we considered an evaluation of fidelity beyond our initial scope. Third, we did not examine treatment outcomes. Although more patient-reported outcome data are becoming available in the VA EMR over time,²⁶ it is generally not representative, so requires advanced matching approaches in order to perform comparative effectiveness analyses.²⁷ Given the complexity of this initial effort to leverage EMR template data, we considered these analyses beyond our scope. Finally, we may have missed EBP use by relying on EMR templates to create our outcome measure. For example, use was low during the first quarter of FY15 when the mandate requiring therapists to use EMR templates to document EBP use came into effect. However, use stabilized after the first quarter of FY15 and we accounted for quarter in our analyses. Regardless, examination of our data shows that psychotherapists reported delivering EBPs more frequently that was reflected by our template-based measures.

Psychotherapists’ apparent under-use of EBP templates is potentially problematic for the VA, as the only operational patient-level measures of PE and CPT receipt are template-based. Without a valid and reliable measurement strategy, it is difficult to understand the effects of VA EBP implementation efforts in order to make improvements where needed. This is especially a concern in residential settings, where patients spend a mean of over 50 d focusing on PTSD treatment. Recently, Sripada et al (2017) made a similar observation about psychotherapy for PTSD in the outpatient setting during the 2015 fiscal year, where a very high number of psychotherapy sessions were delivered to patients with PTSD, but only 3.6% of the patients had a session that was associated with use of a PE or CPT template.²⁸ Although Sripada et al also raised the possibility of therapists’ under-use of EBT templates and suggested incentivizing template use through incorporation of template-based measures into quality indicators, they cautioned that this strategy has the potential to induce gaming. As significant mental health comorbidity is highly common among VA patients with PTSD,²⁹ it is reasonable to assume that some patient–therapist dyads might choose to focus on indications other than PTSD in the outpatient setting and should be able to do so without feeling compelled to represent their sessions as PTSD-focused. In the residential PTSD setting, it is more difficult to rationalize a course of treatment that is not focused on using evidence-based methods to treat PTSD. Even residential programs that have a significant focus on comorbid conditions such as alcohol and traumatic brain injury have excelled at providing EBPs for PTSD.^7,11,12 Therefore, residential PTSD treatment programs could be a reasonable setting in which to pilot incentivizing PE and CPT use through template-based quality indicators. However, given our strong site-level findings, targeting individual providers with these incentives is not likely to be useful as it appears to be extremely difficult for an individual therapist to change practice without a strong organizational context.¹⁷ Thus, incentives to promote the use of PE and CPT might be more effective at the site level rather than the individual provider level. Furthermore, there are other evidence-based psychotherapy protocols such as eye movement desensitization and reprocessing that are recommended by the VA/DoD clinical practice guideline,³⁰ but for which there is currently no EMR template available. Therefore, it may not be reasonable to expect 100% compliance using a template-based measure even in the residential PTSD treatment setting.

In conclusion, use of EMR templates to document delivery of EBPs in VA residential PTSD programs increased rapidly and stabilized following a VA mandate to do so. There was adequate correlation between EBP for PTSD use as measured by EMR templates and therapist reports. Although many residential PTSD patients received EBPs in FY15, they were clustered at relatively few sites. The data suggest that future training and support efforts should target sites that rarely use EBPs.

Funding

Awards from the National Institute of Mental Health (R01MH096810) and VA Health Services Research and Development Service (CDA11-263) supported this work. The sponsors had no role in the conduct of these analyses or in the preparation of this manuscript. The opinions expressed are those of the authors and do not necessarily represent the positions of the National Institute of Mental Health or the US Department of Veterans Affairs.