Feasibility of delivering a virtual 1-day acceptance and commitment therapy workshop to rural veterans through community partnerships

Abstract

Background

This single-arm, open pilot study examined the feasibility and initial efficacy of a 1-day virtual Acceptance and Commitment Therapy (ACT) group workshop for distressed veterans.

Methods

We collaborated with veteran-serving community-based organizations to enhance outreach to veterans, especially those in rural areas. Veterans completed a baseline assessment and two follow-up assessments (1 month, 3 months) after workshop participation. Feasibility outcomes included reach (workshop recruitment and completion rates; veteran characteristics) and acceptability (open-ended survey question about satisfaction). Clinical outcomes included psychological distress (Outcome Questionnaire-45), stressor-related distress (PTSD Checklist-5), community reintegration (Military to Civilian Questionnaire), and meaning and purpose (PROMIS Short Form). Psychological flexibility (Action and Acceptance Questionnaire-II) – the proposed change mechanism underlying ACT – was also measured.

Results

Sixty-four veterans (50% rural, 39% self-identified as female) participated in a virtual workshop (97.1% completion rate). Overall, veterans liked the format and interactive nature of workshops. Convenience was noted as a benefit, while connectivity issues were highlighted as a drawback. Veterans showed improvements in psychological distress (F(2,109) = 3.30; p = 0.041), stressor-related distress (F(2,110) = 9.50; p = 0.0002), community reintegration (F(2,108) = 4.34; p = 0.015), and meaning and purpose (F(2,100) = 4.06; p = 0.020) over time. No between-group differences were detected, based on rurality or gender.

Conclusion

Pilot findings were promising and warrant a larger randomized trial to assess the efficacy of the 1-day virtual ACT workshop. Integrating community-engaged and participatory-research designs can enhance the external validity of these future studies and promote greater health equity.

Highlights

• •Veterans benefited from a virtual 1-day Acceptance and Commitment Therapy workshop.
• •Convenience was a noted advantage of offering a virtual 1-day workshop treatment.
• •While veterans liked the virtual workshop, some preferred to attend in-person.
• •Engaging community-based veteran-service organizations enhanced veteran outreach.

1. Introduction

The stressors of military service place veterans at risk for developing psychiatric disorders [1]. For example, an estimated 23% of veterans deployed during recent conflicts in Iraq and Afghanistan have received a diagnosis of posttraumatic stress disorder (PTSD) [2]. Veterans from older conflicts show similar rates of PTSD (1.9–24% of Persian Gulf War veterans and 8.5–19.3% of Vietnam veterans [3]). Between 9.6% and 17.8% of veteran samples report a depressive and/or anxiety disorder [[4], [5], [6]]. Most veterans meet diagnostic criteria for more than one of these conditions [7]. Failure to receive effective treatment can negatively impact long-term physical health functioning and lead to postmilitary readjustment challenges, such as relationship discord, difficulties with work or school, and poor community reintegration [[7], [8], [9]].

Several factors influence whether veterans engage in mental health services. Negative mental health-related beliefs and perceived stigma are major deterrents to treatment-seeking [10]. Common negative beliefs include views of mental illness and treatment-seeking as signs of weakness, perceptions that treatment will not address presenting concerns, and feared career consequences (e.g., having limited job options or being treated unfairly [11,12]). Military culture can reinforce therapy-inhibiting attitudes such as a high regard for self-reliance and stoicism [13]. Young-to-middle-aged veterans report practical barriers (e.g., competing family, work, or school obligations; time constraints) that make continual treatment engagement less feasible [12,14]. Living in rural, remote areas also significantly impacts treatment-seeking [15,16]. Ongoing shortages of mental health practitioners in rural communities reduce local availability of psychological services [17]. Consequently, rural veterans face an increased travel burden exacerbated by other logistical challenges, such as lack of dependent care, limited time off, and insufficient transportation [15]. Conventional models of psychotherapeutic service delivery (e.g., 8–12 weekly sessions delivered in a mental health facility during traditional business hours) are not well-suited to address these complex needs.

This study leveraged community-engaged methods and nontraditional treatment approaches to develop a mental health program aimed at improving veterans’ access to evidence-based psychotherapy, with an emphasis on rural veterans [18]. A major element of this approach has involved building relationships with veteran-serving community-based organizations (CBOs), including Veterans Health Administration (VHA) community clinics, veteran-based peer-networks, and veteran service organizations, to identify distressed rural veterans and engage them in a 1-day Acceptance and Commitment Therapy (ACT) workshop. We prioritized relationships with these community stakeholders because of their shared and local knowledge, cultural expertise, and credibility within the veteran community. Moreover, engaging community stakeholders in program development is a highly effective approach for promoting health care equity [19,20]. Early in the study, we established an advisory board comprising leaders from these organizations to help identify ways to maximize outreach to distressed rural veterans. Initial strategies included delivering in-person workshops in rural community settings to reduce travel burden and attending community events to increase visibility and build trust with veterans.

Brief, intensive treatment formats are growing in popularity as feasible alternatives to standard delivery (e.g., 8–12 weekly sessions) [21]). We offered a 1-day (5-to 6-h) ACT group workshop because it seemed better aligned with the needs of veterans. The 1-day format allows veterans to receive a full dose of therapy in a single session without their having to make ongoing concessions to attend weekly sessions. We present the treatment as a "workshop" rather than "therapy," as this terminology is less stigmatizing and a better fit for community settings where people may not be explicitly seeking psychotherapeutic services. Finally, workshops are based on ACT – an evidence-based transdiagnostic behavioral intervention with broad applicability to a variety of medical and psychiatric disorders [22,23]. Unified, transdiagnostic approaches such as ACT are well-suited for veteran populations, as they commonly present with more than one psychiatric condition [7]. Additionally, transdiagnostic approaches are beneficial for individuals who do not “fit” into specific diagnostic categories but are experiencing significant distress (subclinical or subsyndromal problems) and difficulties with social and occupational functioning. ACT helps patients develop skills to engage more fully in valued life activities [24]. There is an emphasis on the pursuit of valued life areas and directions, such as intimate relationships, meaningful work and personal growth, even when there are painful emotions, troubling thoughts and a strong motivation to escape or avoid them. Patients learn mindfulness, acceptance and committed action towards valued areas to enhance engagement in valued life areas and counter avoidance to life's challenges. Although ACT does not focus directly on symptom change, symptom reduction is a common byproduct (23).

Feasibility findings before the pandemic were promising. In collaboration with CBOs, 31 male veterans (87% rural) attended a 1-day in-person ACT workshop at a rural CBO (18). Every veteran who attended an in-person workshop completed it. Veterans exhibited improvements in symptom distress and functioning, community reintegration, and meaning and purpose 3 months after workshop completion. Qualitative interviews revealed that veterans enjoyed the format, content and environment of workshops. Many continued to use learned skills or sought new treatment following workshop participation. Of note, our partnerships with community stakeholders blossomed into an informal community network that could offer wraparound services to veterans and their families (e.g., housing assistance, transportation, food banks).

The unanticipated onset of the COVID-19 pandemic disrupted the second year of feasibility testing, which was focused on expanding outreach to rural women veterans. All in-person research and clinical activities were suspended in response to nationwide lockdowns and regulations against close physical contact. We worked aptly to modify the in-person workshop to accommodate the need for distance-based care. Video telehealth (VTH) emerged during the pandemic as a preferred treatment modality for delivering psychotherapeutic interventions because it could approximate face-to-face interactions while minimizing contamination risk [25,26]. Consistent with original study aims of improving access to mental health care, VTH bolsters treatment engagement by eliminating many logistical, financial, and transportation barriers [25]. It could also extend clinical outreach to populations that may not routinely engage in treatment [27]. This article presents preliminary findings on the feasibility, acceptability, and clinical effectiveness of delivering a virtual 1-day ACT workshop to distressed veterans. Potential between-group differences were explored, based on rurality and gender. We describe the process of modifying the workshop for VTH delivery elsewhere [Boykin et al., unpublished data, 2023]. We leveraged our veteran-serving CBO network to enhance outreach and recruitment of veterans in need.

2. Methods

2.1. Sample and procedures

Data were collected from October 1, 2020 to September 2021. Veterans were recruited in Southeast Texas with support from veteran-serving CBOs and word-of-mouth referrals from prior veteran workshop attendees. Interested veterans completed a brief telephone screening and were excluded only if they self-reported conditions that would interfere with workshop participation, specifically, a history of serious mental illness (e.g., bipolar disorder, psychosis), acute suicidal ideation, or severe/untreated substance or alcohol use [18].

Eligible veterans were invited to complete three assessments and attend a virtual 1-day ACT workshop after completing informed consent. Assessments were administered online or by mail at baseline and 1 month and 3 months after workshop completion. Veterans received $30 for completing each assessment. Study procedures were approved by the Baylor College of Medicine's Institutional Review Board and Michael E. DeBakey VA Medical Center's Research and Development Committee.

We partnered with over a dozen VHA and veteran-serving nonprofits across Southeast Texas. Dedicating significant time and resources to develop relationships with these agencies in the year before the pandemic set a strong foundation for maintaining connection with CBOs during the pandemic. Mandatory stay-at-home orders and the closing of nonessential businesses left fewer opportunities to interface with veterans directly. Our CBO partners made it possible to overcome these COVID-related challenges, especially with outreach to rural veterans. They helped disseminate information about upcoming virtual workshops through their email distribution lists, posting fliers to their websites, and making announcements at their online events. They also leveraged their professional networks to connect us with other organizations that predominantly serve women veterans and military families to enhance outreach to women. We communicated with CBOs routinely through telephone and videoconferencing calls.

2.2. Intervention

Virtual 1-day ACT workshops were held as a group, using Zoom for Healthcare. This version of Zoom is HIPAA-compliant and contains features to enhance patient privacy and security, including safeguards to protect against collecting personally identifiable data or unauthorized recording. Each workshop targeted psychological flexibility through instruction, demonstrations, and experiential exercises in two key areas. First, Acceptance and Mindfulness Training (2–3 h) involved practicing skills to create cognitive distance from unhelpful thoughts (such as “I'm not good enough”) and learning to willingly face experiences that cannot be changed. Second, Committed Action Training (2–3 h) involved clarifying who and what are most important to each person and identifying actions that honor those values. The ACT matrix provides an overall framework for discussing and practicing these skills [28]. Using the matrix, patients cultivate psychological flexibility by learning how to discriminate between direct experiencing through the five senses and indirect experiencing through the mind (vertical line) and to sort behaviors into those that move them TOWARD what is important or AWAY from painful experiences (horizontal line). A description of the workshop curriculum can be found in Dindo et al. [29].

At the recommendation of community stakeholders and prior workshop attendees, each workshop was conducted as single-gender groups to enhance a sense of comfort and safety. Groups consisted of three to eight veterans and were cofacilitated by two study therapists. Therapists included licensed psychologists, social workers, and doctoral-level psychology trainees. Workshops lasted about 6 h, with several 10-min breaks and a 30-min lunch break. Veterans were encouraged to attend sessions from a private location. We elected to use a PowerPoint slideshow as a visual aid with the intention of promoting engagement rather than expecting patients to “just listen”. This approach was also consistent with our use of a whiteboard during in-person workshops. Before the workshop, they received an ACT patient workbook, a set of wallet-size ACT cards that highlighted key skills, and information about VHA and community resources for addressing postmilitary challenges (e.g., mental health symptoms, chronic pain, employment).

2.3. Measures

2.3.1. Screening measures

Veterans were excluded from participating if they had a documented history of serious mental illness (e.g., schizophrenia, psychosis disorder, bipolar disorder) or scored a 2 or higher on the 4-item Cut down, Annoyed, Guilty, and Eye-opener screener for problematic alcohol and drug use [30]. A licensed psychologist or social worker completed a risk assessment with veterans who indicated “sometimes” or higher (frequently, always) on the suicide item of the Outcome Questionnaire-45 (OQ-45 [31]). Veterans were excluded for acute suicidality if they indicated ideation with any intent and/or plan.

2.3.2. Feasibility measures

Feasibility outcomes focused on reach and acceptability of the virtual 1-day ACT workshop through our community network [32]. Reach was defined in two ways. First, we measured the rates of recruitment and completion (i.e., attending 80% or more of the session or ≥ 4.5 h). Second, we assessed Veteran demographic characteristics to ensure inclusion of the intended clinical audiences (i.e., rural veterans, men and women). Acceptability (or veteran satisfaction) was assessed using a single, open-ended survey question at the 1-month follow-up assessment (“What did you think of the virtual form of the workshop? [For example, what did you like most about it? What did you like least about it]?”).

2.3.3. Clinical outcome measures

Primary and secondary clinical outcome measures were collected at the baseline, 1-month, and 3-month follow-up assessments. The primary outcome was psychological distress, as measured by the OQ-45 [31]. This well-validated measure contains three subscale scores – Symptom Distress (range from 0 to 100), Interpersonal Relations (range from 0 to 44), and Social Role Performance (range from 0 to 36) – as well as a total score (range from 0 to 180). Higher scores reflect greater distress and functional impairment. Internal consistency was good for the Total Score (α = 0.96), Symptom Distress (0.93), and Interpersonal Relations (0.86). It was acceptable for the Social Role Performance (α = 0.70).

Secondary clinical outcomes were stressor-related distress, community reintegration, and meaning and purpose. The PTSD Checklist-5 (PCL-5 [33]) is a psychometrically sound self-report measure of symptoms related to “stressful” life events (which may include traumatic experiences or COVID-19) within the past 30 days. A total severity score (range from 0 to 80) was computed, with higher scores indicating higher stressor-related distress. The Military to Civilian Questionnaire (M2C-Q [34]) is a validated self-report measure of postmilitary community reintegration difficulties (e.g., dealing with others, keeping friendships, participating in community events). Items were averaged with higher mean scores (range from 0 to 4), indicating greater difficulty with reintegration. Internal consistency was good for the PCL-5 (α = 0.96) and M2C-Q (0.93). The PROMIS Meaning and Purpose 4a Short Form [35] is a reliable self-report measure of the extent to which individuals feel their lives matter or make sense. A total raw score was computed and then converted to a T-score for each veteran.

Psychological flexibility was measured as the proposed mechanism of change in ACT interventions [24]. The Action and Acceptance Questionnaire (AAQ-II; [36]) is a well-validated self-report measure of psychological flexibility. It assesses one's willingness to remain in contact with difficult internal experiences (e.g., thoughts, emotions) while pursuing value-driven behaviors. Items were summed to compute a total score (range from 0 to 49), with lower scores indicating higher psychological flexibility. The AAQ-II showed excellent internal consistency (α = 0.96).

2.4. Data analysis

Feasibility data were measured descriptively (e.g., percentages, means, standard deviations). Independent samples t-tests and chi-square tests were conducted to assess for differences between workshop attendees and veterans who completed baseline but did not attend the workshop. Open-ended survey responses about the acceptability of the virtual ACT workshop were analyzed, using summative content analysis. Summative content analysis involves counting and comparing certain words and phrases subsequently interpreted relative to their contextual use [37]. Finally, linear mixed model for repeated measures was used to assess preliminary clinical effectiveness of the virtual 1-day ACT workshop. Changes from baseline to 3-month follow-up were computed for distress and functioning (OQ-45), stressor-related distress (PCL-5), community reintegration (M2C-Q), meaning and purpose (PROMIS short form), and psychological flexibility (AAQ-II). At each follow-up time point, the estimate of the mean change from baseline was obtained and tested for significance using Dunnett's test. To assess group differences (i.e., rural vs. urban veterans, men vs. women), the model was expanded to include group and group*time interaction effects.

3. Results

3.1. Feasibility outcomes

3.1.1. Reach

Fig. 1 shows participant flow. One-hundred and forty-two veterans were screened for eligibility. Recruitment rates were high (90.1%), with most veterans (n = 128) meeting eligibility criteria. Fourteen veterans were deemed ineligible due to severe, untreated alcohol or substance use (n = 12) or a diagnosis of serious mental illness (n = 2). Of the 128 eligible veterans, 102 completed the baseline assessment, with more than half attending a workshop (68.6%). Sixty-four of seventy veterans attended the workshop virtually. Data from the six veterans who attended an in-person workshop were excluded from current analyses. Completion rates were high (97.1%) with only two participants unable to complete a workshop due to a work emergency or persisting connectivity issues.

Fig. 1.

Consort diagram.

Demographic characteristics of eligible veterans who did and did not attend a workshop are listed in Table 1. More than a third of veterans self-identified as female. Half resided in a rural area. Veterans were middle-aged and predominantly self-identified as non-Hispanic Black, non-Hispanic White, or Hispanic/Latino. There were a few differences between workshop attendees and nonattendees. A higher percentage of workshop attendees lived in rural communities than nonattendees (chi-square = 15.66, p < 0.001). Additionally, workshop attendees reported lower levels of psychological distress (t[99] = 2.45, p < 0.05) and greater sense of meaning and purpose (t[96] = 2.50, p < 0.05) than workshop nonattendees.

Table 1.

Veteran characteristics and military history at baseline.

Variables	Baseline, no workshop (n = 29)		Workshop sample (n = 64)
	N (%)	Mean (SD)	N (%)	Mean (SD)
Age		41.76 (10.83)		44.61 (10.48)
Gender
Male	20 (69%)		39 (61%)
Female	9 (31%)		25 (39%)
Race/Ethnicity
White	6 (21%)		21 (33%)
Black	16 (55%)		28 (44%)
Native Hawaiian/Pacific Islander	0 (0%)		1 (2%)
Hispanic/Latino	5 (17%)		10 (16%)
Other/Unknown	2 (7%)		1 (5%)
Years of education		14.03 (1.70)		14.75(2.55)
Marital status
Single	8 (28%)		13 (20%)
Married	9 (31%)		31 (49%)
Divorced	7 (24%)		4 (22%)
Separated	3 (10%)		2 (3%)
Widowed	0 (0%)		2 (3%)
Cohabitating	1 (3%)		2 (3%)
Residence status
Urban Residence	22 (76%)		32 (50%)
Rural Residence*	5 (17%)		32 (50%)
Unknown	2 (7%)		0 (0%)
Annual household income
< $20,000	7 (24%)		11 (18%)
$20,000–39,999	13 (45%)		15 (23%)
$40,000-$59,999	5 (17%)		15 (23%)
$60,000 or more	4 (14%)		23 (36%)
Employment status
Employed (FT/PT)	12 (42%)		31 (48%)
Not employed	11 (38%)		24 (38%)
Student, Employed	3 (10%)		3 (5%)
Student, Not Employed	3 (10%)		6 (9%)
Military branch
Army/National Guard	15 (52%)		35 (54%)
Navy	10 (35%)		9 (14%)
Air Force	2 (7%)		10 (16%)
Marines	1 (3%)		8 (12%)
Coast Guard	1 (3%)		1 (2%)
Reserves	0 (0%)		1 (2%)
Deployed	18 (62%)		52 (81%)
Psychological distress (OQ-45)*		87.07 (30.01)		75.81 (30.58)
Stressor-related distress (PCL-5)		43.03 (20.65)		36.48 (20.96)
Community reintegration (M2C-Q)		2.12 (0.97)		1.87 (1.01)
Meaning and purpose (PROMIS MP)*		37.22 (10.23)		43.09 (11.59)
Psychological flexibility (AAQ-II)		29.49 (13.80)		25.45 (12.11)

OQ-45 = Outcome Questionnaire-45; PCL-5 = PTSD Checklist-5; M2C-Q – Military to Civilian Questionnaire; PROMIS MP = PROMIS Meaning and Purpose Short Form; AAQ-II – Action and Acceptance Questionnaire-II.

*significant between-group difference at p < 0.05.

While no veterans were excluded for acute suicidality, 26 (34.6% rural, 65.3% male-identifying) reported experiencing suicidal thoughts during the study. Of those 26, suicidal ideation was endorsed by 1 during the screening process, by 14 at the baseline assessment, by 3 at the 1-month or 3-month follow-up assessment, and by 8 across all time points. A notable proportion of these veterans (12 of 26) reported ongoing engagement with mental health practitioners.

3.1.2. Acceptability

Fifty-eight of fifty-nine veterans who completed the 1-month follow-up assessment answered the open-ended question about satisfaction with the virtual workshop. Most (77%) “liked” or “enjoyed” it. Some (14%) commented on how workshops were informative with opportunities for everyone to participate. As one wrote, “It was very informative, and I liked the interaction aspect in order to keep people focused.” Another shared, “I like how we were able to participate, and it wasn't a ‘death by PowerPoint’ presentation.” Several veterans (19%) liked connecting with peers (e.g., “Talking to people who can relate and people who understand veterans is a unique experience that feels like a family reunion”). Some (17%) benefited from hearing about other veterans' experiences and ideas (e.g., “I also like the opportunity to share and listen to other people (sic) struggles, so I know I am not alone.” and “I felt I was able to get a lot out of it and get good information out of the other ladies attending”). Five (9%) made positive comments about the workshop facilitators. Specific comments included: “I liked how the leaders of the workshop genuinely seemed interested in what we had to say,” and “… the administrators were very thoughtful and easy to talk to.”

Benefits and drawbacks to attending the virtual workshop were noted. Twenty-one percent of veterans mentioned convenience as an advantage of the virtual format (e.g., “I liked that we could join from anywhere,” “I like the flexibility of the format”). Three specifically wrote about the convenience of attending from “the comfort of my home,” while four liked “not having to travel.” Safety from COVID-19 exposure was noted by two (e.g., “I liked that I didn't have to travel to a location or worry about Covid-19 concerns from said travel”). Although veterans liked the virtual format, a few (10%) expressed a strong preference for in-person sessions. Their primary reasons for this preference centered on difficulties with feeling connected to peers on the virtual platform (e.g., “I didn't like that using a virtual format is a lot more difficult to get to really know a person; so many nonverbal cues are missed, and those are very important for a program focused on mental health”). Staying on topic and connectivity issues were mentioned by four (7%) as other drawbacks.

3.2. Preliminary clinical outcomes

Veterans who completed a virtual workshop and at least one follow-up assessment (n = 60 of 64) were included in the linear mixed model for repeated measures analysis (see Table 2).

Table 2.

Estimate of mean and mean change from baseline at 1- and 3-months follow-up from linear mixed model for repeated measures analysis to test initial efficacy of a 1-day virtual ACT workshop (n = 60 veterans).

Study phase	Mean	Change from Baseline
	(SEM)	Mean	95% CIa	p-valuea
OQ-45	Time effect F(2,109) = 3.30			0.041
Baseline	76.3 (4.0)	–
1 month	72.2 (4.0)	−4.1	(-9.7, 1.6)	0.189
3 months	69.6 (4.1)	−6.7	(-12.6, −0.7)	0.025
OQ-45 SD	Time effect F(2,110) = 4.67			0.011
Baseline	42.5 (2.3)	–
1 month	42.8 (2.3)	−2.5	(-5.4, 0.4)	0.106
3 months	40.3 (2.3)	−4.9	(-8.5, −1.3)	0.005
OQ-45 IR	Time effect F(2,110) = 1.77			0.175
Baseline	18.4 (0.9)
1 month	18.0 (0.9)	−0.4	(-1.7, 1.0)	0.743
3 months	17.0 (0.9)	−1.4	(-3.2, 0.4)	0.136
OQ-45 SR	Time effect F(2,106) = 0.46			0.635
Baseline	13.1 (0.7)
1 month	12.6 (0.7)	−0.5	(-1.8, 0.7)	0.514
3 months	12.7 (0.7)	−0.4	(-2.1, 1.2)	0.774
PCL-5	Time effect F(2,110) = 9.50			0.0002
Baseline	37.1 (2.7)
1 month	31.4 (2.7)	−5.8	(-10.1, −1.4)	0.007
3 months	28.6 (2.8)	−8.5	(-13.1, −4.0)	<0.0001
M2C-Q	Time effect F(2,108) = 4.34			0.015
Baseline	1.90 (0.13)
1 month	1.71 (0.13)	−0.19	(-0.39, 0.02)	0.074
3 months	1.63 (0.14)	−0.27	(-0.48, −0.05)	0.011
PROMIS MP	Time effect F(2,100) = 4.06			0.020
Baseline	42.9 (1.5)
1 month	45.8 (1.5)	2.9	(0.6, 5.3)	0.010
3 months	44.6 (1.5)	1.7	(-0.8, 4.2)	0.227
AAQ-II	Time effect F(2,110) = 2.67			0.073
Baseline	25.8 (1.6)
1 month	23.5 (1.6)	−2.4	(-5.3, 0.5)	0.124
3 months	23.0 (1.7)	−2.8	(-5.8, 0.2)	0.068

ACT = Acceptance and Commitment Therapy; OQ-45 = Outcome Questionnaire-45; SD = Symptom Distress Scale; IR = Interpersonal Relationship Scale; SR = Social Role Performance Scale; PCL-5 = PTSD Checklist-5; M2C-Q = Military to Civilian Questionnaire; PROMIS MP = PROMIS Meaning and Purpose Short Form; AAQ-II – Action and Acceptance Questionnaire-II.

*significant change at p < 0.05.

^aDunnett's adjustment applied to 95% CI and p-value.

Veterans showed a significant mean change in the primary clinical outcome – distress and functioning (OQ-45 total score) – over time [time effect: F(2,109) = 3.30; p = 0.041]. A greater mean decrease was observed at 3 months than at 1 month. Further assessment of the OQ-45 subscales showed a significant effect of time on mean score change in the symptom distress domain [F(2,110) = 4.67; p = 0.011), but not in the other functional domains (interpersonal relations: F(2,110) = 1.77; p = 0.175; social roles: F(2,106) = 0.46; p = 0.635). Veterans exhibited a higher decrease in symptom distress at 3 months than 1 month.

There was a significant effect of time on mean score change in stressor-related distress (time effect [F(2,110) = 9.50; p = 0.0002]), community reintegration (time effect [F(2,108) = 4.34; p = 0.015]), and meaning and purpose (time effect [F(2,100) = 4.06; p = 0.020]). For stressor-related distress, veterans showed significant reductions in symptom severity across both time points, with a larger mean decrease observed at 3 months. For community reintegration, significant improvements were seen only at 3 months and not at 1 month. While there was a significant mean increase in meaning and purpose at 1 month, no significant mean change was observed at 3 months. Contrary to expectation, the differences in mean score change for psychological flexibility – a proposed mechanism of change in ACT interventions – was not significant across time [time effect: F(2,110) = 2.67; p = 0.073].

Group differences in patient-level outcomes over time were assessed based on rurality and gender. No significant effect of rurality on mean change scores across time (rurality*time interaction) was observed for outcome variables (p values between 0.275 and 0.999). Similarly, there was no significant effect of gender on mean change scores across time (gender*time interaction) for study variables (p values between 0.275 and 0.993).

4. Conclusion

In response to COVID-19 challenges, we adapted a 1-day ACT (in-person) workshop for virtual delivery. This article reports on preliminary findings of the feasibility, acceptability, and efficacy of delivering the virtual workshop to distressed veterans, with support from veteran-serving CBOs. Nearly all who attended a virtual workshop completed it. Veterans liked that the workshop was interactive and provided a way for them to connect with peers in a safe environment. They showed sustained improvements for 3 months across several functional domains, including emotional well-being, community reintegration, and meaning and purpose. It is promising that clinical outcomes did not differ based on rurality or gender. Because this was a feasibility pilot, we elected a single-arm design, recognizing that it would be difficult to discern whether clinical improvements were an effect of treatment or due to extraneous variables (e.g., timing of assessments, changes in life stressors). Nevertheless, these preliminary results are consistent with earlier findings from larger efficacy trials comparing the 1-day ACT workshop (in-person) to a wait list or education-only control condition [[38], [39], [40]]. Larger randomized, multisite trials will be needed to establish the efficacy of the virtual workshop.

VTH has been long recognized as another effective strategy for bolstering treatment engagement across diverse clinical populations [41,42]. Veterans appreciated the convenience of attending workshops from home (or another private location). Other benefits included reduced risk for COVID-19 exposure and fewer logistical barriers to participation (e.g., need to travel). VTH better simulates face-to-face interactions than audio-only or other health care messaging platforms. However, it should not be seen as a replacement for in-person clinical encounters. There will always be patients who prefer in-person sessions, as observed in this study. Connectivity issues (e.g., frozen video, distorted audio, weak broadband signal) can arise at any time and affect participation. Some veterans emphasized the importance of seeing more nonverbal communication than facial expressions to feel “connected” with others. Finally, evidence of a “digital divide” (i.e., inequitable access to technology) among socially disadvantaged groups is an ongoing concern [43,44]. Although no veterans expressed difficulties with accessing the technology needed to join via Zoom, it is crucial to understand and address any technological and financial barriers to VTH use.

Overall, enrollment rates in the virtual workshops were acceptable. More than half of the eligible veterans who completed a baseline assessment attended a virtual workshop. Given the timing of this study (October 2020–September 2021), it is difficult to discern the degree to which nonparticipation was driven by patient motivation, intervention characteristics (e.g., workshop length, modality format), or COVID-related issues. There are many benefits to brief, intensive treatment formats – namely, the increased opportunity to receive an effective dose of treatment in fewer sessions. However, in the context of the pandemic, patients may have had more competing priorities than usual (e.g., care of dependents or loved ones due to school closures or quarantine, changing work situations) that limited their availability [45]. Alternatively, the restrictions imposed by COVID-19 may have led to greater isolation and boredom, therefore increasing Veterans’ interest in participating. Overall, the global impact of COVID-19 on mental and physical well-being cannot be understated [46]. The long-term health, economic, and social implications of COVID-19 are still under investigation and should be assessed within clinical trials to understand their continued influence on treatment outcomes.

It is notable that veterans participating in the virtual 1-day ACT workshop demonstrated small, nonsignificant changes in psychological flexibility. This contrasts with mounting evidence that in-person 1-day ACT workshops yield significant improvements in psychological flexibility that mediate changes in symptom severity [18,[38], [39], [40]]. Web-based and remotely delivered ACT intervention studies have not all produced statistically or clinically significant changes in psychological flexibility (as measured by the AAQ-II [[47], [48], [49]]). One working hypothesis is that virtually based ACT interventions may be less sensitive in detecting that treatment effect [47]. Given the circumstances under which this virtual ACT workshop was developed (i.e., rapid adaptations prompted by COVID-19 social restrictions), further modifications may be needed to enhance the intervention's potency. For example, engagement is a critical aspect of workshops. A major drawback to delivering a virtual group is that the natural flow of group discussion is distilled into one person speaking at a time, which can interfere with building group cohesion. There are also fewer opportunities for patients to have impromptu conversations with one another during breaks and lunch. Although patients have not expressed concerns about the length of workshops (five to 6 h), facilitators noticed increased variability in patient engagement on the virtual platform. Patients were encouraged to attend from a private location and silence electronic devices (e.g., televisions, mobile phones), but it can be difficult to control external distractions for long periods. Throughout the study, we found different strategies that were helpful in scaffolding engagement, such as greater use of Zoom chat, calling on patients by name, taking more short breaks, and adding short ice-breaker activities.

There were many benefits to engaging community stakeholders and veterans in clinical research beyond enhancing outreach to culturally diverse veteran populations. By design, community-engaged and participatory-research approaches leverage the strengths and expertise of patients or community stakeholders to increase the cultural relevance, fit, and adoption of evidence-based practices with the intended clinical audiences [20,50,51]. Many veterans have difficulty engaging in standard treatment delivery models, especially those living in rural areas [12,14,16]. While we understood these challenges, we relied on the expertise and influence of veterans, VHA, and veteran-serving nonprofit stakeholders to identify feasible solutions to engage rural veterans in a 1-day ACT workshop. Another benefit of building collaborative relationships with community stakeholders was the formation of a local network of services that addressed a broader range of mental health and social needs. Notably, it took significant time and resources to cultivate meaningful relationships. Nevertheless, more funding agencies like the Department of Veterans Affairs and Patient-Centered Outcomes Research Institute are encouraging stakeholder-engaged research because of its potential to accelerate the research-to-practice timeline, fill gaps in care, and prevent health inequities [50,51].

This study was not without limitations. The sample size was small but reasonable for an open feasibility pilot trial. While clinical improvements cannot solely be attributed to the virtual 1-day ACT workshop due to the uncontrolled, single arm trial, findings warrant further investigation and replication with larger, more rigorous controlled designs. Treatment fidelity was not measured as part of this study but will be important to assess in future studies. We were careful to maintain treatment fidelity when transporting the in-person workshop into a virtual format. However, without formal assessment, we cannot ensure that treatment fidelity was maintained. Therapist adherence and competence should also be evaluated as virtual interventions require more than skillful delivery of the therapeutic approach, but also competence in using VTH systems.

Other notable study limitations concern measurement of certain study outcomes. First, we opted for a single free-text response item to measure acceptability of the virtual format to ease participant burden and fatigue during the pandemic. Free-text survey items do not necessarily reach the standards of qualitative research as they lack data richness, are sometimes omitted, and cannot be analyzed using rigorous qualitative procedures [52]. Qualitative interviews would be preferable for gain an in-depth understanding veterans’ experiences and suggestions for improvement. Second, the PCL-5 was administered in the absence of a comprehensive measure of trauma exposure and should be interpreted as a measure of distress to life stressors broadly (such as COVID-19). Given the high rates of trauma exposure reported in military samples [53], a formal measure of trauma exposure should be included in future trials that use this population. Finally, some studies have shown that the AAQ-II does not have sufficient discriminant validity from measures of psychological distress, particularly depressive and anxiety symptoms [[54], [55], [56]]. Thus, the AAQ-II may not uniquely assess psychological flexibility although measurement of this complex construct remains a matter of debate [57]. The COVID-19 pandemic has marked a permanent shift in health care delivery toward greater use of VTH and digital technologies. Early comparative-effectiveness studies showed no differences between in-person and VTH delivery of psychotherapeutic interventions [41,42], but it still may be beneficial to replicate these findings with newly adapted VTH interventions. VTH fits well with patient-centered models of care, as it gives patients more choice over when, where, and how they receive care. Toward that end, it will be necessary to implement clinical trial designs that can account for patient preference to help us better understand its impact on treatment engagement, completion, and clinical outcomes. Specific attention should be directed toward the inclusion of underserved patient groups in future studies to promote health equity early in the intervention development process. As we have shown, this can be accomplished through engaging key stakeholders (e.g., patients, administrators, community leaders) who use or have the power to influence use of empirically supported psychotherapies.

Funding

This material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Rural Health, Veterans Rural Health Resource Center - Iowa City (Award #13367) to L. Dindo and G. True and (PROJ-03608) to D. Boykin. This work was also partially supported by the use and resources of the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN13-413) and the South Central Mental Illness Research Education and Clinical Center. All procedures were approved by the Baylor College of Medicine IRB (#H-43608 and H-50441) and the Michael E. DeBakey Veterans Affairs Medical Center. The funders did not play a role in study design; in collection, analysis and interpretation of data; in writing of the report; or in the decision to submit the article for publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.