Seeing the forest for the trees: Predicting attendance in trials for co-occurring PTSD and substance use disorders with a machine learning approach

Abstract

Objective:

High dropout rates are common in randomized clinical trials (RCTs) for comorbid posttraumatic stress disorder and substance use disorders (PTSD+SUD). Optimizing attendance is a priority for PTSD+SUD treatment development, yet research has found few consistent associations to guide responsive strategies. In this study we employed a data-driven pipeline for identifying salient and reliable predictors of attendance.

Method:

In a novel application of the iterative Random Forest algorithm (iRF), we investigated the association of individual level characteristics and session attendance in a completed RCT for PTSD+SUD (n = 70; women = 22 [31.4%]). iRF identified a group of potential predictor candidates for the total trial sessions attended; then, a Poisson regression model assessed the association between the iRF-identified factors and attendance. As a validation set, a parallel regression of significant predictors was conducted on a second, independent RCT for PTSD+SUD (n = 60; women = 48 [80%]).

Results:

Two testable hypotheses were derived from iRF’s variable importance measures. Faster within-treatment improvement of PTSD symptoms was associated with greater session attendance with age moderating this relationship (p = 0.01): faster PTSD symptom improvement predicted fewer sessions attended among younger patients and more sessions among older patients. Full-time employment was also associated with fewer sessions attended (p = .02). In the validation set, the interaction between age and speed of PTSD improvement was significant (p = .05) and the employment association was not.

Conclusions:

Results demonstrate the potential of data-driven methods to identifying meaningful predictors as well as the dynamic contribution of symptom change during treatment to understanding RCT attendance.

Posttraumatic stress disorder (PTSD) is often associated with substance use and substance use disorders (SUDs), with the diagnosis of PTSD conferring a two- to four-fold risk of additionally carrying a SUD diagnosis (Pietrzak et al., 2011). The comorbidity of PTSD and SUD (PTSD+SUD) exacerbates each disorder, yielding a population that suffers from more severe consequences and poorer outcomes than their counterparts with either disorder alone (Debell et al., 2014; Ouimette et al., 1997; Smith & Randall, 2012). Although a body of research supports the safety and utility of concurrent interventions for PTSD+SUD (Coffey et al., 2016; Foa et al., 2017; Mills et al., 2012; Norman et al., 2016; Ruglass et al., 2017), the effectiveness of PTSD+SUD interventions is critically hampered by low attendance in PTSD+SUD clinical trials where 32 to 61% (Brady et al., 2001; Zandberg et al., 2016; Hien, 2009; Ghee et al., 2009; Myers et al., 2015; Szafranski et al., 2017; Coffey et al., 2016) of participants fail to attend the recommended number or all treatment sessions—two commonly used definitions of treatment attrition¹ in randomized controlled trials (RCTs).

Recent reviews and meta-analyses of PTSD+SUD trials (Roberts et al., 2015; Simpson et al., 2017) have noted that low attendance rates are a fundamental challenge for treatment effectiveness and implementation. As comparison, meta-analyses of PTSD trials have reported pooled attrition rates between 16% and 20% (Lewis et al., 2020; Imel et al., 2013; Bradley et al., 2005; Hembree et al., 2003). This difference is particularly notable given that most PTSD controlled trials—and all of PTSD treatment meta-analyses—exclude participants with comorbid SUDs. Indeed, attendance in PTSD+SUD studies bears much closer resemblance to that of interventions for SUDs (where participants with comorbid disorders like PTSD are included) and highlights the challenge of addressing PTSD in the context of addiction. Treatment attendance is notoriously difficult in the face of severe SUD; in pooled estimates of SUD treatment studies, close to one third (30%) of participants do not complete controlled trials (Lappan et al., 2019), with the highest rates in studies targeting cocaine (49%).

Although numerous factors influence treatment attendance in RCTs, including perceived need and symptom improvement (Zandberg et al., 2016; Szafranski et al., 2019), low attendance in PTSD+SUD clinical trials is frequently tied to assumptions about intervention tolerability. Within the category of integrated PTSD+SUD approaches, interventions that directly address traumatic memories or related thoughts (exposure-based) have shown stronger effects on PTSD symptoms than therapies that do not directly discuss traumatic material (Roberts et al., 2015). However, acceptability and retention concerns have followed exposure-based PTSD+SUD psychotherapies. PTSD interventions such as prolonged exposure (PE), cognitive processing therapy, narrative exposure therapy, and eye-movement desensitization and reprocessing rely on a range of cognitive and affective processing strategies—including direct retelling of traumatic memories—and/or the reduction of maladaptive cognitive and behavioral avoidance through therapeutic exposure and discussion. Given the nature of PTSD, discussing trauma-related content may feel aversive for both the client and clinician, and some have suggested that this contributes to missed sessions or treatment dropout (Ruzek et al., 2014). However, single trial and meta-analytic findings have yet to detect differential dropout rates between PTSD+SUD therapies with and without an exposure component (Back et al., 2006; Roberts et al., 2015). Thus, it appears that difficulties attending any intervention—rather than an exposure-based one specifically—accounts for attrition in PTSD+SUD trials. Until attendance rates in PTSD+SUD trials are sufficiently understood and addressed, a clear determination of how to best address PTSD+SUD and harness specific treatment mechanisms of action (i.e., which sessions of which therapies are most necessary?) remains unlikely.

Beyond certain un-modifiable variables like younger age (Adamson et al., 2009; Schottenbauer et al., 2008) and lower education (Belleau et al., 2017; Rizvi et al., 2009), few factors have been found to consistently predict attendance in either PTSD or SUD treatment trials. There is some evidence to suggest that clinical features such as higher severity of PTSD (Schottenbauer et al., 2008), alcohol use disorder (Adamson et al., 2009), longer substance use history (Dutra et al., 2008), and depression (e.g., (Curran et al., 2002) may inhibit treatment completion in SUD trials. But as with demographic characteristics, their predictive power has proved relatively unstable. Unsurprisingly, predicting treatment attendance within PTSD+SUD trials has similarly equivocal results. Some have found that higher education (Belleau et al., 2017; Brady et al., 2001; Foa et al., 2013; Pinto et al., 2011) and being older (Foa et al., 2013; Myers et al., 2015; Pinto et al., 2011) are associated with higher rates of attendance while others (Hien et al., 2004; Morgan-Lopez et al., 2014; Szafranski et al., 2017) have found that no single demographic variable differentiates those who complete treatment from those who do not. Notably, these findings cut across exposure-based and other treatment approaches. Furthermore, suggestive of a connection between being younger and the logistical hurdles associated with child-rearing, two trials with women-only samples found that higher attendance rates were associated with having fewer dependents (Myers et al., 2015) and on-site childcare (Pinto et al., 2011). Clinical predictors of attrition in PTSD+SUD trials have similarly yielded mixed findings. In an early clinical trial of exposure therapy for PTSD and comorbid cocaine dependence, completing less than three exposure sessions was associated with higher levels of baseline avoidance (Brady et al., 2001). Conversely, having more avoidance symptoms was predictive of more treatment engagement (defined as attending at least 6 sessions) in Seeking Safety for women with alcohol use disorder (AUD) and PTSD. Although trending results suggest that higher baseline PTSD severity predicts exposure-based treatment completion in Veterans with PTSD+SUD (Szafranski et al., 2017), it did not predict AUD+PTSD attrition in an analysis of the largest group of candidate predictors to date suggested (Zandberg et al., 2016). Indeed, in this study, trauma type was the only one baseline trauma-related variable that predicted attrition (defined as not completing the full treatment) wherein accidents and “other” traumas predicted higher attrition than physical assault. Lastly, and in contrast to findings from general SUD trials, severity of problematic alcohol or drug use at the start of treatment has generally not been associated with attendance in PTSD+SUD trials (e.g. Belleau et al., 2017; Szafranski et al., 2017; Zandberg et al., 2016)

One potentially important explanation for the inconsistency of predictors of RCT attendance is that treatment non-completion or low attendance may not always be indicative of treatment failure. To this end, researchers have begun incorporating within-treatment response into the study of PTSD+SUD trial attendance. Several studies (Jarnecke et al., 2019; Kline et al., 2021; Szafranski et al., 2019; Zandberg et al., 2016) have tracked the impact of symptom change upon PTSD+SUD intervention attendance and trial completion and found varying interactive effects between within-treatment symptom changes, baseline symptomatology, intervention type, and attrition. Some of their findings support prior work (Erbes et al., 2009; Galovski et al., 2012; Hien et al., 2012; Krishnamurthy et al., 2015; Szafranski, Smith, Gros et al., 2017) that suggest attrition and lower session attendance may not always be synonymous with treatment failure and, for some, may be related to a favorable treatment response. For example, Szafranski et al., (2019) found in a sample of non-completers (n=22) of PE that 40-68% had achieved clinically significant changes in PTSD, depression, and alcohol use. Two studies have specifically examined the predictive value of within-treatment symptoms changes and their interactions with other variables such as level of severity and intervention delivered. In a trial of combined pharmacotherapy and PE for AUD+PTSD, Zandberg et al. (2016) found a curvilinear effect of PTSD improvement rate moderated by baseline PTSD severity. In participants with initially high PTSD severity, attrition was most likely when improvement was either very fast or very slow. However, for participants who had less severe baseline PTSD symptoms, attrition was more likely only after rapid symptom relief. This suggests that individuals drop out of treatments for multiple and interacting reasons that range between having a poor response and a rapid, favorable one. The type of PTSD+SUD intervention also appears to influence the association between within-treatment symptom changes and attendance, although the findings to date are mixed. Faster reductions in drinking days have increased dropout risk only in participants who received PE, but not in those who received alcohol counseling alone (Zandberg et al., 2016). Kline et al. (2021) also found an association exclusive to PE and drinking, but in the opposing direction; increases of within-treatment, weekly drinking was associated with dropout risk in the PE condition, but not in the Seeking Safety (skills-based) condition. Although these findings further reinforce the importance of incorporating within-treatment symptom changes to PTSD+SUD attendance models, their conclusions are inconsistent and generalizability limited by their restriction to alcohol problems and their predominantly male, veteran samples. Additional work is necessary to clarify which predictors of PTSD+SUD treatment attendance are generalizable, including within-treatment clinical markers, in order to understand whether and when lower attendance is problematic and for whom.

A final and critical consideration for advancing the study of attendance in PTSD+SUD trials is methodological and involves addressing limiting factors related to terminology, measurement, and analysis. There is presently an array of overlapping terms (e.g. treatment engagement, adherence, retention, attrition, and dropout) and a lack of agreed-upon definitions. Conceptually, attrition broadly describes treatment that has begun but has not been completed. Attrition may occur at various stages throughout the treatment process—after first inquiry, after treatment has begun, or after treatment has been completed but before follow-up sessions—with each stage representing potentially distinct phenomena and associated factors. Thus, it has been recommended that research should take into account the stage at which attrition occurs when examining predictors (Schottenbauer et al., 2008; Resko & Mendoza, 2012). For the purposes of understanding the potentially interacting roles of treatment type and within-treatment variables, a focus on participants who have attended at least one intervention session (one “dose”) would be prudent. However, even within the period after treatment has begun, trial non-completion has been variously defined by investigators, with some operationalizing it as attending at least one but less than the total number of sessions prescribed (Szafranski et al., 2017; Zandberg et al., 2016), and others arguing that a definition should be tethered to a hypothesized threshold or minimum sessions attended of an intervention. Studies abiding by this rationale employ a pre-set proportion (e.g. 25% of Seeking Safety sessions; [Hien et al., 2009; Myers et al., 2015; Najavits et al., 1998] or number (i.e. three exposure sessions; [Belleau et al., 2017; Brady et al., 2001; Ruglass et al., 2017]) to define attrition as not having received the sessions necessary for a therapeutic effect. Until consensus is reached among these competing standards for treatment non-completion, the measurement of attendance more broadly (i.e., as count data) can provide more descriptive, objective, and generalizable guideposts than results bound to a pre-defined term.

In sum, low trial attendance has questioned both PTSD+SUD treatment and research on its efficacy and effectiveness. Disentangling who is less likely to attend trial sessions, and how this relates to their clinical within-treatment trajectory is germane to understanding critical issues of attendance and retention in PTSD+SUD treatment research. However, research to date has shown that the potential characteristics associated with low retention are numerous and diversely interact with how participants respond to an intervention, with under-attendance resulting from potentially poor or rapid treatment responses. The vast number and inconsistency of known predictors in the PTSD+SUD domain makes difficult separating information about which variables predict RCT attendance on their own and interactively; this poses a challenge for modeling with linear regressions, where clear, well-defined hypotheses are necessary. At this juncture in the study of trial attrition prediction, new approaches for narrowing the selection of variables of interest are thus crucial. Machine learning techniques to facilitate variable selection are an emergent and promising strategy (Guyon & Elisseeff, 2003; Kirpich et al., 2018; Speiser et al., 2019) that holds potential for use in patient-centered research, where small sample sizes are the norm. Applying machine learning approaches as tools for identifying and prioritizing variables of interest is particularly useful in scenarios akin to that of trial attrition, where the roles and interactions among a high number of variables are unclear. One such strategy is a type of ensemble learning, the iterative Random Forest algorithm (iRF; Basu et al., 2018). Unlike machine learning methods like deep learning and artificial intelligence, RF algorithms can have good model performance in small sample settings (Guo et al., 2010). iRF is an enhanced version of the random forest algorithm (Breiman, 2001) ), specifically applied for regression and classification (Liaw & Wiener, 2001). One advantage of iRF is that it enables discovery of both linear and non-linear relationships in a systematic, data-driven way. When applied as a variable selection technique, iRF may be used to facilitate the generation of innovative, data-driven hypotheses, which can then be tested through traditional regression analyses.

Present Study

We employed a data-driven approach to address variabilities in the prediction of treatment attendance, advancing the methodology and study of this important problem. In the present study we used a machine learning algorithm (iRF) to identify important predictors of session attendance from candidate variables in a trial (Ruglass et al., 2017) which compared Concurrent Treatment for SUD and PTSD using PE (COPE; Back et al., 2014) to a SUD-only treatment (Carroll, 1998; Marlatt & Donovan, 2005). Our candidate variables included a set of baseline demographic and clinical characteristics and variables measuring within-treatment changes in PTSD symptoms and problematic substance use. An important aspect in data-driven research is the validation of findings in an independent dataset. In order to corroborate any statistically significant relationships found in the first dataset (Ruglass et al., 2017), we tested the significant treatment attendance predictors in a second dataset RCT (Hien et al., 2015) of a skills-based PTSD+SUD intervention: Seeking Safety (Najavits, 2002) with and without sertraline. iRF has been used in other fields (Zhao et al., 2020) and offers an opportunity to identify associations between treatment attendance and a group of putative predictors that can facilitate novel, testable hypotheses. Given the increasingly available high dimensional data in PTSD+SUD treatment studies, the introduction of this data-driven approach to the field may prove timely.

Method

The first set of analyses (Trial 1) employed data from the completed RCT which compared COPE to RPT and an active monitoring control group (see Ruglass et al., 2017 for complete details of the procedures). Only COPE and RPT arms were utilized in the current analyses. Results from Trial 1 were cross-validated with a second, independent RCT (Trial 2), which compared the efficacy of Seeking Safety (Najavits, 2002) with and without sertraline (see Hien et al., 2015 for complete details of the procedures). Ethical approval was obtained for each of the clinical trials (Clinical Trials Registration: clinicaltrials.gov Identifier: NCT01365247 and NCT00262223) from which these data were drawn. All participants signed written informed consents approved by the institutional review board.

Participants

Eligibility criteria

Trial 1 inclusion criteria consisted of: 1. 18-65 years of age; 2. meeting criteria for full or subthreshold PTSD per Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision (DSM-IV-TR; American Psychiatric Association et al., 2000) and the subthreshold definition requiring that criterion A, B, either C or D, and E and F be met (Grubaugh et al., 2005); 3. meeting DSM-IV-TR diagnostic criteria for lifetime or current alcohol or substance dependence, with alcohol or substance use in the last 90 days. Potential participants were excluded if they: 1. met diagnosis of psychotic, schizoaffective, bipolar, or severe major depressive disorders; 2. Were a current suicide or homicide risk; 3. were currently receiving PTSD-specific psychotherapy; 4. were currently involved in an abusive relationship; 5. had initiated, or changed, psychotropic medication(s) in the eight weeks prior to study entrance; 6. showed signs of gross organic mental syndrome; and 7. refused to be audio/videotaped.

Trial 2 only differed from Trial 1 in that participants must have met DSM-IV-TR criteria for current alcohol abuse or dependence with either 2+ heavy drinking days (i.e., 3+ drinks for women, 4+ drinks for men) in the past 90 days, 14+ drinks over 30 consecutive days, or less than 22 days abstinent in a row; or reporting 1+ alcohol misuse episodes (i.e., 7+ drinks for women in one week, 14+ drinks for men in one week; or drinking 4+ drinks in 2 hours for women or 5+ drinks in 2 hours for men) during the prior 90 days. Individuals with additional comorbid SUDs were allowed to participate. In addition to the exclusion criteria of Trial 1, Trial 2 also excluded individuals with physical or medical conditions that counter-indicated the use of sertraline: 1. having an advanced medical disease; 2. pregnancy or lactation; 2. seizure history (unrelated to alcohol use); 3. history of sertraline allergies; and 5. any other condition prohibiting antidepressant use.

Procedure

Randomization

Trial 1 participants were assigned to condition by an independent biostatistician using urn randomization with a stratification based on sex, baseline SUD severity, and PTSD severity. Research assessors were blind to condition. For Trial 2, participants were randomized using an urn randomization with a stratification based on baseline severity of alcohol/substance use and depression. Participants, psychiatrists, psychotherapists, and assessors were blind to condition, with an independent biostatistician directing psychiatrists on which apparently identical medical kits to provide to participants.

Assessment

The assessments described below were used in Trials 1 and 2. Any differences in this regard are noted.

Demographics

Age, education level, race/ethnicity, income, employment pattern in past three years (i.e., full-time, part-time, and other), marital status were collected during a baseline interview.

Psychiatric, Alcohol, and SUD Diagnoses

The SCID-IV-TR (First et al., 1995) was administered at baseline and follow-up to assess for the primary alcohol or substance-related diagnosis and additional comorbid SUD diagnoses, as well as the presence of any past or current comorbid psychiatric disorder. The SCID-IV-TR is a gold standard semi-structured interview of psychopathology with adequate to good interrater reliability for the SUD and major depression modules (kappa reliabilities are ~.7; (Lobbestael et al., 2011).

PTSD Diagnosis and Severity

Trauma exposure and type were measured at baseline via the Life Events Checklist (LEC; Weathers et al., 2013). The LEC presents 16 types of potentially traumatic events and asks participants to identify if they have directly experienced it, witnessed it, learned about it happening to someone else, or if they are not sure, or it does not apply. It has good convergent validity with similar measures (Gray et al., 2004)

The Clinician-Administered PTSD Scale (CAPS; Blake et al., 1995) was used at baseline and follow-ups to measure the presence of full or subthreshold PTSD diagnosis and provide a dimensional score of PTSD severity symptom severity in the previous 30 days. The CAPS was also used to ascertain the number of Criterion A traumatic events experienced by the participant and their age at time of occurrence. The CAPS total scores range from 0 to 136. Higher scores indicate greater severity. Clinical assessors received formal training in administering the CAPS. The CAPS has strong psychometric properties with good interrater reliability (i.e., kappa values of ~.85; [Pupo et al., 2011]).

The modified PTSD Symptom Scale Self-Report (MPSS-SR; Falsetti et al., 1993) was used to assess PTSD symptoms on a weekly basis during the active treatment period. The MPSS-SR is a 17-item self-report measure that asks respondents to rate the frequency and intensity of PTSD symptoms in the past seven days. Symptom severity is rated using a 5-point, Likert-type scale ranging from 0 (“not at all distressing”) to 4 (“extremely distressing”). Frequency of symptoms is rated using a 4-point, Likert-type scale ranging from 0 (“not at all”) to 3 (“five or more times”). MPSS-SR total scores range from 0 to 119. The MPSS-SR has strong convergent validity with other gold standard PTSD measures (Ruglass et al., 2014).

Alcohol and Substance Use

In Trial 1, the Addiction Severity Index-Lite (McLellan et al., 1980) was used at baseline to assess alcohol and substance use frequency in the past 30 days; participants provide the number of use days of their primary problem substance, of other substances, and more than one substance (poly-substance use). In Trial 2, the Time Line Follow Back (Robinson et al., 2014; Sobell et al., 1996) was used at baseline to assess alcohol and substance use patterns in the past 90 days. With the assistance of a calendar, participants estimate number of alcohol and other substance consumption days. To collect alcohol and substance use frequency during treatment in both Trial 1 and 2, the Substance Use Inventory (SUI; Weiss et al., 1995) was administered at weekly sessions and assessed the self-reported frequency of alcohol and substance use days in the past 7 days.

Emotion Regulation

In Trial 1, the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) was used at baseline to assess emotion dysregulation. The DERS is a 36-item self-report measure that asks participants to indicate the extent to which statements are true of them on a five-point scale from 1 (almost never/0-10% of the time) to 5 (almost always/91-100% of the time). DERS scores range from 36 to 180. Higher scores indicate greater difficulties with emotion regulation. It has strong psychometric properties and can, for example, distinguish between groups who are and are not characterized by emotion dysregulation (Kuo & Linehan, 2009). Emotion regulation was not assessed in Trial 2.

Interventions

Trial 1

COPE

(Back et al., 2014) is an integrated exposure-based PTSD+SUD psychotherapy that draws on both PE for PTSD (Foa et al., 2007) and RPT for SUD (Carroll, 1998; Marlatt & Donovan, 2005). COPE was administered via 12 weekly, 90-minute individual sessions. The first three sessions prioritize psychoeducation, goal setting, and other cognitive-behavioral strategies. Subsequently, imaginal and in vivo exposure to trauma memories and cues is introduced, with participants reviewing imaginal exposure audio recordings and completing in vivo exposures between sessions. RPT-informed interventions are integrated throughout these sessions.

RPT

(Carroll, 1998; Marlatt & Donovan, 2005) is an empirically-supported SUD intervention that promotes the use of coping strategies to mitigate relapse risk, and ameliorate its negative consequences if it occurs. RPT was administered via 12 weekly, 90-minute individual sessions and included psychoeducation, strategies for coping with cravings/urges, managing thoughts about substance use, problem solving, and role plays. The RPT condition does not involve overt discussions of trauma or PTSD.

Trial 2

Seeking Safety.

Seeking Safety (Najavits, 2002) is a skills-based treatment for individuals with PTSD+SUD that emphasizes the notion of safety using cognitive, behavioral, self-management, and interpersonal strategies. Each session focuses on a theme related to PTSD and SUDs and provides training in the acquisition of a cognitive-behavioral skill. Seeking Safety was delivered in an individual format over 12 weekly, 60-minute sessions.

Medication.

Participants received either sertraline or a placebo. Participants in the sertraline condition were started on 50 mg/day and increased dosage up to 200 mg/day over two weeks throughout the duration of the active study period.

Fidelity and Supervision

In both Trial 1 and 2, Masters’ or PhD-level clinicians administered the psychotherapies. Clinicians were trained in the study interventions by attending two- to three-day workshops in addition to receiving weekly supervision throughout the active study periods. In Trial 1, ~25% of sessions were randomly chosen for adherence coding from COPE intervention developers, with additional supervision provided for clinicians rated below adherence. In Trial 2, supervisors reviewed over 50% of sessions for curriculum adherence, with additional supervision provided for clinicians being rated below the competency criterion.

Attendance

Session attendance rates did not differ between interventions tested in Trial 1 (COPE: mean = 6.08, SD = 4.75 vs. RPT: mean = 7.21, SD = 4.40, p = 0.27) or Trial 2 ([Seeking Safety + Sertraline: M = 6.7, SD = 4.0; Seeking Safety + Placebo: M = 6.0, SD = 4.3; t(67) = 0.71; p = .48]. To examine the potential predictive relationships between within-treatment variables and session attendance, we restricted our analysis to participants who had attended at least one intervention session, and thus did not include individuals who did not return for treatment after being randomized. In Trial 1, this consisted of nine participants (five in COPE group and four in RPT group); in Trial 2, seven participants (one in Sertraline and 6 in Placebo group) never attended any treatment session. There was no significant difference in terms of the baseline CAPS score (COPE: beta=6.57, p-value=0.33; SS: beta=4.48, p-value=0.57) and the baseline PSS score (COPE: beta=5.91, p-value=0.69; SS: beta=−5.80, p-value=0.55) between these who attended at least one session and these who did not.

Statistical Analysis

Sample characteristics for participants who attended at least one treatment session in Trial 1 (n = 70) and Trial 2 (n = 60) are described in Tables 1a and 1b using frequencies for categorical variables and means with standard deviations for continuous variables. All analyses were done in R v.3.6.2.

Table 1a.

Trial 1 demographic and baseline clinical characteristics of participants who attended at least one treatment session (N = 70) by treatment group

	M(SD) or N(%)
Characteristic	COPE (n = 33)	RPT (n = 37)
Demographic
Age	42.39 (10.59)	44.62 (8.74)
Female	11 (33.3%)	11 (29.7%)
Race/Ethnicity
Black/African American	17 (51.5%)	25 (67.6%)
Latinx	8 (24.2%)	7 (18.9%)
White	6 (18.2%)	5 (13.5%)
Other	2 (6.1%)	0
Employment pattern (past 3 years)
Full time	12 (36.4%)	6 (16.2%)
Part time	14 (42.4%)	16 (43.2%)
Student/Unemployed/Disability	7 (21.2%)	15 (40.5%)
Education (years)	13.27 (1.83)	12.81 (2.36)
Criterion A trauma exposure
Type
Physical assault	19 (57.6%)	25 (67.6%)
Sexual assault	15 (4.5%)	12 (32.4%)
Accident or disaster	1 (3%)	5 (13.5%)
Sudden injury or death of other	10 (30.3%)	18 (48.6%)
Other	5 (15.2%)	2 (5.4%)
Multiple trauma	19 (57.6%)	29 (78.4%)
Age at first trauma	16.24 (12.72)	18.41 (14.28)
Time since last trauma (years)	16.15 (14.98)	12.68 (11.08)
CAPS severity, total	56.30 (16.81)	56.11 (20.96)
MPSS-SR	54.85 (25.05)	56.97 (25.13)
Alcohol and substance use
Alcohol Dependence	24 (72.7%)	30 (81.1%)
Substance Dependence	21 (63.6%)	27 (73%)
Alcohol and Substance Dependence	12 (36.4%)	22 (59.5%)
Primary substance
Alcohol	16 (48.5%)	15 (40.5%)
Cannabis	3 (9.1%)	2 (5.4%)
Cocaine	5 (15.2%)	5 (13.5%)
Alcohol and stimulants	6 (18.2%)	13 (35.1%)
Other polysubstance	3 (9.1%)	2 (5.4%)
Other diagnoses
Major Depressive Disorder	12 (36.4%)	12 (32.4%)
Anxiety (Panic, Phobia, or GAD)	13 (39.4%)	15 (40.5%)
DERS	89.42 (25.38)	93.22 (20.78)

Table 1b.

Trial 2 demographic and baseline clinical characteristics of participants who attended at least one treatment session (N = 60) by treatment group

Characteristic	Seeking Safety + Sertraline (n = 31)		Seeking Safety + Placebo (n = 29)
	n	%	n	%
Women	25	80.6	23	79.3
Race/ethnicity
Black/African American	16	51.6	18	62.1
White	9	29	5	17.2
Latina/o	3	9.7	4	13.8
Other	3	9.7	2	6.9
Employment
Full-time	12	38.7	17	58.6
Part-time	10	32.3	8	27.6
Student/retired/disabled	9	29	4	27.6
Alcohol Dependence	27	87.1	26	89.9
Drug Dependence
Cannabis	5	15.6	3	10.3
Cocaine	8	25.8	10	34.4
Comorbid AUD and SUD	15	48.4	17	58.6
Lifetime traumatic experiences
Physical assault	25	80.6	22	75.9
Sexual assault	21	67.7	21	72.4
Transportation accident	17	54.8	15	51.7
Current Major Depression	19	61.3	18	62.1
	M	SD	M	SD
Age (years)	42.65	9.56	41.79	8.97
Education (years)	13.88	3.08	13.25	2.06
Age at first trauma	18.9	12.86	15.13	11.02
Time since last trauma (years)	9.27	10.36	12.75	10.21
CAPS severity, total	64.8	19.67	58.69	19.68
MPSS-SR	98.93	26.69	94	21.49

Note. CAPS = Clinician Administered PTSD Scale; MPSS-SR = modified PTSD Symptom Scale Self-Report; GAD = Generalized Anxiety Disorder; DERS = Difficulties in Emotion Regulation Scale; AUD = alcohol use disorder (abuse or dependence); SUD = substance use disorder (abuse or dependence).

Slopes of Improvement During Treatment

Linear mixed models were used to estimate slopes of weekly improvement in weekly MPSS-SR (PTSD symptoms) and past seven days of primary problem substance use during the 12 weeks of treatment (Zandberg et al., 2016). These outcomes were predicted with a random intercept and a random slope and the estimated slopes for each participant were used as predictors and/or moderators of sessions attended. In this study, more positive slopes indicated faster improvement.

Candidate variables

Putative predictors of PTSD+SUD session attendance were identified through a multi-step review of the published literature on trial attrition. First, we conducted a literature review of studies which reported predictors and other relevant aspects of PTSD+SUD trial attrition (Belleau et al. 2017; Brady et al., 2001; Foa et al., 2013; Kline et al., 2021; Jarnecke et al., 2019; Hien et al., 2012: Myers et al., 2015; Pinto et al., 2011; Resko & Mendoza, 2012; Ruglass et al., 2012; Schacht et al., 2017; Szafranski et al., 2019; Zandberg et al., 2016). To compliment and confirm the relatively small pool of studies examining trial attrition in PTSD+SUD, we conducted a broader literature search of predictors of treatment attendance in PTSD-only and SUD-only trials including meta-analyses and systematic reviews of attrition rates in PTSD and SUD RCTs (Lewis et al., 2021; Schottenbauer et al, 2008; Adamson et al., 2009; Brorson et al., 2013; Lappan et al., 2019). We searched PubMed and PsycINFO using the following terms: (post-trauma* or posttrauma* or PTSD) and (treatment or therapy or intervention) and (dropout or engagement or retention or attrition or completion); ("substance use disorder" or "alcohol use disorder" or dependence or "substance abuse" or "alcohol abuse" or addiction) and (dropout or engagement or retention or attrition or completion). Attendance predictors and attendance data were extracted independently by two authors (TLC and SF). For ease of interpretation, one author (TLC) then organized predictor variables into higher order categories (e.g. ”trauma characteristics” and “substance type”). Table 2 presents variables that have demonstrated a significant association with attendance in either PTSD+SUD samples or in both SUD-only and PTSD-only samples. Table 2 also presents, when available, the analogous variable name in Trial 1 and Trial 2. In total, 28 candidate predictors with prior associations to trial attendance were available in Trial 1. A minority of the previously documented PTSD+SUD predictor variables were not available in the current study datasets (income, cognitive functioning, anxiety sensitivity, comorbid personality disorder, and therapeutic alliance).

Table 2.

Candidate Predictors, associations in prior PTSD+SUD, PTSD, and SUD samples, and Trial 1 & 2 analogues

	Prior support for association by sample type			Current study variable name
Variable	PTSD+SUD	PTSD	SUD	Trial 1	Trial 2
Age	●	●	●	Age	Same as Trial 1
Gender		●	●	Gender	Same as Trial 1
Race & ethnicity	●	●	●	Race	Same as Trial 1
Employment	●	●	●	Employment	Same as Trial 1
Education	●	●	●	Education	Same as Trial 1
Income	●	●	●	N/A	N/A
Cognitive functioning		●	●	N/A	N/A
Marital status	●		●	Marital	Same as Trial 1
Severity of substance use	●	●	●	Baseline past 30 days of problem substance; baseline past 7 days of problem substance	Same as Trial 1
SUD type	●	●	●	Problem substance; baseline SUD diagnosis (alcohol vs. substance)	Same as Trial 1
Comorbid SUD diagnosis		●	●	Baseline past 30 days of polysubstance use	Same as Trial 1
PTSD symptom severity	●	●	●	Baseline CAPS severity; baseline MPSSR severity; PTSD diagnosis (full vs. subthreshold)	Same as Trial 1
Trauma characteristics	●	●	●	More than one traumatic event; number of traumatic events; trauma before age 18; age of earliest trauma; sexual assault; physical assault; other trauma; accident	Same as Trial 1
Type of intervention	●	●	●	Intervention group (COPE vs RPT)	Intervention group (SS plus placebo vs. SS plus sertraline)
Anxiety sensitivity / perceived stress sensitivity	●		●	N/A	N/A
Baseline depression		●	●	Baseline MDD diagnosis	Same as Trial 1
Personality disorders		●	●	N/A	N/A
Distress tolerance / Emotion regulation	●		●	Baseline DERS	N/A
Therapeutic alliance	●		●	N/A	N/A
Within-treatment use	●	●	●	Past 7 day use of problem substance slope of improvement	Same as Trial 1
Within-treatment PTSD symptom change	●			MPSS-SR slope of improvement	Same as Trial 1

Note. SUD = substance use disorder; PTSD = Posttraumatic Stress Disorder; CAPS = Clinician Administered PTSD Scale; MPSS-SR = modified PTSD Symptom Scale Self-Report; COPE = Concurrent Treatment of PTSD and Substance Use Disorders Using Prolonged Exposure; RPT = Relapse Prevention Therapy; SS = Seeking Safety; MDD = Major Depressive Disorder. DERS = Difficulties in Emotion Regulation Scale.

Variable selection by Iterative Random Forest (iRF)

To explore which of the putative predictors were associated with treatment sessions attended, we used a two-step procedure. First, we used an iRF algorithm (Basu et al., 2018) to identify a smaller set of variables important for predicting sessions attended based on the variable importance score. RF is a nonparametric, regression-tree-based ensemble learning method for prediction and variable importance estimation (Breiman, 2001). RF has good model performance in simulation studies with sample sizes as small as 50 (Guo et al., 2010). iRF is an improved version of RF which not only stabilizes the variable importance score estimations, but also allows for detecting higher order interaction effects simultaneously. In this study, we used iRF to generate a variable importance score based on mean decrease in the Gini impurity index (GI) for each variable. Briefly, the GI uses the decrease of Gini index (impurity) after a node split as a measure of feature relevance, with larger decrease of impurity indicating more importance in predicting the outcome variable. In addition, iRF was also used to identify the most important interaction term that is predictive of the outcome measure.

The iRF algorithm involves selection of tuning parameters including the number of trees, the number of variables sampled at each node, the tree depth, the number of bootstrap samples, and the number of iterations. In our study, we used the default parameters in the R iRF package. We found that the variable importance estimates of iRF were fairly robust to these parameter choices.

Poisson Regression Model for Predicting Treatment Sessions Attended

Since treatment sessions attended is a count variable in nature, we used a Poisson regression model to assess and quantify the relationship between sessions attended and the top ten predictors, and the top interaction term identified by the iRF algorithm. The optimal number of predictors was determined by visual inspection of the variable importance plot, as there was a sharp decrease in the magnitude of importance for variables ranked 11^th and below (Figure 1). Tests were set at the significance level of 0.05.

Figure 1. Iterative Random Forest variable importance scores for candidate predictors.

Note. MPSS-SR = modified PTSD Symptom Scale Self-Report; CAPS = Clinician Administered PTSD Scale; DERS = Difficulties in Emotion Regulation Scale; MDD = Major Depressive Disorder; SUD = substance use disorder; PTSD = Posttraumatic Stress Disorder.

Validation

In the validation step, we were interested in validating 1) whether any relationships identified in the Poisson regression using data from Trial 1 remained statistically significant in Trial 2; and 2) whether the direction of the relationships remained the same.

Results

Potential Predictors of Sessions Attended in Trial 1

Results from the iRF variable importance scores for all predictors in our data are shown in Fig 1. Top ten candidate variables were considered as potentially important predictors of treatment sessions attended, as there was a clear reduction in variable importance scores for other variables. From Fig 1, the iRF algorithm indicated that age and slope of improvement in MPSS-SR during treatment were the top two most important predictors of sessions attended in Trial 1. Other variables potentially important in predicting sessions attended, by their order of importance, included years since last Criterion A event, baseline CAPS total severity, baseline MPSS-SR severity, age of earliest Criterion A event, slope of weekly improvement of past seven days primary substance use during treatment, employment status in the past three years, baseline DERS total, and baseline primary problem substance use days in past 30 days. In addition, the interaction between age and slope of weekly improvement in MPSS-SR during treatment had the highest importance score by iRF, suggesting that age moderated the relationship between slope of weekly improvement in MPSS-SR during treatment and sessions attended.

Predictive Role of Candidate Variables in Trial 1 Session Attendance

Poisson regression model revealed that the age by slope of weekly improvement in MPSS-SR during treatment interaction was significantly related to sessions attended (p = 0.005), suggesting a moderating role of age (Table 3). To facilitate interpretation of the age by slope of weekly improvement in MPSS-SR during treatment interaction effect, we plotted the relationships between slope of weekly improvement in MPSS-SR during treatment and sessions attended by mean age (i.e., 43.57 yrs), 1 standard deviation (SD) above (53.22 yrs), and 1SD below the mean age (i.e., 33.92 yrs), respectively. As shown in Figure 2, our model suggested that older participants with more weekly improvement in MPSS-SR during treatment tended to attend more treatment sessions, while younger people with more weekly improvement on MPSS-SR during treatment tended to attend less treatment sessions. In addition, past three years employment status (β = 0.213, p = 0.030) was positively associated with sessions attended. On average participants without employment in the past three years attended 1.237 more treatment sessions than those with either full-time or part-time jobs.

Table 3.

Poisson regression model of Trial 1 session attendance predictors

	Estimate	Standard error	p-value
Age	−0.0093	0.0088	0.23
MPSS-SR slope of improvement	−0.4535	0.1801	0.01
Years since last traumatic event	−0.0004	0.0042	0.92
Baseline CAPS severity	0.0019	0.0032	0.55
Baseline MPSS-SR severity	0.0025	0.0025	0.33
Age of earliest traumatic event	−0.0052	0.0042	0.22
Past 7 days of problem substance slope of improvement	−0.2361	0.2603	0.36
Employment	0.2240	0.0969	0.02
Age*MPSS-SR slope of improvement	0.0104	0.0037	0.01

Note. CAPS = Clinician Administered PTSD Scale; MPSS-SR = modified PTSD Symptom Scale Self-Report.

Figure 2. Trial 1 relationships between slope of weekly improvement in MPSS-SR during treatment and sessions attended by mean age, 1 standard deviation (SD) above, and 1SD below the mean age, respectively.

Note. MPSS-SR = modified PTSD Symptom Scale Self-Report

Validation of Trial 1 Predictors in Trial 2

In our validation analysis with Trial 2 data, we built a Poisson regression model (Table 4) using all selected predictors except baseline DERS and baseline primary substance use in the past 30 days, as these two measures were not collected in Trial 2. The direction and statistical significance of the interaction effect (p = 0.049) between participant age by weekly improvement in MPSS-SR during treatment were cross-validated in Trial 2. A similar relationship between the association between past three years of employment status and sessions attended was not observed (β = 0.014, p = 0.905) in Trial 2.

Table 4.

Poisson regression model of Trial 2 session attendance predictors

	Estimate	Standard error	p-value
Age	−0.0002	0.0074	0.98
MPSS-SR slope of improvement	−0.2219	0.1117	0.05
Years since last traumatic event	0.00214	0.0061	0.73
Baseline CAPS severity	−0.0031	0.0032	0.33
Baseline MPSS-SR severity	−0.0006	0.0024	0.78
Age of earliest traumatic event	−0.0001	0.0053	0.99
Past 7 days of problem substance slope of improvement	−0.1284	0.3787	0.73
Employment	0.0144	0.1211	0.91
Age*MPSS-SR slope of improvement	0.0050	0.0026	0.05

Note. CAPS = Clinician Administered PTSD Scale; MPSS-SR = modified PTSD Symptom Scale Self-Report.

Discussion

Achieving optimal attendance rates in PTSD+SUD treatment trials remains elusive and stands as one of the principal barriers to determining how best to deliver PTSD and SUD care. To advance the understanding of PTSD+SUD trial attendance by innovating the variable selection pipeline for predictor analyses, we applied iRF as a data-driven approach to selecting and evaluating predictors of treatment session attendance across two RCTs of distinct interventions. We incorporated the weekly tracking of PTSD symptoms and substance use during treatment, affording the most granular analysis to date on the impact of symptom change upon attendance across multiple approaches for PTSD+SUD. Our findings revealed that in an exposure-based RCT, participants’ age interacted with self-reported improvement in PTSD symptoms during treatment to predict session attendance; this interaction was replicated and confirmed as significant in a second RCT for combined behavioral therapy (skills-based) and pharmacotherapy. The only other significant predictor of attendance was employment status at study enrollment, and unlike the interaction between age and PTSD improvement, it was not predictive in the second trial.

These results demonstrate that a machine learning approach can help identify salient predictors of attendance in PTSD+SUD trials, regardless of whether the trial was exposure-based or more skills-based with pharmacotherapy. iRF indicated an age and PTSD symptom improvement interaction predicted session attendance. Subsequent regression analyses in the training data (Trial 1) showed the significance of this interaction effect. Then, we also validated this significant interaction effect in an independent data set (Trial 2). Taken together, with the given sample size, these results demonstrate that iRF identified a reproduceable interaction between two predictors related to session attendance with both treatment types (exposure- and skills-based). It is also important to note that several predictors top-ranked by iRF were not found to be significantly related to session attendance in the subsequent regression analyses. One potential reason for this could be that these variables are highly correlated with age, PTSD symptom improvement, and/or their interaction. Despite this limitation, our study suggests that iRF has the potential to identify important variables, allowing for the formulation of novel, testable hypotheses that cannot be easily derived from existing theory and literature.

This study is also the first to include the contribution of weekly-assessed PTSD and substance use symptom trajectories during treatment and to examine their predictive role in session attendance among individuals with a variety of problematic substance types and comorbid PTSD. These results highlight the importance of considering within-treatment PTSD and substance use symptom change to understanding attendance rates in PTSD+SUD trials. Moreover, because the training (Trial 1) and validation (Trial 2) samples differed in several ways (e.g., type of problematic substance, gender breakdown), the generalizability of study findings regarding age, PTSD symptom improvement, and session attendance are strengthened. The cross-validation of the iRF-identified interaction (between PTSD improvement and age) in Trial 2 suggests that within-treatment improvement is likely central to attendance regardless of the intervention type and should no longer be overlooked in questions of treatment retention and completion. The salience of within-treatment PTSD improvement is further supported by the current findings’ convergence with the only other published work on the predictive value of within-treatment symptom change in individuals with PTSD+SUD (Zandberg et al., 2016). Divergences from such works are also worthy of attention; whereas baseline PTSD severity moderated the effect of rate of PTSD change on attendance in Zandberg et al. (2016), only the baseline characteristic of age interacted with PTSD symptom change in our studies. Indeed, the inconsistency of baseline variable prediction was evident within the current study results themselves: being unemployed at study onset predicted more session attendance in Trial 1, but not in Trial 2.

It is possible that these discrepancies reflect the arc of the extant PTSD+SUD literature on trial attendance: significant associations are found between a single baseline demographic or clinical characteristic and trial attendance, are not often replicated, and are then interpreted as study-specific, non-generalizable events. Just as our study findings stress the relevance of dynamic changes in symptoms in response to trial interventions, they question the continued utility of mono-variable, static models to predict PTSD+SUD attendance patterns. With the application of a machine learning approach, we ascertained what predictive relationships were potentially generalizable, and which were not.

In both RCTs examined, younger individuals who showed faster improvement attended fewer PTSD+SUD treatment sessions, whereas older individuals who showed greater week-by-week improvement attended more sessions. This novel finding adds nuance to prior research showing that older age is associated with greater session attendance in PTSD+SUD intervention trials (Foa et al., 2013; Myers et al., 2015; Pinto et al., 2011). Older adults are likely to have more longstanding problems with PTSD+SUD and, thus, a potentially greater reservoir of treatment experiences than younger individuals. Older adults with PTSD+SUD may also be more aware of the waxing and waning, and at times intractable nature of PTSD+SUD, which may motivate them to commit to, and engage more fully with, PTSD+SUD interventions. By contrast, younger adults, whose development of PTSD+SUD may be more nascent, might underestimate PTSD+SUD severity, likelihood of relapse, or benefits of treatment. Consequently, they may be quicker to terminate interventions in response to rapid signs of improvement. This is not uncommon in SUD-only treatments (e.g., (McKellar et al., 2006). It would therefore be important to identify whether the gains that are associated with fewer sessions attended for younger adults, and more session attendance for older adults, are differentially maintained over extended periods following treatment. Indeed, it is possible that younger participants who leave trials after rapid PTSD improvement are more likely to experience a subsequent recurrence in PTSD or SUD symptoms, while older adults maintain their gains due to continued attention to PTSD+SUD even after experiencing improvements. Longitudinal examinations of trajectories of substance use patterns after PTSD+SUD treatment supports this possibility; López-Castro et al. (2015) found that being older, experiencing PTSD symptom improvement during treatment, and continuing to seek SUD care in the year following treatment distinguished membership from a high-risk and low-risk substance use class in women with PTSD+SUD.

Present findings carry implications for the potential refinement of known strategies to improve attendance in RCTs for PTSD+SUD. Although three imaginal exposure sessions (i.e. repeated, present-tense retelling of the traumatic memory) are recommended by its developers (Foa et al., 2007), the average PTSD+SUD trial participant attends only one such exposure session (Mills et al., 2012; Sannibale et al., 2013; Ruglass et al., 2017). Contingency management (CM) has been shown to improve session attendance and enhance outcomes in this specific population. In a RCT of an exposure-based intervention for comorbid opioid use disorder and PTSD, financially incentivizing attendance resulted in participants attending more exposure sessions (a median of six versus a median of 0), with significantly more reduction in PTSD symptoms achieved in the incentivized group compared to the non-incentivized (Schacht et al., 2017). Our results, which signal the risk (for some) of attrition with rapid improvement, may be helpful in guiding the personalizing of CM efforts—to either those who might most benefit from incentivization as well as when incentivizing may be most effective.

The interaction between PTSD symptom improvement and age predicting attendance further suggests that it may be particularly important to preemptively bolster attendance in younger patients. To this end, clinicians may consider highlighting the potentially waxing and waning course of PTSD, and thus recommending continued engagement in treatment despite symptom improvement to sustain gains, during orientation phases of treatment with younger patients. Clinicians are also advised to identify and troubleshoot barriers to attendance that may particularly affect younger patients (e.g., childcare demands, greater psychosocial instability, possibly lowered overall motivation due to less life experience/lower chronicity of symptoms) and inform their decision to discontinue treatment as soon as they perceive benefits from it. Alternatively, younger patients who show quick symptom improvement may not require additional treatment, and clinicians should not presume that more treatment is always better for these individuals. Decisions regarding treatment termination should be made based on individual case formulations, integrating key elements of treatment termination phases (e.g., relapse prevention planning, consolidation of gains). Younger patients who exhibit faster reductions in PTSD may be less likely to receive these elements, which may compromise the maintenance of gains over time. Clinicians are thus advised to collaborate with younger patients in particular to delineate signs of initial and sustained improvement early in treatment, so that the decision to discontinue treatment or adjust the treatment plan for flexible dosing (e.g., Galovski et al., 2012) can be made intentionally and with due process rather than haphazardly. Moreover, younger patients may benefit from learning about the importance of giving clinicians notice of termination plans so that key treatment termination elements can be integrated into final therapy sessions.

In addition to the use of gold-standard diagnostic instruments for assessing PTSD and SUD, the present study is strengthened by its ethnically diverse sample and range of SUD types that resemble the PTSD+SUD clients seeking care at community clinics. However, its limitations are important to note. Firstly, this study cannot shed light on early attrition—dropout that occurred prior to the onset of treatment—since we constrained our analyses to those individuals who attended at least one treatment session in order to model the predictive utility of within-treatment symptom change. Understanding the factors influencing whether individuals with PTSD+SUD initiate treatment after initial interest is a key area of research and deserves specific examination. Secondly, the present study is a secondary analysis and as such, is constrained by the sample sizes and measures of the pre-existing RCTs. An analysis of larger trials may have been able to detect other associations between candidate variables and session attendance. Furthermore, the RCT selected for cross-validation (Trial 2) employed all but two of the same candidate predictors as Trial 1. Thus, the relative importance of emotion dysregulation and baseline thirty-day count of problematic substance use in predicting attendance in Trial 2 remains unknown. In a similar vein, several constructs identified in the literature review as potentially important for predicting session attendance (income, cognitive functioning, anxiety sensitivity, comorbid personality disorder, and therapeutic alliance) could not be examined because they were not included in the current study’s datasets. To address these issues and move forward the study of RCT attendance, we encourage future researchers to 1) pool data from multiple PTSD+SUD trials with a variety of treatment types (e.g., exposure-based, skills-based, etc. to increase heterogeneity, external validity, and power, 2) employ data harmonization techniques to address measurement variability, and to then 3) incorporate machine learning methods such as iRF to facilitate data-driven inquiries about how multiple variables may interact to predict complex outcomes such as non-attendance or attrition. In the current study, the iRF variable importance score of intervention type did not produce a signal for attendance prediction in the training dataset (Trial 1, COPE vs. RPT). However, it is unknown whether this is the case in other PTSD+SUD intervention types. The recommended pooling and harmonization of multiple trial data will also afford researchers the opportunity to comprehensively examine the role of intervention type in predicting attendance patterns. Lastly, in this study we used iRF to identify potentially important predictors of treatment attendance. In general, iRF can be substituted by other machine learning approaches (e.g., xgboost used in Papini et al., 2018) to identify useful predictors. Future research is tasked with comparing which machine learning method is optimal in terms of identifying robust predictors of treatment attendance.

Conclusions

Directly targeting low attendance in RCTs for PTSD+SUD reflects a principal area of future work and our study findings can provide guidance for some potentially fruitful directions. Most importantly, study results underscore the value of continually assessing symptom improvement during treatment trials. The interactive role of symptom change in attendance suggests the benefit of better understanding each client’s unique web of treatment expectancies and motivations. This has implications for both clinical care and intervention research. Directly addressing and then helping clients with PTSD+SUD frame their progress—or lack thereof—within their broader treatment goals may solidify engagement, increase utilization, and resolve ambivalence about continued care. Explicitly linking attendance to PTSD symptom changes in early discussions regarding the maintenance of PTSD+SUD and its associated treatment rationale may bolster adherence into integrated PTSD+SUD protocols.

Inquiring, tracking, and responding to attendance-related beliefs and behaviors during RCTs may offer critical inroads to helping individuals make the most of integrated PTSD+SUD care. From a clinical research standpoint, our findings emphasize the necessity of collecting data specifically geared to answering adherence-related questions. Indeed, reasons for non-attendance are not often collected or examined systematically in efficacy trials; future intervention studies could embed questions regarding these reasons—such as polling participants for a host of motives for why they do or do not attend their sessions—and ideally, doing so in an ongoing way and prior to dropout. In particular, our findings highlight the interaction between distal and proximal factors. Such consistent data collection could inform future predictive models that are multi-layered with structural (e.g. transportation), interpersonal (e.g. family commitments), and intrapersonal (e.g. treatment expectancies) reasons. Additionally, because we study attendance in a randomized controlled trial setting, there is still a question as to how closely these analyses replicate the real-life experiences of PTSD+SUD treatment utilization. Further patient-centered qualitative studies would be warranted to examine such questions.

In conclusion, our study reflects the first application of a data-driven approach to the problem of detecting meaningful predictors of PTSD +SUD trial attendance in both exposure- and skills-based treatment studies. We view our findings as proof-of-concept that machine learning approaches such as iRF can be creatively used to help researcher’s derive novel and testable hypotheses, even in the context of relatively small sample sizes. The validation of the significant interaction effect of age and PTSD symptom in an independent data set suggests that the proposed approach could help generate meaningful and reliable knowledge in small sample size settings. As such, this study provides an initial template for future efforts that can further capitalize on ensemble learning approaches to examine within-treatment changes and model attendance patterns. Such research can produce critical insights for the clinical practice and implementation of PTSD+SUD treatments in the “real world”.

Public health significance:

This study suggests that attendance in treatments for co-occurring PTSD and substance use disorders may be affected by multiple factors, including how PTSD symptoms change during an intervention.

Appendix Table

Studies which employ the same datasets as the current submission

Study	Dataset	Variables examined	Status
1	Trial 1	Treatment outcomes (PTSD+SUD symptoms); attendance	Published
2	Trial 1	Baseline emotion dysregulation; treatment outcomes	Published
3	Trial 1	PTSD+SUD symptoms, multiple time-points (baseline; within-trial; follow-up periods)	Published
4	Trial 1	Trauma type; treatment outcomes	Published
5	Trial 1	Race/ethnicity; religious affiliation; treatment outcomes	Published
6	Trial 1	Criminal justice involvement; history of violent offending; treatment outcomes	Published
7	Trial 1 & Trial 2	Cannabis use at baseline; within-treatment cannabis use; treatment outcomes	Published
8	Trial 1	PTSD+SUD symptoms, multiple time-points (baseline; within-trial; follow-up periods)	Published
9	Trial 2	Treatment outcomes; attendance Published	Published
10	Trial 2	Race/ethnicity; sex; attendance; treatment outcomes	Published

Note. Studies #1, #9, and #10 reported on trial attendance. Studies #1 and #9 reported the average number of sessions attended in each condition and the results of tests of significance difference between conditions. Study #10 examined the relationship between participant race/ethnic group, trial attendance, and treatment outcomes.