Predictors of Urine Toxicology and Other Biologic Specimen Missingness in Randomized Trials of Substance Use Disorders

Abstract

Background:

High levels of missing outcome data for biologically confirmed substance use (BCSU) threaten the validity of substance use disorder (SUD) clinical trials. Underlying attributes of clinical trials could explain BCSU missingness and identify targets for improved trial design.

Methods:

We reviewed 21 clinical trials funded by the NIDA National Drug Abuse Treatment Clinical Trials Network (CTN) and published from 2005–2018 that examined pharmacologic and psychosocial interventions for SUD. We used configurational analysis—a Boolean algebra approach that identifies an attribute or combination of attributes predictive of an outcome—to identify trial design features and participant characteristics associated with high levels of BCSU missingness. Associations were identified by configuration complexity, consistency, coverage, and robustness. We limited results using a consistency threshold of 0.75 and summarized model fit using the product of consistency and coverage.

Results:

For trial design features, the final solution consisted of two pathways: psychosocial treatment as a trial intervention OR larger trial arm size (complexity=2, consistency=0.79, coverage=0.93, robustness score=0.71). For participant characteristics, the final solution consisted of two pathways: interventions targeting individuals with poly- or nonspecific substance use OR younger age (complexity=2, consistency=0.75, coverage=0.86, robustness score=1.00).

Conclusions:

Psychosocial treatments, larger trial arm size, interventions targeting individuals with poly- or nonspecific substance use, and younger age among trial participants were predictive of missing BCSU data in SUD clinical trials. Interventions to mitigate missing data that focus on these attributes may reduce threats to validity and improve utility of SUD clinical trials.

1. Introduction

Randomized trials designed to evaluate the efficacy and/or effectiveness of interventions to treat substance use disorders (SUDs) typically include biologically confirmed substance use (BCSU) specimens scheduled to be collected at fixed points in time after randomization. Depending on the substance of investigation, these specimens may include urine toxicology testing, hair sampling, and breathalyzer results. In SUD trials, like many other trials, high rates of missing outcome data are common (1–4). Missing outcome data threaten the validity of randomized trials because inference about treatment effects must then rely on untestable assumptions about the missing data mechanism (5–7).

To address missingness in BCSU outcomes, SUD trials have employed statistical techniques to mitigate potential biases, including single imputation (e.g., assume all missing BCSU results to be positive) (8, 9), multiple imputation (e.g., recorded values are used to predict missing values) (4, 10, 11), and other methods (4, 12, 13). These methods notwithstanding, a report from the National Research Council concluded that there is no universal way to adequately account for missing data (5, 6). Thus, optimizing participation and retention in SUD trials is the preferred approach to address data missingness.

Underlying attributes of clinical trials could explain missingness in BCSU outcome measures and identify targets for future interventions to minimize missing outcome data (14). For example, there is some evidence that trial design features, such as the type of intervention, compensation amount, or burden of assessments may impact missingness rates among individuals enrolling in clinical trials (15–24). Wakim, et al (16), used publicly available data from the first 24 trials of the National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) to examine associations between trial design features and the availability of the primary outcome measure(s) and found that trials involving medication-based interventions tended to have higher rates of outcome data missingness, potentially due to medication side effects.

While less studied than design features, trial participant characteristics, such as race/ethnicity (25), age (26–29), educational attainment (14), and income (25) may also impact rates of missingness. Additionally, the literature shows that addressing various participant-specific needs may improve trial participation and retention (18, 30). Because SUD clinical trials impact real-world practice (25, 31, 32), identifying and addressing underlying characteristics related to participation and retention in such trials may convey benefits that extend beyond the realm of clinical trial evaluation.

In this paper, we build on the work of Wakim, et al, to examine the association of underlying attributes in SUD trials on BCSU missingness in several important ways. First, we examine associations between biologically confirmed substance use missingness on both design features and participant characteristics. Second, we examine the degree of missingness (low vs. high) rather than a dichotomous outcome related to trial dropout. Third, we examine associations at the level of each trial treatment arm (rather than each trial) to study the relationship between design features and participant characteristics on the availability of BCSU results. Fourth, we employ configurational analysis (33, 34)—a Boolean algebra (i.e., and/or operations applied to true/false variables) approach that identifies minimally sufficient and necessary conditions for the occurrence of an outcome—rather than analysis of variance to examine these associations.

2. Methods

2.1. Study Design

This study comprised part of a research grant funded by NIDA to develop and implement new statistical methods for analyzing studies of randomized trials with repeatedly measured binary outcomes (e.g., abstinent or not abstinent from substance use) subject to non-monotone missing data patterns (35–55). Data were extracted from 21 CTN studies published from 2005–2018, representing 45 treatment trial arms (a single trial is comprised of multiple treatment arms). Analytic files were provided by The EMMES Company, LLC, who served as the CTN Clinical Coordinating Center. CTN trials were included based on the following criteria: (1) publicly available data at the time of grant submission; (2) randomized clinical trial; and (3) collected biological samples to detect substance use. The original set included 29 studies; nine studies were removed (CTN 0005, CTN 0011, CTN 0018, CTN 0019, CTN 0027, CTN 0028, CTN 0032, CTN 0049, AWARE) and one study was added (CTN 0013). Studies were removed if they had too few visits (two or less) where biological outcome measures were collected (i.e., 0005, 0011); if the intervention was not designed to directly impact substance use, such as HIV-related trials (i.e., CTN 0018, 0019, 0032, 0049, AWARE); or if attributes of interest were not collected/reported in the trial (i.e., CTN 0027, 0028). The final analytic set included 14 studies from the original analysis conducted by Wakim, et al. (published 2005–2013), and seven additional studies conducted after Wakim, et al. completed their analysis (published 2014–2018): CTN 0037, CTN 0044, CTN 0046, CTN 0047, CTN 0048, CTN 0051, CTN 0053.

The following trial design features were summarized for each treatment arm within each study: (1) number of assigned participants, (2) type of intervention (pharmacologic or psychosocial), (3) participant compensation for research visits, and (4) number of clinical sites. To account for burden of assessments, we used (5) a ratio of the number of total participant encounters during the trial (nearly all of which include a BCSU assessment) to the total trial length (in weeks). For example, consider two trials with 10 scheduled participants encounters: the first trial spans three months, while the second trial spans three weeks. The higher ratio of the second trial reflects the higher burden of assessments.

The following participant characteristics were also summarized: (1) type of primary substance use targeted (opioids, stimulants, polysubstance or nonspecific SUD, nicotine, cannabis), (2) average age, (3) percent female, (4) percent white, (5) percent Hispanic, (6) percent unemployed, and (7) average number of years of education. For our analysis, the outcome was average percent missing BCSU outcomes, where for each participant the percent missing was defined as one minus the number of documented BCSU outcomes divided by number of expected BCSU outcomes. Data were not consistently available on income and this variable (i.e., proxy for socioeconomic status) was not included.

Configurational analysis (33, 34) was used to identify determinants of outcome missingness, separately for trial design features and participant characteristics. This analytic technique, in its most common form, requires all included attributes and outcomes to be categorical. Towards this end, we used medians to create high/low categorizations for most attributes (Table 1). We categorized education using a high school/GED threshold of years (<12 vs. ≥12); substance use as poly vs. non-poly; and high level of missingness as >25%. Our threshold for missingness was determined through expert consensus among the authors prior to conducting the analysis and fell within a range of thresholds described in the methodological literature (56–61).

Table 1.

Categorization thresholds for attributes of interest.

Attribute	Categorization*
Design Features
Number of assigned participants	>211 vs ≤211
Type of intervention	Pharmacologic vs. Psychosocial
Compensation	>$240 vs. ≤$240
Number of clinics	>9 vs. ≤9
Number of clinical visits per trial week	>0.77 vs. ≤0.77
Participant Characteristics
Substance use targeted	Poly vs. Non-Poly
Mean age	>36 vs. ≤36
Percent female	>34 vs. ≤34
Percent white	>52 vs. ≤52
Percent Hispanic	>13 vs. ≤13
Percent unemployed	>26 vs. ≤26
Mean years of education	<12 vs. ≥12

^*Each numerical threshold based on the median value for each respective attribute across all included trials, except for years of education (which is based on years to complete high school/GED).

2.2. Configurational Analysis

Configurational analysis is a formal mathematical approach that draws upon Boolean algebra, set theory, and logic to identify minimally sufficient (i.e., redundancy-free) conditions for the occurrence of the outcome (33, 34). Such minimally sufficient conditions are then combined to yield “solutions” consisting of one or multiple “pathways”, such as a single attribute (e.g., lower average age) or a conjunction of attributes (e.g., lower average age AND lower average number of years of education), which jointly are minimally necessary for the outcome.

For example, given multiple possible implementation strategies available to a hospital, a specific strategy may yield a desired outcome (e.g., high vaccination uptake or shorter hospitalization times) alone or in combination with other strategies, whereas other strategy combinations may contribute to undesirable outcomes and still other strategy combinations may also yield the desired outcome (62). In configurational analysis, contribution of multiple strategies in combination to achieve the desired outcome is called “conjunctivity,” while the ability of achieving the desired outcome through different, separate combinations is referred to as “disjunctivity.”

Discovering contributing structures exhibiting conjunctivity and disjunctivity calls for methods that embed individual factors in complex Boolean AND- and OR-functions. The problem, however, is that the space of possible Boolean functions over even a handful of factors is vast. For n binary factors there exist 2²ⁿ possible Boolean functions, and if factors contain more than two binary values, the number grows more exponentially. That means methods capable of discovering contributing structures with conjunctivity and disjunctivity must find ways to efficiently navigate in the vast space of possibilities. That is the purpose of methods for configurational analysis.

In configurational analysis for this study, each trial arm is considered to be a “case”. The rationale for using configurational analysis was two-fold: a) the sample was too small to use regression and b) there were strong a priori reasons to expect that attributes contributing to outcome missingness in substance disorder trials might differ by trial design feature or participant characteristic categories (14–29). Across both sets of attributes, there are 2¹² ≈ 4,096 possible configurations—too many configurations relative to the low number of cases, which would limit interpretability of results. To address this issue, we conducted separate configurational analyses for trial design features (32 possible configurations) and participant characteristics (128 possible configurations). Configurational analyses were conducted using the “cna” package (63) in R Version 4.3.1.

For each set of attributes (design and participant), we executed the configurational analysis in two steps. First, we conducted an exploratory data analysis to inform factor selection. Specifically, we used the msc (i.e., “minimally sufficient conditions”) function within the “cna” package to identify pairs of attributes that were highly dependent. For any pair of highly dependent attributes identified, distinguishing the relative impact of each attribute would be difficult; thus, for highly dependent attributes, one element of the pair was dropped based on theoretical considerations. In the second step, all one-, two- and three-factor configurations of the remaining attributes that met a preset consistency threshold of 75% were identified. In configurational analysis, consistency of a given configuration indicates how reliably it yields the outcome; it is calculated as the number of cases within a configuration that have the specified outcome divided by the total number of cases with the same configuration (akin to a positive predictive value). Coverage indicates how well a configuration accounts for an outcome and is calculated as the number of cases of a specified outcome within a configuration divided by the total number of cases with the same outcome (akin to sensitivity). Robustness measures the degree (on a scale of 0 to 1) to which a model overlaps with other models resulting from re-analyses of the data at systematically varied consistency and coverage thresholds (64).

Among configurations meeting our consistency threshold in each analysis, we then identified a single solution with the best fit (in terms of product of consistency and coverage) and high level of robustness. We prioritized configurations that (a) aligned with theory, logic, and background knowledge and (b) had low complexity (i.e., minimum number of solution pathways) to avoid overfitting.

3. Results

Summary information for the 45 trial arms included across the 21 CTN studies is shown in Table 2. All trials had two or three arms with arm size ranging from 72 to 431 participants. Pharmacologic interventions comprised 17 (38%) of all trial arms, with the remainder comprised of psychosocial interventions; trial arms for interventions targeting individuals with poly- or nonspecific substance use comprised 22 (49%) of all trial arms, with the remainder targeting stimulant use (29%), opioid use (18%), nicotine use (9%), or cannabis use (4%).

Table 2.

Summary of trial attributes.

CTN STUDY	Treatment Arm	Design Features						Participant Characteristics							Percent Missing (Number of Assessment*)
		Arm Size	Type of Intervention	Number of Clinic Visits	Trial Length (Weeks)	Compensation	Number of Clinical Sites	Substance Use Targeted	Mean Age	% Female	% White	% Hispanic	% Unemployed	Mean Education (Years)
0002	A	74	Pharmacologic	18	26	$125	6	Opioid	40	31	41	19	36	≥12	59 (8)
	B	156	Pharmacologic	18	26	$125	6	Opioid	38	27	40	22	45	≥12	35 (8)
0003	A	255	Pharmacologic	17	19	$210	11	Opioid	36	33	73	7	11	≥12	42 (3)
	B	261	Pharmacologic	17	22	$210	11	Opioid	36	33	69	11	15	≥12	41(3)
0004	A	245	Psychosocial	8	17	$70	5	Poly	36	28	39	14	13	≥12	34 (8)
	B	216	Psychosocial	8	17	$70	5	Poly	34	30	45	11	8	≥12	36 (8)
0006	A	222	Psychosocial	27	26	$400	8	Stimulant	36	54	40	14	19	<12	62 (13)
	B	252	Psychosocial	27	26	$400	8	Stimulant	36	55	32	19	19	<12	48 (13)
0007	A	198	Psychosocial	27	26	$400	6	Poly	42	49	25	16	35	<12	34 (14)
	B	204	Psychosocial	27	26	$400	6	Poly	42	39	25	19	27	<12	30 (14)
0009	A	72	Psychosocial	13	29	$210	7	Nicotine	42	47	39	31	39	<12	14 (12)
	B	153	Pharmacologic	13	29	$210	7	Nicotine	42	48	35	35	38	<12	22 (12)
0013	A	98	Psychosocial	8	20	$225	4	Poly	25	100	43	17	27	<12	17 (7)
	B	102	Psychosocial	8	20	$225	4	Poly	27	100	31	25	25	<12	25 (7)
0014	A	235	Psychosocial	13	52	$160	8	Poly	16	23	33	45	0	<12	35 (13)
	B	237	Psychosocial	13	52	$160	8	Poly	16	20	35	43	0	<12	34 (13)
0015	A	177	Psychosocial	11	54	$240	7	Poly	39	100	49	9	42	≥12	39 (11)
	B	176	Psychosocial	11	54	$240	7	Poly	39	100	48	4	40	≥12	39 (11)
0017	A	211	Psychosocial	5	24	$170	8	Poly	36	27	69	11	28	≥12	30 (5)
	B	212	Psychosocial	5	24	$170	8	Poly	36	23	73	8	28	<12	30 (5)
	C	209	Psychosocial	5	24	$170	8	Poly	37	23	64	9	27	<12	30 (5)
0021	A	222	Psychosocial	8	16	$70	5	Poly	32	11	0	100	0	<12	33 (7)
	B	214	Psychosocial	8	16	$70	5	Poly	33	11	0	100	0	<12	32 (7)
0029	A	128	Pharmacologic	13	15	$500	6	Nicotine	37	47	78	6	2	≥12	17 (13)
	B	127	Pharmacologic	13	15	$500	6	Nicotine	38	40	83	8	0	≥12	15 (13)
0030	A	180	Pharmacologic	10	12	$275	10	Opioid	33	44	91	5	17	≥12	12 (10)
	B	180	Pharmacologic	10	12	$275	10	Opioid	32	40	93	5	11	≥12	13 (10)
0031	A	237	Psychosocial	11	26	$200	10	Stimulant	38	56	55	6	35	≥12	19 (5)
	B	234	Psychosocial	11	26	$200	10	Stimulant	38	62	52	6	35	≥12	24 (5)
0037	A	150	Psychosocial	37	37	$1440	9	Stimulant	39	39	44	8	25	≥12	30 (24)
	B	152	Psychosocial	37	37	$1440	9	Stimulant	38	41	55	13	36	≥12	36 (24)
0044	A	252	Psychosocial	27	26	$470	10	Poly	35	40	65	12	23	<12	39 (24)
	B	255	Psychosocial	27	26	$470	10	Poly	36	36	60	10	29	≥12	32 (24)
0046	A	271	Psychosocial	23	28	$580	12	Poly	37	50	58	12	39	<12	10 (12)
	B	267	Pharmacologic	23	28	$580	12	Poly	37	46	61	13	39	<12	11 (12)
0047	A	431	Psychosocial	4	52	$275	6	Poly	36	28	56	25	26	≥12	49 (4)
	B	427	Psychosocial	4	52	$275	6	Poly	36	33	54	23	26	≥12	49 (4)
	C	427	Psychosocial	4	52	$275	6	Poly	37	30	53	23	19	≥12	50 (4)
0048	A	102	Pharmacologic	28	20	$765	11	Stimulant	47	23	27	11	37	≥12	16 (11)
	B	100	Pharmacologic	28	20	$765	11	Stimulant	47	20	26	12	37	≥12	19 (11)
	C	100	Pharmacologic	28	20	$765	11	Stimulant	46	22	25	8	37	≥12	19 (11)
0051	A	287	Pharmacologic	30	36	$710	8	Opioid	34	28	79	19	27	≥12	40 (11)
	B	283	Pharmacologic	30	36	$710	8	Opioid	34	31	77	16	26	≥12	54 (11)
0053	A	149	Pharmacologic	26	17	$1210	6	Cannabis	31	34	65	23	22	≥12	22 (14)
	B	153	Pharmacologic	26	17	$1210	6	Cannabis	30	24	62	20	38	≥12	21 (14)

^*Number of BCSU outcome assessments in the analytic files. Polysubstance use refers to either a specific combination of substance use disorders targeted (CTN 0007: stimulant + opioid; CTN 0017: injectable substances; CTN 0044: illicit +/− non-illicit; CTN 0046: nicotine + stimulants) or non-specific combination of substance use disorders (CTN 0004, CTN 0013, CTN 0014, CTN 0015, CTN 0021, CTN 0047).

3.1. Design Features

In the configurational analysis for trial design features, compensation and study burden ratio were highly dependent. We therefore removed compensation from the analysis given prior evidence that study burden is a relatively stronger driver of missingness (22–24, 65). Among the 16 possible configurations across the remaining four design feature variables, 13 were observed in the analysis. As is often the case in configurational analysis, there were multiple solutions that fit the data equally well (66), meaning that the evidence in the data alone did not determine one single model and that the ultimate model choice had to be made by finding an optimal balance between solution scores and available background knowledge. The three most robust solution candidates (i.e., highest robustness scores) are depicted in Table 3. In all of these solutions, psychosocial treatment as an intervention was a factor associated with high outcome missingness. The final solution selected—a two-pathway solution consisting of psychosocial treatment as an intervention OR trial arm size >211—was based on its best fit score (0.73), relatively low complexity (two-pathway solution), and high robustness score (0.71) (Figure 1).

Table 3.

Results of configurational analysis related to high levels of missingness in biologically confirmed substance use in CTN clinical trials (consistency threshold=0.75). Highlighted solutions represent the final solution for each analysis.

Solution	Complexity	Fit			Robustness
		Consistency	Coverage	Product	Norm Score
Design Features
Intervention (psychosocial)	1	0.79	0.79	0.62	1.00
Intervention (psychosocial) OR Number of assigned participants (>211)	2	0.79	0.93	0.73	0.71
Number of assigned participants (>211) OR  [Intervention (psychosocial) AND Clinic Visits/Week (>0.77)]	3	0.85	0.79	0.67	0.46
Participant Characteristics
Substance use targeted (poly) OR Mean age (≤36)	2	0.75	0.86	0.65	1.00
Substance use targeted (poly) OR [Percent unemployed (>26) AND Percent white (>52)]	3	0.85	0.82	0.70	0.87
Substance use targeted (poly) OR [Percent unemployed (>26) AND Percent female (>34)]	3	0.82	0.82	0.67	0.66

Figure 1.

Graphical representation of alternate pathways in the final solutions for each configurational analysis: (A) trial design features, and (B) trial participant characteristics.

3.2. Participant Characteristics

None of the seven participant characteristics were dependent. Among 128 possible configurations of participant characteristic variables, 27 configurations were observed in the analysis. Of these, the three most robust solution candidates all comprised two pathways with either single attributes or conjunctions of attributes (Table 3). Moreover, they all featured interventions targeting individuals with poly- and/or nonspecific substance use as one pathway leading to high BCSU missingness. The other pathway was younger age in the first candidate solution, higher percentage unemployment and white race in the second, and the conjunction of higher percentage unemployment AND female sex in the third. We selected the first candidate as our final solution—a two-pathway solution consisting of interventions targeting poly- or nonspecific substance use OR younger age—because it had the lowest complexity (2), highest robustness score=1.00, and only a marginally lower fit than the other two solutions (Figure 1).

4. Discussion

This study of CTN SUD clinical trials examines the contribution of trial design features and participant characteristics with BCSU missingness in a novel way. As the overall number of completed trials is relatively small, traditional statistical approaches have limited ability to detect associations in underlying trial attributes. Configurational analysis is an alternative approach to identify attributes or combinations of attributes that contribute to BCSU missingness.

We found that among trial design features, psychosocial treatment interventions and larger trial arm size contributed to a high degree of BCSU missingness. By comparison, Wakim, et al., whose analysis of variance included 14 of the same 21 clinical trials included here, found a modest association between BCSU missingness and pharmacologic treatment but not psychosocial treatment (16). This difference in findings is likely attributable to both the addition of new clinical trials and the utility of configurational analysis to identify multiple pathways that contribute to an outcome for a relatively small sample and relatively large number of variables (33).

While psychosocial treatment interventions are known to address psychosocial problems that negatively impact adherence to pharmacologic treatments (67), their evidence for improving SUD-related treatment outcomes, including treatment retention, is strongest when used in combination with pharmacologic therapy (68–71). Treatment retention and BCSU missingness are closely related, with the latter representing a more useful measure in clinical trials, since some participants “retained” in a trial may have numerous instances of missing BCSU data throughout the trial period. However, because treatment retention and missing BCSU data are related—with some studies even defining retention as the capture of BCSU data (16)—absence of pharmacologic therapy could explain higher observed levels of BCSU missingness in trial arms that included only a psychosocial treatment.

To mitigate the higher expected burden of BCSU missingness and threat to trial validity in SUD randomized trials that study psychosocial treatment interventions, unique strategies may be helpful. For example, Rioux and colleagues recommend designing clinical trials to minimize missing data bias with planned missing data to curtail the more problematic nonrandom missingness (72). In this model, trial resources are concentrated on a subset of the broader trial population who is selected at random for data collection. Other potential strategies include participant education, adherence assessments (e.g., learn participant’s explanation for nonadherence), involving significant others in supporting trial participant adherence (73), and other strategic outreach to trial participants who have missed their visits (30). Various models of patient-community-based research could also incorporate the perspectives of people with SUD to design studies that better meet their preferences and needs.

In our analysis, treatment arm size represents another important parameter in CTN SUD trials that could influence participant engagement. Larger multi-site trials often present additional logistical challenges and resource constraints that impact participant retention and could impact outcome data missingness (74). Unfortunately, adverse effects of missing data on statistical inference can increase as sample size increases (75). Better recognition of and attention to threats to participant engagement in larger trials might improve data quality, especially when larger participant numbers are needed to achieve statistical power and/or results that are generalizable to a broader number of demographic subgroups.

Surprisingly, assessment burden did not contribute to BCSU missingness in the final solution. The literature is replete with evidence of perceived participation burden among clinical trial participants (76, 77). Measuring participant burden and burden-to-compensation ratios to ensure optimal conditions for trial participation have already been proposed (78, 79) and may have an important role for disproportionately marginalized participants who are enrolled in SUD clinical trials. Further study to reconcile our findings with the perceived importance of addressing assessment burden in the literature is warranted.

Similar to trial design features, the final solution for participant characteristics comprised multiple solution pathways: interventions targeting individuals with poly- and/or nonspecific substance use or younger age of trial participants. The contribution of polysubstance use to missing BCSU data is consistent with other studies, which have shown polysubstance use predicts low levels of treatment engagement (80), high rates of nonadherence for treatments such as antimicrobial therapy following hospitalization (81), and lower baseline rates of abstinence and treatment participation than those with single SUDs such as opioid use disorder or alcohol use disorder (82, 83). Taken together, low levels of abstinence and engagement in treatment may discourage participants with polysubstance use history from completing visits and BCSU testing in clinical trials.

Missing data from low levels of participant engagement among this population may be driven by an underlying feature of BCSU testing described by Ren and colleagues: SUD clinical trial participants have foreknowledge about their anticipated results when completing BCSU testing, which can influence their decision whether to appear or skip a study visit (58). Strategies that target participant education and perception of BCSU testing regardless of abstinence or treatment engagement (or trial interventions that may seem unassociated with biological processes) on how trial data improves SUD care may be helpful. Moreover, most trials keep BCSU data confidential and do not share that data with their standard treatment providers, where there may be consequences for continued or resumed use. Additional education may help patients understand this distinction between trial participation and standard treatment.

There is also evidence that psychosocial instability plays a role in BCSU missingness. Prior work has shown that emerging adults ages 18–25 have higher levels of substance use and lower levels of SUD treatment retention than older adults (84, 85), which is thought to be due to multiple psychosocial factors, including less developed executive functioning and relative instability in social relationships and employment (86, 87). In our study, we found younger age (trial arm mean age ranging 16–36 years) contributed to missing BCSU data and may therefore be a marker of psychosocial instability. While this age range differs from prior studies of emerging adults, recent studies suggest neurocognitive development does not always finish by age 25 and may continue to age 30 or later.

There may be ways to address BCSU missingness among younger adults. In one study, key informant interviews of individuals with active or recent substance use and psychosocial instability found that certain interventions, such as peer recovery coaches and use of behavioral activation techniques, may engage patients in care (88). Future studies could examine the utility of such interventions for individuals with psychosocial instability enrolled in clinical trials—especially those targeting polysubstance use—to mitigate missingness in BCSU data.

4.1. Limitations

Our study has limitations. First, while we assessed trial design features and participant characteristics available in the trial data, other characteristics, such as severity of illness among participants or duration of individual visits, might reasonably predict BCSU missingness but are not readily available in comparable form across SUD trials. Efforts to standardize data in CTN SUD trials could enable more robust analyses. Second, the number of available CTN SUD trials was limited. While the use of configurational analysis conveys significant advantages for analysis of a small sample, it does not eliminate limitations. For example, in our approach we can readily draw conclusions between studies involving polysubstance use and a single substance, however, the number of trials available to assess differences between studies of single substances (e.g., nicotine vs. opioid) is insufficient. Over time, as more trials are completed, these differences may be assessed more readily, including participant-level analyses of specific characteristics. Because of the small sample relative to the number of attributes we considered, we were unable to look at all attributes simultaneously. Third, in rare instances the intervention comprised both psychosocial and pharmacologic treatment (e.g., CTN 0030). In these cases, we selected the treatment modality we felt most relevant to the study. Fourth, while we used the best information available to make modeling decisions, limitations of available evidence could potentially introduce bias into those decisions.

5. Conclusion

Missing outcome data in SUD clinical trials threaten trial validity. Because optimizing participation and retention in SUD trials is the preferred approach to address data missingness, an understanding of underlying attributes that contribute to missing outcome data is key. Our analysis of trial design features and participant characteristics identified trials involving psychosocial treatment as an intervention, larger trial arm size, interventions targeting individuals with poly- or nonspecific substance use, and younger age among trial participants as attributes predictive of high levels of missing outcome data. Psychosocial instability may also play a role. Further study and interventions to mitigate data missingness in trials with these attributes could reduce threats to validity, improve the utility of SUD clinical trials, and ultimately improve patient health outcomes.

Highlights.

• Biologically confirmed substance use data is often missing in SUD clinical trials.

• Certain attributes of trials and their participants could explain BCSU missingness.

• In this study, we used configurational analysis to identify associated attributes.

• Psychosocial treatment, arm size, polysubstance use, and young age were predictive.

• Interventions to mitigate data missingness should focus on these attributes.