Prospective cohort study of fentanyl test strip use and distribution in three states: the stay safe study protocol and implementation recommendations for researchers

Abstract

Background

The Stay Safe Study is the first observational prospective cohort study investigating fentanyl test strip (FTS) use by people who use drugs (PWUD) and distribution by community organizations that provide harm reduction services (e.g., naloxone distribution) in three states (Kentucky, New York, and Ohio). The purpose of this paper is to describe the study design, along with implementation successes and challenges. A related goal is to provide recommendations and encourage researchers to undertake multi-state and multilevel studies of FTS use and distribution.

Methods

The Stay Safe Study has one primary, five secondary, and three exploratory objectives. From May-December 2023, we collected survey, interview, and oral fluid drug test data with the primary population of PWUD attending service locations of partner organizations that distribute FTS. We collected survey and interview data with a population of community organizations providing harm reduction services that have a distribution relationship to FTS. There was no intervention, and the study did not distribute FTS.

Results

A total of 1,156 PWUD participants were enrolled in the study and were invited to complete four weekly survey assessments. Of these, 732 PWUD participants (97.6% of the target) completed at least two of the four weekly surveys and reported drug use at least once during the 28-day observation period. A subset of enrolled participants completed a one-time oral fluid data collection visit (48-hour self-report survey and oral fluid specimen collection) (n = 267) and one-on-one semi-structured interviews (n = 120). From the population of 36 organizations providing harm reduction services, employees of 28 (78%) in 22 counties across 3 states completed a web-based survey and employees of 24 (66.7%) completed an interview. Study findings will be presented in subsequent publications [1, 2].

Conclusion

The research team successfully enrolled a large sample of PWUD participants and organizations that provide harm reduction services in three states, followed a cohort of PWUD participants, and generated rich data using three types of instruments. Factors contributing to implementation success include drawing on the community expertise of the parent HEALing Communities Study, selecting incentive amounts that acknowledged the time and expertise of PWUD participants, centralizing survey data collection, and tracking implementation challenges and solutions. Areas of implementation complexity included conducting research on site at the service locations of 14 partner organizations, facilitating timely incentive payments, overseeing research staff, and returning oral fluid results to PWUD participants.

Background

The ongoing overdose crisis in the United States (US) has exacted an increasingly heavy toll on public health. Since 1999, more than 1.1 million people have died from a drug overdose in the US [3]. Drug overdose deaths involving synthetic opioids, excluding methadone, particularly illicitly manufactured fentanyl, increased from 57,834 in 2021 [4] to 74,702 in 2023 [5]. Overdose deaths from fentanyl are a serious public health threat because fentanyl is present in illicit opioids, stimulants, crushed pills, recreational substances such as MDMA (also known as Ecstasy), and counterfeit medications [4, 6–8].

The prevalence of fentanyl in the drug supply has varied regionally, over time, and with specific substances. By 2023, fentanyl was detected in nearly half of heroin samples seized by the US Drug Enforcement Administration (DEA) nationwide [9]. In some states in the US Northeast plus Kentucky and Ohio, the co-occurrence of fentanyl with cocaine and methamphetamine samples has exceeded the national average [9]. Since 2018, the proportion of samples with fentanyl seized by the US DEA has increased in Kentucky [KY], New York [NY], and Ohio [OH]), among other states [9]. As well, the proportion of heroin samples with co-occurring fentanyl has increased in those three states during this time [9].

Fentanyl test strips (FTS) are a low-barrier drug-checking tool that can detect multiple fentanyl analogs in a range of substances at the point of consumption [10–14]. While investigation of FTS practices has grown rapidly since 2016 [15], evidence derives primarily from single-instrument feasibility and acceptability studies using qualitative or quantitative designs and small sample sizes (n = 11–93) of people who use drugs (PWUD) [10–13, 16–23], which limit generalizability. Most FTS studies have been conducted at a single site, typically a syringe service program or other harm reduction service provision sites. At the time of this writing and confirmed by a recent scoping review [15], just four out of 26 FTS studies have sample sizes of PWUD > 100 (n = 105–426) [10, 14, 24], and all but two [14, 24] were conducted in a single state or province. Two studies tested an FTS intervention [22, 25], and another FTS study used a urinalysis test to investigate concordance of self-reported fentanyl consumption with biomarker results [26].

Findings are mixed on whether PWUD who obtain a positive FTS result modify their drug use behaviors [10, 11], suggesting the need to look more closely at the impact of FTS use, risk perception, and behavior change [27]. A single study using a cross-sectional, mixed-methods design compared substance use behaviors of PWUD who do and do not use FTS [16]. Researchers leading this study found that FTS use by PWUD is associated with both safer behaviors (e.g., using less drug supply than normal) and riskier behaviors (e.g., using more drug supply than normal). Significant gaps remain in understanding whether use of FTS by PWUD is associated with a reduction in overdose risk [14, 21, 22, 25]. There is limited evidence on community-based organizations and FTS: their distribution strategies, the reach of their FTS programs to populations at risk of fentanyl-related overdose, and their educational strategies with PWUD for correct FTS use and overdose prevention. Understanding FTS use and distribution with a multi-state study is critical for informing intervention strategies and to optimize community-based distribution of FTS as an overdose prevention strategy [28].

Due to increasing evidence of the acceptability and feasibility of using FTS, drug paraphernalia statutes that have criminalized distribution and/or possession of FTS have been overturned in all but four states [29]. These changes may directly implicate the FTS procurement and distribution strategies of organizations that provide harm reduction services, which may in turn affect access to FTS by PWUD [29]. The three states have had a disparate legal landscape, with statutes that have variously criminalized FTS distribution and/or possession (i.e., these activities are subject to prosecution), decriminalized these activities (i.e., these activities are illegal but not subject to prosecution), or legalized these activities (i.e., removed all legal prohibitions to these activities) (Fig. 1). Just prior to the start of data collection, OH and KY decriminalized FTS distribution and possession, which brought the two states into alignment with NY. Just before the end of data collection, NY legalized FTS distribution by healthcare providers, a change not seen in KY or OH.

Fig. 1.

Decriminalization and legalization of fentanyl test strips in Kentucky, New York, and Ohio during Stay Safe Study implementation

In this paper, we present the study protocol for the Stay Safe Study. The Stay Safe Study is the first prospective cohort study of FTS use and distribution conducted in three states (KY, NY, and OH). The primary goal of this paper is to describe the study design and implementation successes and challenges of this complex study design. We also provide recommendations to researchers [30] for developing more robust studies of the effectiveness and distribution of drug-checking tools such as FTS [28].

Methods

Study structure and development

The Stay Safe Study is an ancillary study of the HEALing Communities Study (HCS), a community-level, two-arm, cluster randomized clinical trial evaluating the effectiveness of the Communities That HEAL intervention on reducing opioid-related overdose fatalities in 67 communities in four states [31]. The Stay Safe Study cross-state research team consists of members from three state teams, a data coordinating center (DCC) led by RTI International, and scientists with the National Institute on Drug Abuse (NIDA) and the Substance Abuse and Mental Health Services Administration (SAMHSA). The DCC facilitated study meetings, coordinated central institutional review board (IRB) submission, programmed surveys, managed data collection oversight (e.g., triaged data collection problems in real-time), hosted survey data collection on its portal, conducted quantitative data analysis, and coordinated the study-wide qualitative data analysis team.

The study received IRB approval from Advarra (Pro00068304) on March 16, 2023, with additional IRB approval from Columbia University for the NY team on August 29, 2023. Data collection took place from May to December 2023.

Definition of terms

Screened sample

All participants from the population of PWUD who completed the screener survey to determine eligibility for enrollment in the study.

Eligible sample

All participants from the population of PWUD in the screened sample who met all eligibility criteria (18 years of age or older, able to read and speak English, able to understand and provide informed consent, residing in KY, NY, or OH at enrollment and during the 28-day observation period, and using any of heroin, fentanyl, powder cocaine, crack or rock cocaine, methamphetamine, pain pills or opioids not prescribed to the participant, benzodiazepines not prescribed to the participant, or stimulants not prescribed to the participant over the previous 30 days).

Enrolled sample

All participants from the population of PWUD in the eligible sample who completed the baseline survey.

Full sample

All participants from the population of PWUD in the enrolled sample who completed at least two of the four weekly surveys and reported drug use at least once during the 28-day observation period.

FTS sample

All participants from the population of PWUD in the full sample who used fentanyl test strips at least once during the 28-day observation period.

Oral fluid sample

All participants from the population of PWUD in the enrolled sample who completed the oral fluid visit (48-hour self-report survey and self-administered specimen collection) and whose oral fluid specimen generated analyzable biomarker results.

Organizational participants

Community organizations offering harm reduction services (e.g., naloxone distribution) that met inclusion criteria for distribution of fentanyl test strips (any of: currently distributing FTS, had distributed FTS in the past but stopped, intended to distribute FTS in the next year, or intended to distribute FTS but encountered barriers and never started) and participated in the study (completed a survey and/or interview).

Partner organizations

Community organizations (e.g., community clinics, syringe service programs, recovery community centers, and local public health departments) providing harm reduction services that partnered with state teams to host PWUD data collection at their service locations. All partner organizations distributed FTS at the time of data collection.

Qualitative sample

All participants from the population of PWUD who completed a one-on-one, semi-structured qualitative interview remotely after completing all survey events.

Research team

Scientists and research staff from the three state teams, the study DCC (RTI International), NIDA, and SAMHSA.

Study rationale

The primary impetus for the Stay Safe Study was the need for empirical evidence of the effectiveness of FTS use in reducing overdose risk. That investigation was beyond the scope of the HCS. The current study aimed to robustly assess the evidence in support of FTS use with a well-designed, well-powered, multilevel study implemented in diverse settings across three states. The decision to undertake this ancillary study to the HCS was facilitated by SAMHSA approval for use of FTS by federally funded researchers in April 2021 [32]. A second impetus was the need for multi-state study designs that investigated how multilevel contextual factors shape FTS distribution and use across states [28].

Populations and instruments

The study recruited two distinct populations: PWUD, and organizations providing harm reduction services. We recruited PWUD who attended the service locations of 14 partner organizations that distributed FTS at the time of our study. Partner organizations were community clinics, local public health departments, SSPs, and recovery community centers (which do not provide syringe services or clinical services). The study used three data collection instruments with PWUD participants: web-based surveys, one-on-one semi-structured interviews, and self-administered oral swabs to collect oral fluid specimens (see PWUD data collection and Table 1).

Table 1.

Data collection activities, timing, and incentives available to PWUD participants

Study Activity	Window to Complete	Incentive Amount	Activities at each visit	Approach
Screener survey	Day 0	$10	Eligibility determination	In-person (tablet at data collection site)
Baseline survey	Day 0	$50	Questions to collect participant contact information and data on demographics, information on past 30-day drug use, FTS use, and ORRBs	In-person (tablet at data collection site)
Weekly survey #1	Days 5–9	$40	Assess substance and FTS use in the past 5–9 days	In-person (tablet at data collection site) Remote (survey link)
Weekly survey #2	Days 12–16	$45	Assess substance and FTS use in the past 5–9 days	In-person (tablet at data collection site) Remote (survey link)
Weekly survey #3	Days 19–23	$50	Assess substance and FTS use in the past 5–9 days	In-person (tablet at data collection site) Remote (survey link)
Weekly survey #4	Days 26–30	$55	Assess substance and FTS use in the past 5–9 days	In-person (tablet at data collection site) Remote (survey link)
Oral fluid drug test and survey	Days 26–37	$25	Collect information about self-reported drug use in the previous 48 hours and saliva sample collection	In-person (tablet and saliva swab collection kit at data collection site)
PWUD interview	Up to 90 days after window for weekly survey #4	$50	Collect information about awareness of fentanyl in drugs, explore attitudes and patterns of FTS use, identify routine FTS practices, and identify barriers and facilitators to FTS access; along with information about ORRBs and overdoses	In-person (data collection site) Remote (Zoom platform)

All state teams invited persons with lived and living experience of drug use (PWUD and harm reduction employees) to review recruitment materials, surveys, interview guides and oral fluid collection procedures, and revise for appropriate length, language, and approach. Members of the Survivors Union of the Bluegrass, members of an HCS community advisory board, and employees of state organizations that provide harm reduction services, some of whom have lived and living experience, reviewed materials and procedures. We systematically revised all materials and instruments based on their input. The research team did not convene a community advisory board, and PWUD did not participate in data analysis.

We also recruited organizations providing harm reduction services (e.g., naloxone distribution) that had one of four distribution relationships to FTS at the time of our study (see inclusion criteria for organizational participants). The study used two data collection instruments with organizational participants: a survey and one-on-one semi-structured interviews.

Objectives and outcomes

The study has one primary, five secondary, and three exploratory objectives (Table 2). There was no intervention, and the study did not distribute FTS to participants.

Table 2.

Study objectives

Objectives	Outcome	Population	Instrument(s)
Primary Objective
1. Examine the association between FTS use at baseline and ORRBs during the 28-day observation period	Composite score measuring the number and frequency of eight distinct ORRBs on each day drugs are used during the 28-day observation period (range 0–15)	Full sample1	Weekly surveys 1–4
Secondary Objectives
1. Among FTS users, examine the association between FTS results and ORRBs during the 28-day observation period	Number of ORRBs following each FTS use on each day during the 28-day observation period	FTS sample2	Weekly surveys 1–4
2. Examine the concordance between self-reported substance use and oral fluid results	Oral fluid test positive for fentanyl (yes/no)	Oral fluid sample3	Past 48-hour survey Oral fluid collection
3. Examine the association between FTS use at baseline and the occurrence of a non-fatal overdose during the 28-day observation period	Self-reported non-fatal overdose on each day drugs are used during the 28-day observation period (yes/no)	Full sample1	Weekly surveys 1–4
4. Examine the perspectives and experiences of FTS use among PWUD.	Qualitative: no outcome identified	Qualitative sample4	PWUD interview
5. Examine the perspectives and experiences of organizations distributing FTS, facilitators and barriers to FTS distribution, and cost of FTS strategies	Qualitative: no outcome identified	Organzational partiicipants5	Organizational survey Organizational interview
Exploratory Objectives
1. Characterize patterns of FTS use among PWUD during the 28-day observation period	Using FTS at least once on each day drugs are used during the 28-day observation period (yes/no)	FTS sample2	Weekly surveys 1–4
2. Examine associations between social determinants of health and FTS use during the 28-day observation period	Using FTS at least once on each day drugs are used during the 28-day observation period (yes/no)	FTS sample2	Weekly surveys 1–4
3. Examine associations between social determinants of health and ORRBs during the 28-day observation period	Composite score measuring the number and frequency of eight distinct ORRBs on each day drugs are used during the 28-day observation period (range 0–15)	Full sample1	Weekly surveys 1–4

¹ Full sample is defined by all participants enrolled in the study who completed the baseline survey and at least two of the weekly surveys and reported drug use at least once during the 28-day observation period

² FTS sample is defined by all participants in the Full sample who used FTS at least once during the 28-day observation period

³ Oral fluid sample is defined as all participants who completed the baseline survey, the oral fluid survey and the oral fluid swab, had self-reported use of either non-opioids only (e.g., illicit stimulants or benzodiazepines but not illicit opioids) or any opioid (e.g., illicit opioids only or co-use of illicit opioids with illicit stimulants or benzodiazepines) in the 48 h prior to completing the oral fluid collection, and had valid results for the oral fluid test

⁴ Qualitative sample is defined by all participants in the full sample who completed the qualitative interview

⁵ Organizational participants are defined by all organizations that completed the organizational survey

The primary objective of the Stay Safe Study is to test the hypothesis that PWUD who report FTS use at baseline also report using a higher number and frequency of overdose risk reduction behaviors (ORRBs) during the 28-day observation period compared to PWUD who do not report FTS use at baseline. An ORRB is any behavior by an individual, acting alone or with others, which may reduce the risk of overdose (fatal or non-fatal) from substance use. Examples of ORRBs among PWUD are consuming less of the substance [33, 34], consuming with others [33, 35, 36], using a small amount of the substance to test potency and toxicity [33, 36, 37], discarding the substance if fentanyl is present [34], and having naloxone nearby [10, 22, 34, 35, 37].

The study includes five secondary objectives. The first four investigate behaviors of PWUD participants and the fifth investigates practices of organizational participants: (1) among PWUD who use FTS at baseline, examine the association between their FTS results and overdose risk reduction behaviors; (2) examine the concordance between self-reported substance use among PWUD and results of oral fluid testing; (3) examine the association between FTS use at baseline by PWUD and the occurrence of a non-fatal overdose during the 28-day observation period; (4) examine the perspectives and experiences of FTS use among PWUD; and (5) examine the perspectives and experiences of organizations distributing FTS, facilitators and barriers to FTS distribution, and cost of FTS strategies. Secondary objectives are measured by the following outcomes: (1) Number of ORRBs following each FTS use on each day during the 28-day observation period; (2) Oral fluid test positive for fentanyl (yes/no); and (3) Self-reported non-fatal overdose on each day drugs are used during the 28-day observation period (yes/no). Secondary objectives (4) and (5) were assessed qualitatively.

Sample, setting, and procedures

Team staffing models

Two states hired research staff living in communities near the service locations of partner organizations to manage study advertising, recruitment, and data collection. A third state hired employees of partner organizations who worked at their service locations to advertise the study, recruit PWUD participants, and collect data. Employees of these partner organizations had previously completed research ethics training and had developed rapport with their clientele [30, 38]. This staffing approach supported the team by having employees trusted by study participants conduct recruitment and eliminating extensive travel to data collection sites. Throughout this paper, we refer to all team members as “research staff.”

Study partner organizations

The study recruited a convenience sample of 14 partner organizations to host research activities at their service locations (KY = 5, NY = 3, OH = 6). Eleven of the 14 partner organizations (78.6%) were located in metropolitan areas (core or high commuting) and three (21.4%) were located in micropolitan area cores (primary flow within an Urban Cluster of 10,000 to 49,999) according to the US Department of Agriculture Rural-Urban Commuting Area Codes [39].

Partner organizations were any community-based organization that provided harm reduction services from a fixed service location (including parked vans and adjacent parking lots) or a mobile van service. For selection in this study, all were required to distribute FTS and serve a clientele that included (but was not limited to) PWUD. To recruit partner organizations, state teams approached organizations that were (1) community coalition members in the parent HCS, and/or (2) part of their research networks. Teams also used convenience sampling to identify interested organizations. Compensation models for partner organizations to host and/or conduct research activities differed by state. One state provided $2,000 to each organization with an additional $25 for each individual screened and $30 for each organizational participant that completed an interview. A second state provided a flat $5,000 to each partner organization that hosted data collection. A third state provided no compensation. In addition, some partner organizations had policies prohibiting receipt of compensation.

PWUD participants

Inclusion criteria

PWUD participants were required to be 18 years of age or older, able to read and speak English, able to understand and provide informed consent, and residing in KY, NY, or OH at enrollment and during the 28-day observation period. Participants were eligible for enrollment if they had used any of the following substances over the previous 30 days: heroin; fentanyl; powder cocaine; crack or rock cocaine; methamphetamine; pain pills or opioids not prescribed to the participant (such as hydrocodone, or oxycodone); benzodiazepines not prescribed to the participant (such as alprazolam or clonazepam); or stimulants not prescribed to the participant (such as amphetamine or methylphenidate). The PWUD participant cohort consisted of (1) individuals who self-reported FTS use in the 30 days prior to the baseline survey or planned FTS use in the 28-day observation period and (2) individuals who self-reported not using FTS in the 30 days prior to the baseline survey and had no plans to use FTS in the 28-day observation period.

Incentive payments

The study paid PWUD participants per data collection event [40] (Table 1). Although cash payments can promote the dignity of participants [41], the research team selected card payment systems because they offered significant benefits over cash (e.g., electronic tracking and increased safety for all). One card system offered additional functionality over major brand payment cards and was selected by two state teams. This card system allowed state teams to program payment amounts for each data collection event in advance, which reduced error.

Study advertising and setting

The research team reviewed draft advertising materials for inclusivity and agreed on a single poster with images that included persons of color and person-first language. Each state advertised the study at indoor and outdoor service locations of the 14 partner organizations, which became the study data collection sites. Teams in KY and OH began advertising in May 2023; the NY team followed in August 2023. At each data collection site, research staff recruited individuals, explained study purpose and procedures, obtained informed consent, and collected data for the screener, baseline, weekly, and 48-hour oral fluid surveys, as well as the self-administered oral fluid specimen collection using an oral swab (see PWUD data collection). One state team recruited PWUD participants in the mobile van unit of a partner organization, allowing for an expanded outreach of diverse participants.

Estimating recruitment targets

The DCC calculated a full sample target of N = 750 PWUD for the study (see Sample size). Because state teams knew in advance that not all enrolled PWUD participants would complete four weekly surveys during the observation period, they intentionally recruited and enrolled a large number of PWUD participants to meet this full sample target.

PWUD data collection

Screener, baseline, and weekly surveys

Research staff used a web-based screener survey loaded on electronic tablets with interested individuals at the data location sites to determine their study eligibility. Those who did not meet the eligibility criteria were compensated $10 for their time, thanked for their participation, and excluded.

With eligible individuals who chose to enroll, research staff reviewed the data collection timeline and events, then completed verbal and electronic informed consent. Immediately afterwards, eligible individuals completed a baseline survey that collected (1) participant demographics (e.g., race, ethnicity, gender identity, sexual orientation, marital status, educational attainment, health insurance status) and (2) information about substances used in the past 30 days, frequency of use, route of administration, and their baseline knowledge and awareness of FTS. The survey included questions on FTS prior use and previous education on how to use FTS. The final portion asked about use of ORRBs (e.g., having naloxone nearby), mixing of drugs, and overdose incidence in the prior 30 days. Individuals were asked to provide contact information (e.g., email address, mobile or home telephone numbers, and/or Facebook messaging coordinates) to receive reminders and instructions for subsequent data collection.

After completing the baseline survey, the individual was deemed an enrolled PWUD participant. Enrolled PWUD participants were invited to return to the data collection site every seven days for a maximum of four times to complete a weekly survey during their 28-day observation period. Participants were eligible to complete each weekly survey either at the recruitment site or remotely in a setting that was deemed private with an Internet connection by logging into the DCC study survey portal with their username and password.

The weekly survey asked about FTS use, ORRBs, and use of specific drugs for each day of the past week (i.e., heroin, fentanyl, powder cocaine, crack or rock cocaine, methamphetamine, pain pills or opioids not prescribed to the participant, benzodiazepines not prescribed to the participant, and stimulants not prescribed to the participant). For each day that the participant indicated drug use in the weekly survey, they were asked to indicate which drugs they used, whether they used FTS, reasons they did or did not use FTS, and the results of FTS use for the first two FTS testing events on that day. Participants were also asked to identify ORRBs they used. After indicating the result from up to two daily instances of FTS use (positive, negative, or undetermined), participants were asked about their drug use behaviors following the FTS result(s).

Oral fluid visit

Participants were invited to attend an in-person oral fluid visit in the Day 26–37 window after their study enrollment and after their last weekly survey window had closed. The visit consisted of two self-administered activities: completion of an electronic survey asking participants to identify all substances consumed in the past 48 hours, and collection of an oral fluid specimen using a swab.

The research team selected the oral fluid test over a urine drug test because oral fluid specimen collection is easy to implement and specimens do not require refrigeration. There was no study target for completed oral fluid visits (see PWUD data collection).

The study convened medical directors from each state to discuss best approaches in each state to administering the oral fluid visit, and sought additional input from PWUD in KY. The study contracted the laboratory Millennium Health to provide oral fluid specimen kits and shipping materials to each state team and to conduct analysis of collected specimens.

Research staff did not mention the oral fluid visit to participants until after the fourth weekly survey to minimize changes to routine substance use practices. A separate informed consent process and form were used to allow participants to opt in. Each participant was asked in the 48-hour self-report survey whether they wished to receive a summary report of their oral fluid results.

Research staff did not perform any specimen testing; they mailed collected specimens and documentation to Millennium Health. Millennium Health tested specimens and returned results to the DCC, which uploaded them to the study portal. Research staff downloaded oral fluid results from the DCC portal and communicated them to PWUD participants in person and/or by telephone.

Interview

The research team set a target of 120 completed one-on-one semi-structured interviews with PWUD (40 per state). Experienced qualitative researchers conducted interviews lasting up to an hour with a subset of participants from the enrolled PWUD sample. Interviews explored participant awareness of fentanyl in their supply, attitudes towards FTS, FTS use patterns, barriers and facilitators to obtaining FTS, and use of ORRBs. Interviews also solicited advice participants would give about FTS to a friend who uses drugs, recommendations for FTS programs, and a change they would make to prevent fatal fentanyl-related overdose.

To include diverse perspectives in the interview sample, the leads of each state team developed a weighted, purposive sampling frame to optimize representation of underrepresented groups. Leads identified 28 demographic and substance use-related characteristics (29 characteristics in NY to include xylazine test strip status) across seven variables in the quantitative survey (eight in NY to include xylazine test strip status) on which to sample (Table 3). These variables were theoretically driven on the basis of known risk and access differences between groups [42, 43]. Guided by state-specific frequency tables for the 28 characteristics, team leads prioritized recruiting eligible participants from variable groups with the smallest survey frequencies. Once the PWUD interview samples were selected for each state, the interviewers selected the appropriate interview guide that corresponded to the participant’s two-cell FTS status.

Table 3.

Variables and groups for weighted sampling of PWUD interview participants

Variables	Groups
Gender	Gender: TG (Transgender)
	Gender: GNC (Gender Non-Conforming)
	Gender: I (Intersex)
	Gender: M (Male)
	Gender: F (Female)
Age	Age: 18–29
	Age: 30–59
	Age: 60+
Race	Race: AS (Asian)
	Race: NH/PI (Native Hawaiian/Pacific Islander)
	Race: B/AA (Black/African American)
	Race: AI/AN (American Indian/Alaska Native)
	Race: WH (White)
Ethnicity (Hispanic)	Ethnicity: H (Hispanic)
	Ethnicity: NH (Non-Hispanic)
Past 30 days substance use	Past 30: Stim (Stimulants)
	Past 30: Benzo (Benzodiazepines)
	Past 30: 3 + subs (3 + substances)
	Past 30: Crack/cocaine
	Past 30: Rx Op (Prescription Opioid)
	Past 30: Meth (Methamphetamine)
	Past 30: Fentanyl
	Past 30: Heroin
Past 30 days xylazine use	Past 30: Xylazine*
Past 30 days overdose	Past 30 OD (Overdose)
FTS subgroup	FTS: N-N
	FTS: Y-Y
	FTS: N-Y
	FTS: Y-N

*NY investigated use of xylazine test strips by PWUD participants

FTS subgroup status was assigned to each participant based on their responses to questions about FTS use in (1) the baseline survey, and (2) the weekly surveys. Those responses constituted a two-by-two subgroup designation: Subgroup 1 (N-N): participants who did not anticipate using FTS at baseline and did not use FTS throughout the 28-day observation period; Subgroup 2 (Y-Y): participants who anticipated using FTS at baseline and did use FTS during the 28-day observation period; Subgroup 3 (N-Y): participants who did not anticipate using FTS at baseline and reported using FTS during the 28-day observation period; and, Subgroup 4 (Y-N): participants who anticipated using FTS at baseline and did not report using FTS during the 28-day observation period. Each interview guide corresponded to one of four subgroups, with a target of 30 participants in each subgroup. Domains and questions in each guide were tailored to each FTS subgroup. Interviewers used the PWUD interview guide that corresponded to the FTS subgroup status of the participant they interviewed.

Organizational participants

Sample and setting

The study also recruited a population of 28 community organizations that provide harm reduction services, including the 14 partner organizations that hosted data collection in the three states. The purpose was to identify multilevel facilitators and barriers to FTS distribution, reach of their FTS programs to at-risk populations, costs of FTS purchasing and distribution, and the role of contextual factors in FTS use and ORRBs among PWUD. Research staff used email and phone communication to advertise the study to potential organization-level participants, and conducted all data collection remotely (phone, online surveys, or Zoom interview). Research staff used a separate informed consent process and electronic consent form for the survey and interview.

Inclusion criteria

Community organizations providing harm reduction services were eligible to participate in the survey and interview if they reported any of the following: (1) currently distributing FTS, (2) had distributed FTS in the past but stopped, (3) intending to distribute FTS in the next year, or (4) intended to distribute FTS but encountered barriers and never started. All partner organizations that participated de facto met inclusion criterion (1) (currently distributing FTS). The research team chose broad inclusion criteria to recruit a large number of organizational participants that had encountered a range of legal, policy, and financial barriers for FTS distribution, such as criminalization statutes and restrictive harm reduction climates. Leads or employees of organizations who had sufficient knowledge of the organization and made decisions for their organization’s FTS distribution practices were eligible to participate.

Incentive payments

State teams tailored incentives to organizational participants based on past practice. Two states offered incentive payments to organizations for survey and/or interview participation, and one state offered no incentive payments.

Organization data collection

Survey

Using an emailed invitation script, each state team invited organizational participants to complete a web-based survey hosted on the study portal. The survey solicited organization location, self-classification, staffing size, population(s) served, and FTS program details (e.g., costs and number ordered monthly).

Interview

Each team invited the same organizations to participate in a one-on-one semi-structured interview led by an experienced qualitative researcher. The interviewer solicited information about policies and practices for FTS distribution including changes over the last year, strategies to reach at-risk and minority populations, and client educational practices. They also solicited recommendations about FTS distribution, an overarching message about FTS, and changes recommended to prevent PWUD from having a fatal overdose from fentanyl.

Educational materials

Interviewers also asked each organization to provide copies of FTS educational materials and resources available in any format (e.g., electronic and hard documents, videos links, and links to social media advertising and messaging) used to communicate FTS availability or instructions for use.

Sample size, data monitoring, and analysis

Sample size

A sample size of 750 PWUD in the full sample was selected to ensure that a 0.19–0.43 standard deviation difference in the mean of the daily composite score across the 28-day observation period between PWUD who self-reported FTS use at baseline and PWUD who self-reported no FTS use at baseline could be detected with 80% power. This sample size was determined by assuming an average of 30% missing response across the 28-day observation period, 0.80 correlation between any two of the daily scores measured on the same participant, and a two-sided significance level of 0.05. Estimates were based on a Generalized Estimation Equation test of the time-averaged difference of two groups in a repeated measured design of a continuous outcome (available in PASS 2022).

Data monitoring

Remote data monitoring was conducted by the DCC throughout the study to monitor enrollment and data quality. Any participant self-reported data identified as having data quality issues was documented but ultimately, no revisions were made to the data collected in the web-based survey given the nature of self-reported survey data. Any data quality issues identified with oral fluid results were resolved with Millenium Health prior to database lock.

Survey data analysis

Statistical analyses of all objectives using survey data were conducted by statisticians from the DCC. A detailed statistical analysis plan for the primary objective and all secondary objectives that required quantitative analyses was drafted prior to database lock and finalized prior to dissemination of results to the research team. Due to the nature of the exploratory objectives, a statistical analysis plan for these objectives that required quantitative analyses was drafted after database lock. The analysis population for each objective was selected based on the completion of the baseline survey, weekly surveys, drug use during the observation period, FTS use during the observation period, and receipt of oral fluid results (Table 1). Cross-sectional outcomes were analyzed using a generalized linear model with an appropriate distribution and link function and fixed effects included the exposure of interest as well as a set of pre-specified baseline covariates. Outcomes measured repeatedly over time were analyzed using a generalized estimating equation approach with an appropriate distribution and link function and fixed effects included the exposure of interest, a set of pre-specified baseline covariates, as well as a set of time-varying covariates. Missing data were assumed to be missing completely at random, with the caveat that multiple imputation be used if the percentage of participants not included in the primary analysis of each objective due to missing data was greater than 5%. All hypotheses were tested using a two-sided significance level of 0.05. No adjustment was made for multiple comparisons across outcomes or analyses within a specific objective and so all p-values presented are purely descriptive in nature.

Qualitative data analysis

The DCC contracted a professional transcriptionist to transcribe all interviews. A cross-state qualitative research team led by the DCC and staffed by each state team met weekly for several months to develop two codebooks, one each for the PWUD and organization interview data. The cross-state qualitative team used NVivo Plus 12 for Windows [44]. Staff from each state team were responsible for coding interview transcripts from their own state using a combination of inductive and deductive coding, and analyzing findings. They reviewed coded transcripts, summarized codes, and identified quotes to illustrate and support their findings. Main topics identified by the interview guide and codebook were used to summarize participant responses, with quotes to support findings. State teams presented results at each cross-state meeting; these were discussed until the team reached consensus. The cross-state qualitative team then conducted thematic analysis guided by specific research questions generated by the lead of the respective manuscripts. For the organization interview data only, the cross-state team incorporated the reach construct from the RE-AIM implementation science framework into the codebook, which mirrored its use in the interview guide for organizations [45–47]. Each state team was involved in all steps of the qualitative process.

Results

Enrollment and data collection have been completed and data analysis is underway. Study findings will be presented in subsequent publications [1, 2]. From May-December 2023, the study screened 1,405 individuals and enrolled 1,156 PWUD (82.3% of screened individuals) (Fig. 2). The full sample was 732 PWUD (63.3% percent of enrolled participants). A total of 309 PWUD completed oral fluid collection, of which n = 267 were used for the oral fluid analysis, and 120 PWUD completed one-on-one interviews.

Fig. 2.

CONSORT diagram of those screened, eligible, enrolled, and followed during the study

The study achieved 97.6% (732/750) of its full sample target for PWUD participants. It accomplished 100% of the study target of 120 interviews with the PWUD sample. There were more weekly surveys completed in Weeks 2, 3, and 4 of the 28-day observation period compared to Week 1, even with participant withdrawals (Fig. 2). From the population of 36 organizations providing harm reduction services and meeting the study eligibility criteria, employees of 28 organizations (77.8%) in 22 counties across 3 states completed the survey and employees of 24 organizations (66.7%) completed the interview. The study began with fourteen partner organizations that hosted data collection and ended with seven (see Discussion).

Discussion

To our knowledge, few researchers have provided methodological reflections on protocol implementation for harm reduction studies and none for FTS studies. To encourage researchers to develop large-sample and multi-state FTS study designs that engage community partner organizations, and to provide insight into implementation, we discuss factors that contributed to implementation success along with areas in which we encountered complexity [30].

Several factors supported successful implementation of this three-state study protocol. An overarching factor was the ability to tailor staffing models and data collection approaches to accommodate the unique conditions at each data collection site. Another factor was the study was able to leverage existing infrastructure created for the HCS, which was in its fourth year when the Stay Safe Study was launched [31, 48]. Infrastructure included a DCC, regulatory support, local community expertise within community coalitions, and established partnerships with organizations delivering harm reduction services. HCS staff have substantial experience implementing a large community-engaged study and supported teams in identifying potential partner organizations, helping with recruitment of research staff, and purchasing study materials. A related benefit of our association with the HCS was that organizations that were members of HCS community coalitions were receptive to hosting our study data collection as partners and/or participating in the study.

Factors unrelated to operational support from the HCS were associated with successful implementation. In line with best practices for conducting research with PWUD [49, 50], the research team selected appropriate incentive amounts for PWUD for each data collection event and set an increasing weekly survey incentive rate (by $5 each week) during the 28-day observation period to encourage retention (Table 1). The DCC hosted centralized survey data collection which facilitated troubleshooting for server and login problems. This approach also eliminated the need for state teams to host their own survey data collection. The DCC hosted a study SmartSheet (smartsheet.com) for all research staff to post implementation challenges on the day they arose. This central forum allowed teams to suggest resolutions, make recommendations to the DCC on data collection operations, which were quickly implemented, and learn from each team’s experience.

Protocol development and implementation, which spanned over two years, was complex and included diverse approaches to solving challenge encounters. We illustrate lessons learned with examples for four implementation areas. In Table 4, we provide recommendations to researchers.

Table 4.

Lesson learned and recommendations

Implementation area	Lessons learned and recommendations
Conducting research in communities	Lessons learned: Conducting research in communities offers many benefits but introduces challenges requiring rapid responses and pivoting. Research activities may conflict with the direct service provision site mandates or business operations of partner organizations. Not all sites are suitable for data collection, even if partner organizations agree to data collection on site. On-site research may create safety risks for research staff. Recommendations: Teams are guests of partner organizations and must work with the schedules, workflow, and spacing needs of direct service provision sites. Initiate regular communication with partner organizations to maintain trust and share information. Provide weekly team updates and scheduling changes, actively solicit their concerns, and resolve conflicts together. Be willing to pause data collection while seeking solutions. Consider situating recruitment and data collection activities in nearby accessible community spaces (e.g., public health departments, parking lots of clinics and hospitals or the sites) or leasing a mobile study van or tent that can be stationed adjacent to sites, to offer privacy to participants. Assess ongoing suitability of each site throughout data collection. Consider providing additional incentives to PWUD participants who complete the study to improve retention. Create a central SmartSheet or discussion forum for teams to identify real-time challenges and resolutions.
Making timely payments	Lessons learned: Late payments to PWUD participants and difficulty accessing those payments can distress participants and cause high workloads for the team. Multiple compensation models for organizational participants may be appropriate. Payment to partner organizations may need to be formalized in memorandums of understanding. These may take months to generate and delay payment. Recommendations: Prioritize equitable and timely compensation that values the expertise of PWUD participants, organizational participants, and partner organizations. Select participant incentive payment systems supported by the institution, communicate activation instructions to participants with handouts, and allows for delays with the payment system (e.g., system upgrades or institutional delays making contracted payments). Set realistic timelines to load payments to cards that accommodate surges in participant volume (e.g., before the weekend). Communicate payment timelines to participants at every contact to set expectations and reduce distress. Reassure partner organizations that compensation payments are being approved and provide updates on when they will receive them.
Ov ersight of research staff	Lessons learned: Collecting data with vulnerable participants can be physically and emotionally demanding for research staff, even with experience. Long driving times to reach data collection sites increases fatigue and accident risk, particularly during inclement weather. Staff may be both energized and exhausted from meeting with participants and challenged by setting limits on interactions. Staff who manage participant communications and incentive payments may work beyond effort allocation at peak periods, including weekends and holidays. Employees of direct service provision sites who are contracted to recruit and collect data may be inconsistent in their research practices. Recommendations: The well-being of research staff contributes to team cohesion and commitment to study completion. Monitor staff well-being and hours. When staff develop fatigue, reduce the number of data collection sites, contract part-time study personnel, ask other staff to assist with data collection, or pause activities to provide rest. Allow staff to cancel data collection if inclement weather poses a driving or outdoors hazard. Encourage staff to respect their limits (e.g., leave sites when their hours are completed) and prioritize their safety. Provide additional on-site training to employees of data collection sites and check on the consistency of their practices.
Returning oral fluid results to participants	Lesson learned: Returning oral fluid results to PWUD participants can provide actionable information but increases operational complexity by adding non-data collection contact events. Return of oral fluid results can be burdensome to research staff, partner organizations, and/or participants. Staff may need to schedule appointments with participants and add additional driving once data collection has ended. Recommendations: Acknowledge the validity of different implementation approaches, along with potential burdens to all parties. Solicit preferences of partner organizations for approaches to return results that are minimally disruptive. Balance pragmatic considerations for most effective return approaches with ethical concerns about protecting participant privacy and integrity of results. There may be no single best practice, and several approaches can achieve the same objectives. Communicate the intention and purpose of returning oral fluid results to partner organizations so that they understand this is a non-data collection event and can communicate this to PWUD participants.

Conducting research in communities. Partner organizations operate sites that provide harm reduction services to a diverse clientele. They juggle many demands throughout the day in tight spaces and with a small number of staff. Sites were frequently crowded with little room for teams to work. At times, data collection conflicted with their operations. As a result, some partner organizations asked teams to move their data collection onto adjacent sidewalks or parking lots. However, outdoor data collection created unpredictability and instability by exposing research staff and PWUD participants to weather hazards [30]. One partner organization asked a team to end data collection because noise from participants and long lines outside business offices interrupted meetings and phone calls. Research staff who travelled with mobile vans that provided harm reduction services to neighborhoods had less opportunity to offer privacy to PWUD inside the vehicles, which limited enrollment and data completion. One team ended data collection at a site because the team lead felt they could not guarantee the safety of research staff.

Making timely payments. During the first week of data collection, PWUD participants in one state attempted to activate their electronic payment cards using personal identifiers that were not registered to the cards. Participants experienced a high level of distress when they could not retrieve their payments and sent a large volume of phone calls, voice mails, and email messages to the team. Some participants showed in-person at data collection sites with distress after being unable to retrieve their payments, which increased the work of research staff and impacted data collection for other participants. The team resolved this by distributing an approved activation handout for PWUD participants.

Oversight of research staff. In one state, employees of direct service provision sites who were contracted to collect PWUD data did not consistently follow data collection practices for the ODT, which then required additional IRB review. In another state, two research staff drove 40 hours per week to six data collection sites and worked an additional 30 hours per week to meet high demand for study enrollment. To allow staff time to rest and meet demand for participants, the team halted recruitment for two months and resumed data collection with just two sites.

Returning oral fluid results. The research team agreed to return oral fluid results to PWUD participants to provide potentially actionable information to participants about their supply. Medical directors for all teams reviewed and approved the oral consent process and form, and the oral fluid collection procedures. Two states delivered oral fluid results to PWUD participants using either telephone calls or a combination of telephone calls and in-person delivery at data collection sites. One state hand-delivered all oral fluid results in a sealed envelope to PWUD participants at data collection sites. While this approach increased the team’s confidence in protecting participants’ privacy, it was burdensome to participants, with few returning to receive their oral fluid results.

Strengths and limitations of the study

A key strength of this prospective cohort design study was the use of a multi-state design and the recruitment of 14 partner organizations hosting data collection. This approach generated both diversity of PWUD participants and the largest sample to date of PWUD participants in an FTS study. State teams implemented a common protocol which optimized feasibility and study rigor, although it resulted in different engagement and retention rates in each state. Additional strengths included the use of weekly surveys during the 28-day observation period to investigate daily patterns of drug and FTS use by PWUD participants, use of an oral fluid drug test to check concordance and discordance of self-reported substance use by PWUD participants, and the inclusion of community organizations that provide harm reduction services to build the limited evidence base on organizational barriers and facilitators to FTS distribution.

There are several limitations of the Stay Safe Study design and implementation. The research team solicited input from PWUD and staff of partner organizations on instrument design and the oral fluid visit. However, the time available to fully consult with PWUD and staff was necessarily limited by the goal of meeting study milestones within the award period (e.g., launching the study and completing data collection and analysis). This was a notable tension in study implementation. It was managed with compromise by the research team, who agreed to shorten the period of time for consulting with PWUD and organization staff on instrument design while adjusting study deadlines to accommodate consultation and revision, which all agreed were vital. We recommend researchers who develop multi-team, community-engaged studies build adequate time into their designs to fully involve PWUD and community partners in all stages of the design and implementation of the research.

The PWUD sample was recruited from the service locations of community clinics, local public health departments, and SSPs, as well as recovery community centers (which do not provide syringe services or clinical services). Although it is reasonable to assume that most PWUD participants were clients of these service locations, we did not confirm this. Any adult community member who visited a recruitment location was eligible to complete the study screener. By recruiting participants at community clinics, recovery centers, and local public health departments, which have a different (and in some cases, broader) service scope than SSPs, our study population may have been more representative of the general population than if we had recruited solely at SSPs.

Eleven data collection sites were located in metropolitan areas according to rural-urban commuting area codes, which resulted in a highly urban sample of PWUD participants. Populations at significant risk of fentanyl-related overdose, such as non-Hispanic/Black stimulant users and youth who use substances recreationally may be underrepresented in our PWUD enrolled and full samples.

To protect the privacy of PWUD participants and foster trust, the study did not require PWUD participants to provide photo identification at enrollment. This decision led to a small number of duplicate enrollments. Where possible, the DCC identified each of these from identical contact information and removed duplicate enrollments from the data set. Partner organizations that hosted PWUD data collection activities and organizational participants were recruited opportunistically and were not a representative sample from the three states. All partner organizations hosting data collection sites were required to distribute FTS as an inclusion criterion, which may have overrepresented FTS use and/or rates of FTS acceptability in the enrolled PWUD sample.

All service locations of partner organizations where the team recruited PWUD participants were distributing FTS at the time of recruitment. This may have led to over-recruitment of PWUD participants exposed to FTS distribution and other harm reduction services. We collected data about past 30-day FTS use and intention to use FTS during the 28-day prospective period, but did not collect baseline data about PWUD prior exposure to other harm reduction services or resources. Consequently, prior utilization of harm reduction services by PWUD is unknown. Finally, survey and interview data collected from PWUD were self-reported, which limits validity and reproducibility.

Conclusions

To our knowledge, this is the first multi-state study investigating FTS use by a sample of PWUD and distribution by a sample of community organizations. The research team successfully enrolled a large sample of PWUD and organizational participants, followed a cohort of PWUD for 28 days, and generated rich data using three types of instruments. Although the study focus is the use of FTS for overdose prevention and distribution of FTS, harm reduction, the study design and protocol can be implemented in drug treatment programs. We encourage researchers to consider using this design for studies beyond harm reduction.

A key lesson learned is that in a multi-state study with two populations, the desire to harmonize data collection approaches and integrate PWUD input into instrument development and data analysis must be balanced with the need of state teams to tailor protocol implementation to data collection sites in their states and achieve study milestones on time. Each state team makes the best decisions for study implementation based on their budget, staffing, target populations, and recruitment and data collection approaches. There is often no single correct approach. As we saw with this multi-state study, each team must be prepared to rapidly pivot when challenges arise in study implementation—in their own state and across the study. Ultimately, relationships of trust with the partner organizations that agree to host participant recruitment and data collection, and with the PWUD and organizations that participate in data collection events, drive and shape implementation success.

Abbreviations

DCC

Data coordinating center

DEA

Drug Enforcement Administration

FTS

Fentanyl test strips

HCS

HEALing Communities Study

IRB

Institutional Review Board

KY

Kentucky

NIDA

National Institute on Drug Abuse

NIH

National Institutes of Health

NY

New York

OH

Ohio

ORRB

Overdose risk reduction behavior

PWUD

People who use drugs

SAMHSA

Substance Abuse and Mental Health Services Administration

US

United States

Author contributions

D.C.B., R.C., R.K.C., J.E.C., J.L.D., M.E.D., N.E-B., A.F-B., L.C.F., S.F., B.F., K.G., L.G., M.E.H., M.H., R.D.J., M.K., B.S.L., K.M., E.O., C.R., R.V-S., J.V., and S. L.W. developed the study concept and design. J.E.C., J.L.D., N.E-B., S.F., K.G., L.G., R.D.J., K.E.L., B.S.L., K.M., E.O., C.R., R.V-S., and S.L.W. obtained funding. J.E.C., M.E.D., A.F-B., K.G., B.S.L., M.R., and R.V-S. performed data collection. D.C.B., M.E.H. E.O., and M.R. were responsible for data management. D.C.B. and M.E.H. performed statistical analysis. D.C.B., J.E.C., and M.E.D. prepared Figure 2. D.C.B., R.K.C., J.E.C., M.E.D., N.E-B., K.G., E.O., and R.V-S. drafted the manuscript. All authors critically revised the manuscript for review.

Funding

This research was supported by the National Institutes of Health and the Substance Abuse and Mental Health Services Administration through the NIH HEAL (Helping to End Addiction Long-term^®) Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049415, UM1DA049417. Drs. Villani and Chandler were substantially involved in UM1DA049394, UM1DA049406, UM1DA049415, and UM1DA049417, consistent with their roles as Scientific Officers. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Substance Abuse and Mental Health Services Administration, or the NIH HEAL Initiative^®.

The HEALing Communities Study protocol (Pro00038088) and the Stay Safe Study protocol (Pro0006830) were approved by Advarra Inc., the HEALing Communities Study single Institutional Review Board.

We wish to acknowledge the participation of the HEALing Communities Study communities, community coalitions, community harm reduction agencies, Community Advisory Boards, and state government officials who partnered with us on this study.

Data availability

Consideration will be given to sharing the instruments upon request. Such requests should be sent to the corresponding author.

Declarations

Ethics approval and consent to participate

The study was approved by Advarra (Pro00068304) on March 16, 2023, with additional approval by Columbia University IRB (iIRB-AAAU5847) on August 29, 2023.

Consent for publication

Not applicable.

Competing interests

Amanda Fallin-Bennett is the co-founder of Voices of Hope, a Kentucky partner organization in the Stay Safe Study.