Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Feb 5.
Published in final edited form as: Contemp Clin Trials. 2015 Aug 5;44:95–102. doi: 10.1016/j.cct.2015.08.002

A Non-Inferiority Trial of an Evidence-Based Secondary HIV Prevention Behavioral Intervention Compared to an Adapted, Abbreviated Version: Rationale and Intervention Description

Roman Shrestha a,b, Archana Krishnan b,c, Frederick L Altice b,c,d, Michael Copenhaver b,e,*
PMCID: PMC4744142  NIHMSID: NIHMS718953  PMID: 26253181

Abstract

Background

Real-world clinical settings like addiction treatment programs are ill-equipped to deploy and sustain the existing-resource-demanding evidence-based interventions (EBIs) that target HIV-infected people who use drugs (PWUDs), and this has left a critical void in current HIV prevention efforts. In response to this unmet need, we have conducted formative research in addiction treatment settings that has resulted in Holistic Health for HIV (3H+) – an empirically adapted, substantially abbreviated version of Holistic Health Recovery Program (HHRP+), a CDC-recommended EBI targeting HIV-infected PWUDs.

Methods

Using a non-inferiority randomized controlled trial design, we will determine whether the abbreviated 3H+ intervention is comparable (i.e., within a 10% margin) and cost-effective relative to the original HHRP+ intervention in terms of reducing HIV risk behaviors and improving antiretroviral therapy (ART) adherence among HIV-infected PWUDs in addiction treatment who report drug- or sex-related HIV risk behaviors.

Conclusions

This article provides a description of the development and adaptation of the 3H+ intervention, the innovative non-inferiority comparative experimental design for testing the 3H+ to the HHRP+. Furthermore, it provides empirical evidence from a formal cost-effectiveness analysis justifying the cost-effectiveness of the 3H+ intervention when compared to the HHRP+ intervention. If confirmed to be comparable and more cost-effective, as hypothesized, the 3H+ intervention has the potential to be readily and immediately integrated within common clinical settings where large numbers of HIV-infected PWUDs receive clinical services.

Keywords: Secondary HIV prevention, intervention adaptation, antiretroviral adherence, HIV risk reduction, non-inferiority prospective trial, drug users

1. Introduction

Despite numerous evidence-based behavioral HIV-prevention interventions made available by the Centers for Disease Control and Prevention (CDC), HIV incidence in the U.S. has remained unchanged for the past 15 years [1]. Particularly in the Northeast, where almost half of all new HIV infections are attributed to people who use drugs (PWUDs), they remain a priority population [2]. HIV-infected PWUDs continue to fuel the U.S. HIV epidemic via drug- and sex-related HIV risk behaviors that are preventable through appropriately designed and properly situated evidence-based interventions (EBIs) [38]. Scale-up of EBIs in ‘real-world’ clinical settings, like drug treatment programs, are ill-equipped to deploy and sustain the existing resource-demanding EBIs that target HIV-infected PWUDs due to organizational factors within clinical settings (e.g., resource-limitations) and individual factors (e.g., lack of willingness to participate in comprehensive interventions), and this has left a critical void in current HIV-prevention efforts [9, 10]. Thus, efficacious and cost-effective EBIs are urgently needed.

Addiction treatment programs are ideal settings to implement secondary HIV-prevention and antiretroviral therapy (ART) adherence interventions among HIV-infected PWUDs. To date, however, few–if any–EBIs have been designed for implementation among HIV-infected PWUDs in the context of addiction treatment programs such methadone maintenance programs (MMPs), where such interventions are particularly relevant. The few EBIs that are potentially applicable to HIV-infected PWUDs have not been widely disseminated and adopted in drug-related clinical settings due to numerous factors (e.g., time constraints, existing treatment routines, personnel demands, patient attributes) that typically distinguish real-world clinical settings from idealized research settings in which interventions are often originally tested. Consequently, clinical settings are less willing and able to commit scarce resources toward delivering, monitoring, and evaluating complex EBIs [911].

Driven both by organizational factors within clinical settings (e.g., resource-limitations) and participant factors (e.g., lack of willingness to participate in lengthy/complex interventions), researchers have begun developing and testing brief, ecologically valid, EBI strategies [12, 13] for use within a range of non-addiction treatment settings such as STD and HIV specialty clinics where routine medical care is provided to HIV-infected patients. Though these interventions have been well-designed for implementation in non-addiction treatment settings, their content does not address the unique drug- and sex-related HIV risk-reduction and ART adherence needs that are characteristic of HIV-infected PWUDs. ART adherence is prioritized since achieving viral suppression has been associated with marked reductions in HIV transmission to sexual and injection partners [14, 15]. Thus, there remains a significant void with regard to EBIs that can target HIV-infected PWUDs in clinical settings and address both HIV risk-reduction and ART adherence.

In response to this unmet need, we have adapted and optimized EBI approaches for implementation among priority populations in addiction treatment settings. Through our formative research in addiction treatment settings, we developed the Holistic Health for HIV (3H+) intervention [16], an empirically-adapted, substantially abbreviated version of Holistic Health Recovery Program (HHRP+) [17], a CDC-recommended EBI targeting HIV-infected PWUDs [18]. The purpose of this paper is to: 1) provide a step-by-step description of the development and adaptation of the 3H+ intervention; 2) describe the innovative non-inferiority comparative experimental design for testing the 3H+ intervention to the ‘gold-standard’; and 3) provide empirical evidence from a formal cost-effectiveness analysis justifying the cost-effectiveness of the 3H+ intervention when compared to the original HHRP+.

2. Methods

2.1. Study design

Given that EBIs currently exist for HIV-infected PWUDs, we deemed it unethical to test the abbreviated 3H+ intervention against anything other than one with documented efficacy. Thus, our research team was funded to implement a non-inferiority RCT design [1921] to compare the abbreviated 3H+ intervention with the existing HHRP+ [17], the gold standard for secondary HIV prevention for HIV-infected PWUDs (Figure 1). This design will allow us to test the primary hypothesis that the experimental intervention (E=3H+) is at least as effective as (non-inferior to) the ‘gold standard’ active comparison intervention (S=HHRP+) by a margin of ≤10% (i.e., the experimental intervention to be no more than 10% worse than the standard intervention for the primary outcome of no HIV risk behavior). Our ongoing study is testing the relative efficacy and cost-effectiveness of the 3H+ intervention [16], vs. the original EBI – the HHRP+ [17]. In order to test this hypothesis, a RCT design (see Figure 1) has been implemented comparing the abbreviated 3H+ intervention to the HHRP+ intervention, with both conditions receiving standard of care opioid replacement therapy (drug treatment). Though it is not possible to eliminate all threats to internal validity, the randomized controlled trial design allows us to minimizing such threats over the course of the study. Ethical oversight

Figure 1.

Figure 1

Overview of study design

Note: T measured in weeks

The study protocol was approved by the Investigational Review Board (IRB) at the University of Connecticut, the Human Investigation Committee at Yale University, and received board approval from the APT Foundation MMP, Inc. Clinical trial registration was completed at ClinicalTrials.gov (NCT01741311). A Certificate of Confidentiality (CoC) was obtained from the National Institutes of Health (NIH).

2.2. Research goals

Given the translational gap of currently available EBIs targeting HIV-infected PWUDs into common clinical settings (e.g., addiction treatment programs, HIV clinical care settings, etc.), this study’s aim is to examine whether an adapted, brief version of an EBI (i.e., 3H+ intervention) is comparable to the original EBI (i.e., HHRP+ intervention) in reducing HIV risk behaviors and improving ART adherence. We hypothesize that the 3H+ intervention will be found comparable (i.e., within a 10% margin) and cost-effective relative to the original HHRP+ in terms of reducing HIV risk behaviors and improving ART adherence among HIV-infected PWDUs in addiction treatment who report unsafe injection drug use practices or sexual risk behavior. Thus, the primary outcomes focus on ART adherence and several aspects of sex- and drug-related HIV risk-reduction information, motivation, behavioral skills, and behavior. The secondary outcomes focus on measures of ART adherence, viral suppression (HIV-1 RNA < 400 and < 50 copies/mL), and mean change in CD4 lymphocyte count.

2.3. Sample size and power calculations

We calculated the sample size needed to detect the difference in primary outcomes with at least 80% power and a two-sided significance level of p<0.05. We made the following assumptions: (1) the proportion of subjects free from high risk behavior 36 weeks (T48) after the HHRP+ intervention would be 92% [17]; (2) alpha=0.05; (3) beta = 0.20; (4) equal allocation of participants among the two intervention conditions; and (5) 10% allowable non-inferiority difference between the HHRP+ and 3H+ interventions would be 82%. Using an intention-to-treat (ITT) analysis, 116 subjects per arm (N=232) will be required to confirm our hypothesis. With known attrition from our studies of HIV-infected PWUDs [(86% to 93% (average 91%)] being conservative - as subjects in this trial will also be on methadone - and conservatively allowing for 10% attrition, a total of 256 subjects, equally divided between 3H+ and HHRP+, will be required. Given that 232 participants will be available for analysis, we will have 81.5% power to test our secondary hypothesis of non-inferiority of 3H+ relative to HHRP+ when examining high-level ART adherence. For this secondary endpoint, we assume the proportion of participants achieving high-level ART adherence in the HHRP+ group will be 80% [17]. The 3H+ intervention will be considered non-inferior to HHRP+ if the percentage of participants achieving high-level ART adherence in the 3H+group is not less than a margin of δ0=15% vs. the HHRP+ group (i.e., not less than 65%) [2225].

2.4. Study procedures

2.4.1. Recruitment and screening

Recruitment started in 2012 and will continue until 2016 in New Haven, CT from community-based, non-research oriented programs and health services centers (e.g., homeless shelters, addiction treatment centers, medical clinics). Participants are recruited via peers, word-of-mouth and flyers, direct referral from counselors in addiction treatment programs, and HIV clinical care settings within the greater New Haven, CT area. Potential study participants are screened initially by phone or in person. During screening session, we determine whether the patient meets inclusion criteria, including: i) being HIV-infected; ii) meeting DSM-V criteria for opioid dependence and seeking methadone maintenance treatment; iii) reporting drug- or sex-related HIV risk behavior in the past 6-months; iv) age 18 or older and able to provide the informed consent; v) able to read and understand the questionnaires and the Audio Computer Assisted Self Interview (ACASI); vi) available for the full duration of the study with no anticipated circumstances impeding participation and vii) not actively suicidal, homicidal, or psychotic. Those meeting screening criteria are informed about the research study and scheduled an appointment with the research staff in a confidential setting to participate in the study.

2.4.2. Informed consent, enrollment, and reimbursement

Per standard Good Clinical Practice guidelines, the research assistant administers informed consent and assesses the participant’s willingness to enroll in the study at the initial in-person visit. After informed consent completion, all enrolled participants undergo a baseline assessment. The baseline assessment, which takes approximately 90 minutes to complete, is conducted prior to Week 1 of intervention participation and repeated following Week 12, and then again at 3- (T24), 6- (T36) and 9-(T48) month measurement points post intervention (see Figure 1).

During the enrollment process, participants are asked to complete release of information (ROI), allowing study staff to contact their community primary care providers and pharmacy for information regarding laboratory results (i.e., CD4 count and viral load) and prescription refill information, respectively. Study participants are reimbursed for the time required for them to complete assessments and weekly group sessions (see Table 1).

Table 1.

Study activity and measures

Study activity Study time point from day of screening (Weeks)
−1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 24 36 48
Study activity
 Screening for eligibility X
 Informed consent X
 Research interview X X X X X
 Randomization X
 ROI X X X X X
MINI-International Neuropsychiatric Interview X
Demographic information X X X X X
IMB measures for ART adherence X X X X X
Drug-related HIV risk reduction
 IMB measures X X X X X
 Reported drug use X X X X X X X X X X X X X X X X X
Sex-related HIV risk reduction
 IMB measures X X X X X
 Reported risky sexual practices X X X X X X X X X X X X X X X X X
Neuropsychological Impairment Scale assessment X X X X X
ART adherence
 Visual Analog Scale X X X X X
 Prescription refill X X X X X
HIV biological outcome measures
 HIV-1 RNA level X X X X X
 CD4 count X X X X X
Weekly groups
 Intervention group X X X X X
 Comparison group X X X X X X X X X X X X
Urine toxicology X X X X X X X X X X X X X X X X X
Payments $90 $25 $25 $25 $25 $25 $25 $25 $25 $25 $25 $25 $25 $90 $90 $90 $90

2.5. Randomization

Upon completing the baseline assessment, in addition to standard of care opioid replacement drug treatment, each participant is randomized to receive: (a) 12 weekly group sessions that comprise the HHRP+ intervention or (b) 4 weekly group sessions and a 12-week booster session that comprise the 3H+ intervention. Standard of drug treatment care involves routine services that are included in methadone maintenance treatment (daily methadone and case management). The clinicians supervising the MMP are blind to the intervention condition each participant is receiving. Randomization is done using a computerized “urn” randomization to ensure adequate representation of women and minorities to each intervention arm.

The research team members are keenly aware of the potential for contamination across study conditions. Contamination is being minimized in several ways: 1) intervention sessions associated with each condition are purposefully scheduled at different times and in different locations (but by the same intervention facilitators to control for the possible influence of employing different facilitators per condition); 2) participants in the two conditions are assessed independently and using ACASI, thus avoiding direct interaction and reducing social desirability bias; and 3) couples and housemates are randomized to the same condition. Thus, interactions among participants do not differ from the interactions that would naturally occur just by participating in the routine services provided by the drug treatment program.

2.6. Theoretical framework

Both the original (HHRP+) [17] and adapted version (3H+) [16] intervention are based on the information-motivation-behavioral skills (IMB) model of health behavior change [26, 27], which specifies that HIV prevention information, motivation, and behavioral skills are the fundamental determinants of HIV risk and HIV risk reduction behavior. From this perspective, HIV prevention information (i.e., knowledge) that is directly relevant to an individual’s practice of risk-reduction behavior is a prerequisite for engaging in risk-reduction action. Similarly, HIV prevention motivation to act on what one knows about HIV prevention – including personal motivation (attitudes about personally taking preventive actions) and social motivation (perceived social support/social pressure to take preventive actions) – is an additional prerequisite for the initiating and maintain HIV risk-reduction behavior. Finally, HIV prevention behavioral skills for taking HIV preventive actions are a third prerequisite for engaging in HIV risk-reduction behavior, and determine the extent to which an informed and motivated individual is capable of skillfully initiating and maintaining HIV risk reduction behavior [26, 27].

2.7. Description of the comparison and intervention conditions

2.7.1. HHRP+: Comparison group

The comparison intervention condition – the original version of HHRP+ [17] – has been identified by the CDC as an EBI for this specific risk population [18] and, as such, serves as a ‘gold standard’ among interventions targeting HIV-infected PWUDs in the context of drug treatment programs. It is composed of 12 two-hour weekly manual-guided group sessions with comprehensive HIV risk reduction content that addresses the medical, emotional, and spiritual needs of drug-involved individuals living with HIV (Table 2). Each session is designed to last 2 hours and is co-facilitated by two trained clinicians in a private room in a research site (i.e., APT Foundation MMP). Co-facilitators address potential motivational conflicts of HIV+ individuals by providing them with self-protective as well as altruistic reasons for examining and changing their HIV risk behavior and improving adherence behavior. Material is presented using cognitive remediation strategies [28] including a multi-modal presentation of material, behavioral games and role plays, frequent review of material, and use of memory books.

Table 2.

Outline of the HHRP+ HIV risk reduction and antiretroviral adherence intervention

Group Topics Information, Motivation, and Skills Taught
1. Reaching your goals Improving memory and concentration, setting goals, establishing priorities, action initiation
2. Reducing the harm of injection drug use Identifying the harm of injection drug user, learning harm reduction techniques (e.g., needle cleaning), reducing cue-elicited craving
3. Sexual harm reduction with latex Identifying the harm of unsafe sexual practices, learning harm reduction techniques (e.g., condom selection and application)
4. Negotiating harm reduction with partners Harm reduction negotiation and communication skills, eroticizing safer sexual practices
5. Preventing relapse to risky behavior Learning relapse prevention skills, identifying early warning signs, understanding seemingly irrelevant decisions
6. Health care participation Understanding HIV and immune system, strategies for improving health, developing a partnership with health care providers, and enhancing antiretroviral adherence skills
7. Healthy lifestyle choices Coping skills, stress management, nutritional guidelines and food hygiene
8. Introduction to the 12 steps Identifying what is and is not controllable, understanding when to let go and when to take action, identifying one’s personal source of strength, increasing motivation for change
9. Overcoming stigma Understanding the consequence of stigmatization, decreasing the strength of “addict” self-identity, identifying and strengthening cognitive, affective, and behavioral attributes of healthier, non-drug using lifestyle
10. Motivation: Overcoming helplessness Understanding the source and consequences of helplessness, identifying situations in which you can become empowered, assessing readiness for change, increasing motivation to pursue a healthy lifestyle
11. Moving beyond grief Understanding the stages of grief, facing and coping with fears about HIV, identifying and prioritizing that which has personal meaning
12. Healthy social relationships and activities Identifying and maintaining healthy social relationships, communicating HIV status, identifying and engaging in healthy social activities

2.7.2. 3H+: Intervention group

The 3H+ intervention was developed through an iterative process involving the integration of qualitative research [16], theoretical underpinnings [26, 27], and findings from a systematic review of the relevant research literature [29]. As an initial step toward developing an intervention with potential for wide-scale implementation in common clinical settings, we conducted a meta-analysis of RCTs evaluating HIV risk reduction interventions specifically targeting PWUDs [30]. The results of the meta-analysis showed that interventions were most beneficial when they: (1) were conducted in the context of drug treatment, (2) focused equally on drug- and sex-related HIV risk, (3) provided interpersonal skills training on safer injection practices with injection partners, and (4) were led by two facilitators [30]. We incorporated the findings from the meta-analysis into the design and adaptation of the 3H+ intervention [16].

Guided by the Assessment-Decision-Administration-Production-Topical experts-Integration-Training-Testing (ADAPT-ITT) model of intervention adaptation [31], we then conducted formative qualitative research, including structured focus group interviews with 12 treatment providers and stakeholders within the proposed research performance site (i.e., APT Foundation MMP) and with 21 members of the target population (i.e., HIV+ DUs). Our qualitative research questions were designed to elicit a range of responses from patients and providers that would collectively inform an intervention approach that could be rapidly and seamlessly integrated into similar/common drug treatment programs and be delivered as part of routine services. Thus, our qualitative research questions (available upon request) were focused accordingly.

Based on these qualitative data and data from our meta-analysis [30], we determined that the intervention would: (1) be brief and based on an EBI targeting HIV+ DUs (HHRP+ was the best fit), (2) consist of 4 weekly group sessions of one hour each, (3) contain content that focuses explicitly on drug- and sex-related risk reduction and ART adherence, and (4) be readily available to HIV+ PWUDs in drug treatment. We also deemed it critical to retain the delivery strategies of the original EBI (HHRP+) to accommodate the moderate levels of cognitive impairment that characterize our target population [32, 33]. Collectively, our formative research data informed our design of an approach that could be optimally implemented within similar real world drug treatment programs.

Based on the formative work, we conducted a preliminary pilot study of the 3H+ intervention among HIV-infected opioid-dependent persons who reported drug- or sex-related HIV risk behaviors and who were prescribed ART [16]. Improvements were found in the behavioral skills required to properly adhere to HIV medication regimens. Enhancements were found in all measured aspects of sex-risk reduction outcomes including HIV knowledge, motivation to reduce sex-risk behavior, behavioral skills related to engaging in reduced sexual risk, and reduced risk behavior. Improvements in drug use outcomes included enhancements in risk reduction skills as well as reduced heroin and cocaine use. The pilot test provided solid evidence of feasibility and preliminary evidence of efficacy of 3H+ intervention in terms of enhancing both sexual- and drug-related HIV risk reduction and ART adherence outcomes [16].

Building on the adaptation and refinement of original HHRP+ intervention, patients assigned to the 3H+ group receive four 60-minute weekly group sessions (see Table 3) and a 60-minute booster session at 12 weeks which is designed to review and maintain HIV risk reduction and ART adherence skills. Our theory-based [26, 27], manual-guided intervention is a modified coping skills training approach that is delivered in a group modality by two bachelor’s-level trained facilitators in a private room at a drug treatment facility (i.e., APT Foundation methadone maintenance program). Material is presented using a motivational enhancement therapeutic style to address high risk drug- and sex-related HIV risk behaviors and ART adherence [16]. Facilitators are supervised by the PI, a licensed clinical psychologist with extensive experience in all aspects of HHRP-based interventions among the target population. Facilitators will meet with the project director and the PI for weekly supervision. Importantly, it applies cognitive enhancement strategies such as multimodal presentation of the material, which allows us to accommodate the otherwise detrimental impact of cognitive impairment on intervention engagement and participation [16, 28].

Table 3.

Outline of the 3H+ HIV risk reduction and antiretroviral adherence intervention

Group Topics Information, Motivation, and Skills Taught
1. Health care participation Understanding HIV and immune system, strategies for improving health, developing a partnership with health care providers, and enhancing antiretroviral adherence skills
2. Reducing the harm of injection drug use Identifying the harm of injection drug use, learning harm reduction techniques (e.g., needle cleaning), reducing cue-elicited craving
3. Harm reduction with latex Identifying the harm of unsafe sexual practices, learning harm reduction techniques (e.g., condom selection and application)
4. Negotiating harm reduction with partners Harm reduction negotiation and communication skills, eroticizing safer sexual practices

2.8. Outcome measures

Participants are assessed at five standard time points (pre-intervention, post-intervention, and at 3, 6, and 9 months following the intervention). Research indicates that 3-month retrospective recall of HIV risk behavior is reliable [34]. Thus, we have staggered assessment points accordingly and have included 6- and 9-month follow-ups in order to examine the durability of the intervention as well as the decay or emergence of intervention effects. All of the self-report measures are administered using ACASI, an approach documented to reduce self-report biases [35, 36] and minimize participant response reactivity [37].

Framed by the IMB model of health behavior change [26, 27], we are assessing HIV risk reduction information, motivation, behavioral skills, and behavior. Table 1 displays a detailed description of the specific constructs measured including: Information (knowledge) about HIV and antiretroviral adherence (calculated as percent correct), Motivation to reduce HIV risk and adhere to antiretroviral medication-taking, and Behavioral skills about reducing HIV risk and improving antiretroviral adherence. The assessment also captures step-by-step hands-on demonstrations of sex-risk reduction skills (using male and female replicas) and drug-risk reduction skills (demonstrating proper needle cleaning), as well as reported event-level partner-by-partner sex- and drug-related HIV risk behavior (Table 1).

Further, ART adherence is being assessed using two additional approaches that are recommended by guidelines by the International Association of Physicians in AIDS Care (IAPAC) [38]: i) self-report using the visual analogue scale (VAS) [39] - and ii) pharmacy refill data. These self-reported measures have recently been compared to MEMS cap data [40]. The VAS is empirically validated, consistently under-reports adherence, but effectively measures a “difference” in adherence that changes in response to an intervention [39]. Pharmacy refill data, which include pharmacy refill records of the ART medications, will be used as a confirmatory adherence measures to validate the VAS. As used in prior studies, a release of information to obtain pharmacy data from both Medicaid and individual pharmacies will be deployed [16]. We will first assess the correlation (r) between VAS and pharmacy adherence data, particularly for defining adherence >95%. If the correlation is less than optimal (r<0.70), we will report both measures, but rely on pharmacy refill data to be most conservative. As an alternative approach, we will measure the absolute changes in adherence from baseline to the post-intervention (12 weeks), 3-(T24), 6-(T36) and 9- (T48) month follow-ups. Last, we will measure treatment non-persistence, using timeline follow-back assessments to examine ART treatment gaps.

HIV-1 RNA level (viral load, VL) is being measured in terms of mean change in VL from baseline to the final (T48) follow-up point. An additional approach includes examining the proportion of subjects with a non-detectable (<400 and <50 copies/mL) VL, and mean change in CD4 lymphocyte count. Four-panel (heroin, cocaine, oxycodone, and benzodiazepine) immunoassay (I/A) urinalyses (with confirmation of positive results) are conducted at baseline, weekly during the 12 week intervention phase, at post-intervention, and at follow-ups to detect the most common illicit substances of abuse in this patient population. Urine toxicology results will be used to validate the self-report use of substances.

2.8.1. Potential covariates

A number of covariates are being measured to examine the differential impact of certain variables known to influence either the primary or secondary outcomes. For instance, factors that have been associated with non-adherence have included: depressive symptoms, active drug use, neurocognitive impairment, destabilized living circumstances [33, 4143]. Similar covariates have been reported for HIV risk-taking behaviors. We therefore measure these using standardized instruments including: DSM-V criteria for substance use disorders (M.I.N.I) [44], mental illness (M.I.N.I.) [44], depressive symptoms (CES-D) [45], active drug use (urine toxicology screening using the NIDA-5 panel measuring heroin, cocaine, oxycodone, benzodiazepines and marijuana), and neurocognitive impairment (using the Neuropsychological Impairment Scale)[46].

2.9. Strategies to minimize attrition

To date, retention across the 3 follow up periods as of February 2015 is 84%, just below our expected range (86% to 93%). Using experience gained from past and current research, several steps have been taken to reduce the potential for attrition [4749]. These include rapid assignment (usually the same day) to study conditions after provision of informed consent, thorough explanation of study conditions, close monitoring of participants’ clinical status, integration of the research with the clinical program, and accessibility to patients of study staff for questions and problems. Participant locator forms, including name, address, phone number, family/friend contact are collected and updated at each visit. Phone calls are used in between appointments to update information. Research staff, with participant’s prior permission, call participants the week before the date of their assessment appointments. Appointment cards are also issued at each visit for subsequent visits. Furthermore, participants are reimbursed for the time and effort required for them to complete the assessments that we have found to be essential to retention [49]. Based on previous research using similar procedures, we expect to retain over 90% of participants over the entire study period (48 weeks).

2.10. Statistical analyses

Logistic regression models will be used to assess the log odds of being free from injection or sexual transmission risk behaviors at T48 (36 weeks after the end of intervention) using intervention assignment as a binary coded predictor while controlling for any potential confounds. Non-inferiority will be based on a one-sided test at α=.025 level. Equivalently, if the lower bound of the one-sided 97.5% confidence limit around the estimated difference in proportions lies above – δ0 (i.e., −10%), then the non-inferiority of the 3H+ intervention will be demonstrated. A traditional intent-to-treat (ITT) analysis tends to be biased toward non-inferiority [50] when non-adherence to protocol is high. Therefore, data from our study will be analyzed using both an ITT and per-protocol approach. Similar logistic regression analyses will be conducted for the secondary endpoint, proportion achieving high adherence.

In addition, we will perform 2 (intervention condition) x 5 (repeated follow-up assessments) mixed design analyses, entering pre-intervention scores as covariates and additional theoretically and empirically relevant covariates as possible moderators, including interactions, or mediators. The specific statistical procedures that we employ will be determined by inspection of the distributions of variables in relation to the assumptions of the statistical tests. For variables that are approximately normally distributed, we will use multivariate analyses of covariance for conceptually related sets of variables, with significant multivariate tests followed by subsequent analyses of covariance. We anticipate including the IMB theoretical constructs (e.g., risk reduction knowledge, readiness to change, behavioral intentions, self-efficacy) in these analyses because these variables have approximated normal distributions in previous research [49].

Using growth curve mediation modeling in Mplus [51] and mediation package from R [52, 53], we will test the relationship between study conditions and the risk behavior outcome variables mediated by changes in the theoretical variables of interest (variables based on IMB model) over time, assuming the time points cluster within individuals. A causal inference framework will be used when identification assumptions need to be relaxed [54, 55]. The possible mediators will be included in the final model if they showed a significant intervention effect when tested as an outcome. Using standardized path coefficients, we will examine the relative associations of each mediator. Variables that violate distributional assumptions of normality will be analyzed using generalized linear model (GLM) procedures, which enable the use of non-Gaussian error models. These analyses enable the use of the Poisson or negative-binomial error model for study endpoints that represent frequency of an occurrence or practice, such as rates of HIV risk and protective behaviors. While we do not expect differential rates of dropout between groups or high loss to follow-up at 9 months, sensitivity analysis using several forms of imputation (e.g., multiple imputation) will be performed to examine the robustness of conclusions from the primary analysis [5658].

3. Cost-effectiveness analysis

Understanding the potential population-wide health gains and costs of secondary HIV prevention is an important consideration for policymakers tasked with allocating scarce program resources. In preparation for our final cost-effectiveness analysis, our team has published a customized HIV epidemic model evaluating the relative epidemic impact and cost-effectiveness of HHRP+ vs. 3H+ for HIV-infected PWUDs in the U.S. [59]. In the absence of final outcomes from our ongoing RCT (115 of 256 projected participants recruited to date), we estimated the projected health benefits and costs of implementing HHRP+ vs. 3H+, through use of a mathematical epidemic model, at various levels of implementation, based on the results of the original HHRP studies which were compared to treatment as usual (TAU) and information from databases that are widely accepted for this purpose and available to the public (e.g. CDC HIV prevalence rates, WHO guidelines, US Bureau of Labor Statistics, etc.). The model used HIV transmission probabilities and contact rates among PWUDs and between PWUDs and the general adult population to project future HIV incidence, prevalence, and HIV-related mortality over a 10-year horizon [59].

Briefly, our study projected that both behavioral intervention programs targeting HIV-infected PWUDs (HHRP+ or 3H+) would be effective and cost-effective in terms of reducing HIV incidence among PWUDs as well as the general population of adults in the US [59]. The 3H+ intervention was projected to be more effective than HHRP+, however, because of its cost-effective attributes such as being much briefer, easy to implement in clinical settings, and having a greater likelihood of being more widely and correctly implemented. In general, findings point to the potential population-wide benefits of 3H+ as a more cost-effective alternative to HHRP+ when used in conjunction with opioid replacement therapy in a clinical context.

4. Implementation issues

When working with HIV-infected PWUDs, there are unique implementation and logistical concerns that require considerable experience, logistical problem-solving, and ethical oversight. Participant recruitment has been one of the most challenging aspects of this ongoing study. Recruitment for this type of study can present significant challenges as potential participants must meet relatively stringent inclusion criteria. In addition, the use of convenience sampling from the greater New Haven area – as opposed to attempting, for example, systematic sampling from multiple facilities in the area – may have required additional recruitment efforts. Furthermore, this study involves delivering a relatively intensive intervention (HHRP+), multiple assessment points, and an extended follow-up period, and any of these factors may be expected to influence participation over time. In addition, it is necessary that all referral sources clearly understand and communicate study inclusion/exclusion criteria.

In an attempt to maximize our recruitment rate, we have identified a number of strategies. First, we have expanded our recruitment radius to include towns surrounding the New Haven area. The research team conducts in-service meetings with physicians, nurses, and counselors at local HIV clinics, addiction treatment centers, and medical groups to identify and refer potential clients. During the presentations, research personnel distribute packets of information sheet contacting brief descriptions of the study, inclusion and exclusion criteria, and contact information. Second, the research staff members conduct community outreach, and post flyers in public venues known to be higher-density activity areas for the target population of high risk drug users. Third, with the expansion of recruitment radius, transportation has emerged as an important logistical factor, with participants having to frequently commute some distance to the research site. We therefore provide transportation vouchers or reimburse participants for the burden imposed by travel (via bus passes, train tickets, tokens). In conjunction with the University of Connecticut IRB, careful consideration was given to the market rate for similar studies in this area in order to ensure that we are respectful of the time required for patients to fully participate.

5. Summary

Despite decades of HIV prevention strategies targeting high risk populations, a significant number of HIV-infected PWUDs continue to engage in behaviors that transmit HIV and that place them at greater risk of acquiring other infections that may contribute to disease progression [3]. In addition, problematic ART adherence is well documented in this risk population. A range of evidence-based HIV risk reduction interventions are widely available, but few – if any – were designed for implementation within clinical settings, such as MMP, where high-risk DUs commonly seek treatment services. Driven by the dearth of EBIs that are applicable to high-risk HIV-infected PWUDs in treatment, we designed the 3H+ intervention [16] – an adapted version of the comprehensive evidence-based HHRP+ [17]. The systematic adaptation and testing process described here was designed to more effectively address the HIV prevention needs among HIV-infected PWUDs within real-world clinical settings such as drug treatment programs.

To our knowledge, this is the first study to use a non-inferiority RCT to examine the relative efficacy and cost-effectiveness of an adapted, brief, version of an EBI vs. an original EBI targeting HIV-infected PWUDs, and it provides an exemplar for building on the existing intervention science. This study was designed to provide good internal validity by controlling known confounders, by having a target sample powered to detect the difference in primary outcome using similar sample, using validated measurements, and by using statistical methods that have been proven to reduce Type I and Type II errors. This research is especially crucial given the disproportionate HIV/AIDS burden borne by HIV-infected PWUDs and the gap currently existing in HIV/AIDS prevention strategies targeting DUs. If the 3H+ intervention shows relative efficacy and cost-effectiveness vs. HHRP+, as predicted, it can be readily disseminated for implementation as part of routine care within common drug treatment programs – a true integration of HIV prevention science and drug treatment services. Furthermore, the process outlined herein could serve as a model for adapting other EBIs for optimal implementation in a range of real-world clinical settings (e.g. prisons, HIV clinical care sites, etc.).

Acknowledgments

The funding for this research is provided by the National Institutes on Drug Abuse for research (R01 DA032290 to MMC) and for career development (K02 033139 to MMC; K24 DA017072 to FLA). The authors would also like to thank the study participants, research staff, and all collaborating agencies for making this study possible.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Centers for Disease Control and P. HIV Surveillance Report, 2012. Vol. 24. Centers for Disease Control and Prevention; 2014. p. 83. [Google Scholar]
  • 2.Avants SK, Margolin A, Usubiaga MH, Doebrick C. Targeting HIV-related outcomes with intravenous drug users maintained on methadone: a randomized clinical trial of a harm reduction group therapy. Journal of substance abuse treatment. 2004;26:67–78. doi: 10.1016/S0740-5472(03)00159-4. [DOI] [PubMed] [Google Scholar]
  • 3.Volkow ND, Montaner J. The urgency of providing comprehensive and integrated treatment for substance abusers with HIV. Health affairs (Project Hope) 2011;30:1411–9. doi: 10.1377/hlthaff.2011.0663. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Strathdee SA, Hallett TB, Bobrova N, Rhodes T, Booth R, Abdool R, et al. HIV and risk environment for injecting drug users: the past, present, and future. The Lancet. 376:268–84. doi: 10.1016/S0140-6736(10)60743-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Noar S. Behavioral Interventions to Reduce HIV-related Sexual Risk Behavior: Review and Synthesis of Meta-Analytic Evidence. AIDS and Behavior. 2008;12:335–53. doi: 10.1007/s10461-007-9313-9. [DOI] [PubMed] [Google Scholar]
  • 6.Arasteh K, Jarlais DCD, Perlis TE. Alcohol and HIV sexual risk behaviors among injection drug users. Drug & Alcohol Dependence. 95:54–61. doi: 10.1016/j.drugalcdep.2007.12.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Dutta A, Wirtz AL, Baral S, Beyrer C, Cleghorn FR. Key harm reduction interventions and their impact on the reduction of risky behavior and HIV incidence among people who inject drugs in low-income and middle-income countries. Current Opinion in HIV and AIDS. 2012;7:362–8. doi: 10.1097/COH.0b013e328354a0b5. [DOI] [PubMed] [Google Scholar]
  • 8.Marshall BDL, Friedman SR, Monteiro JFG, Paczkowski M, Tempalski B, Pouget ER, et al. Prevention And Treatment Produced Large Decreases In HIV Incidence In A Model Of People Who Inject Drugs. Health Affairs. 2014;33:401–9. doi: 10.1377/hlthaff.2013.0824. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Morgenstern J, Morgan TJ, McCrady BS, Keller DS, Carroll KM. Manual-guided cognitive-behavioral therapy training: a promising method for disseminating empirically supported substance abuse treatments to the practice community. Psychology of addictive behaviors : journal of the Society of Psychologists in Addictive Behaviors. 2001;15:83–8. [PubMed] [Google Scholar]
  • 10.Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of consulting and clinical psychology. 2005;73:106–15. doi: 10.1037/0022-006X.73.1.106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Safren SA, Wingood G, Altice FL. Strategies for primary HIV prevention that target behavioral change. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2007;45 (Suppl 4):S300–7. doi: 10.1086/522554. [DOI] [PubMed] [Google Scholar]
  • 12.Fisher JD, Cornman DH, Osborn CY, Amico KR, Fisher WA, Friedland GA. Clinician-initiated HIV risk reduction intervention for HIV-positive persons: Formative Research, Acceptability, and Fidelity of the Options Project. Journal of acquired immune deficiency syndromes (1999) 2004;37 (Suppl 2):S78–87. doi: 10.1097/01.qai.0000140605.51640.5c. [DOI] [PubMed] [Google Scholar]
  • 13.Kalichman SC, Rompa D, Cage M, DiFonzo K, Simpson D, Austin J, et al. Effectiveness of an intervention to reduce HIV transmission risks in HIV-positive people. American Journal of Preventive Medicine. 2001;21:84–92. doi: 10.1016/s0749-3797(01)00324-5. [DOI] [PubMed] [Google Scholar]
  • 14.Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365:493–505. doi: 10.1056/NEJMoa1105243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Wood E, Milloy MJ, Montaner JS. HIV treatment as prevention among injection drug users. Curr Opin HIV AIDS. 2012;7:151–6. doi: 10.1097/COH.0b013e32834f9927. [DOI] [PubMed] [Google Scholar]
  • 16.Copenhaver MM, Lee IC, Margolin A, Bruce RD, Altice FL. Testing an optimized community-based human immunodeficiency virus (HIV) risk reduction and antiretroviral adherence intervention for HIV-infected injection drug users. Substance abuse : official publication of the Association for Medical Education and Research in Substance Abuse. 2011;32:16–26. doi: 10.1080/08897077.2011.540466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Margolin A, Avants SK, Warburton LA, Hawkins KA, Shi J. A randomized clinical trial of a manual-guided risk reduction intervention for HIV-positive injection drug users. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2003;22:223–8. [PubMed] [Google Scholar]
  • 18.Centers for Disease Control and P. Holistic Health Recovery Program (HHRP) 2003. [Google Scholar]
  • 19.D’Agostino RB, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues – the encounters of academic consultants in statistics. Statistics in Medicine. 2003;22:169–86. doi: 10.1002/sim.1425. [DOI] [PubMed] [Google Scholar]
  • 20.Schumi J, Wittes J. Through the looking glass: understanding non-inferiority. Trials. 2011;12:106. doi: 10.1186/1745-6215-12-106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Grobler AC, Karim SSA. Design challenges facing clinical trials of the effectiveness of new HIV prevention technologies. AIDS (London, England) 2012;26:529–32. doi: 10.1097/QAD.0b013e3283509a29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Altice FL, Maru DS, Bruce RD, Springer SA, Friedland GH. Superiority of directly administered antiretroviral therapy over self-administered therapy among HIV-infected drug users: a prospective, randomized, controlled trial. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2007;45:770–8. doi: 10.1086/521166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Simoni JM, Frick PA, Pantalone DW, Turner BJ. Antiretroviral adherence interventions: a review of current literature and ongoing studies. Topics in HIV medicine : a publication of the International AIDS Society, USA. 2003;11:185–98. [PubMed] [Google Scholar]
  • 24.Simoni JM, Frick PA, Huang B. A longitudinal evaluation of a social support model of medication adherence among HIV-positive men and women on antiretroviral therapy. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2006;25:74–81. doi: 10.1037/0278-6133.25.1.74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Simoni JM, Pantalone DW, Plummer MD, Huang B. A randomized controlled trial of a peer support intervention targeting antiretroviral medication adherence and depressive symptomatology in HIV-positive men and women. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2007;26:488–95. doi: 10.1037/0278-6133.26.4.488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Fisher JD, Fisher WA. Changing AIDS-risk behavior. Psychological bulletin. 1992;111:455–74. doi: 10.1037/0033-2909.111.3.455. [DOI] [PubMed] [Google Scholar]
  • 27.Fisher JD, Fisher WA, Amico KR, Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2006;25:462–73. doi: 10.1037/0278-6133.25.4.462. [DOI] [PubMed] [Google Scholar]
  • 28.Copenhaver M, Avants SK, Warburton LA, Margolin A. Intervening effectively with drug abusers infected with HIV: taking into account the potential for cognitive impairment. Journal of psychoactive drugs. 2003;35:209–18. doi: 10.1080/02791072.2003.10400002. [DOI] [PubMed] [Google Scholar]
  • 29.Copenhaver MM, Lee IC. Optimizing a community-friendly HIV risk reduction intervention for injection drug users in treatment: a structural equation modeling approach. Journal of urban health : bulletin of the New York Academy of Medicine. 2006;83:1132–42. doi: 10.1007/s11524-006-9090-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Copenhaver MM, Johnson BT, Lee IC, Harman JJ, Carey MP, Team SR. Behavioral HIV risk reduction among people who inject drugs: meta-analytic evidence of efficacy. Journal of substance abuse treatment. 2006;31:163–71. doi: 10.1016/j.jsat.2006.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Wingood GM, DiClemente RJ. The ADAPT-ITT model: a novel method of adapting evidence-based HIV Interventions. Journal of acquired immune deficiency syndromes (1999) 2008;47 (Suppl 1):S40–6. doi: 10.1097/QAI.0b013e3181605df1. [DOI] [PubMed] [Google Scholar]
  • 32.Shrestha R, Huedo-Medina TB, Copenhaver MM. Sex-Related Differences in Self-Reported Neurocognitive Impairment among High-Risk Cocaine Users in Methadone Maintenance Treatment Program. Substance Abuse: Research and Treatment. 2015;9:17–24. doi: 10.4137/SART.S23332. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Anand P, Springer SA, Copenhaver MM, Altice FL. Neurocognitive Impairment and HIV Risk Factors: A Reciprocal Relationship. AIDS and behavior. 2010;14:1213–26. doi: 10.1007/s10461-010-9684-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Schroder KE, Carey M, Vanable P. Methodological challenges in research on sexual risk behavior: I. Item content, scaling, and data analytical options. Annals of Behavioral Medicine. 2003;26:76–103. doi: 10.1207/s15324796abm2602_02. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Turner CF, Rogers SM, Hendershot TP, Miller HG, Thornberry JP. Improving representation of linguistic minorities in health surveys. Public health reports (Washington, DC: 1974) 1996;111:276–9. [PMC free article] [PubMed] [Google Scholar]
  • 36.Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science (New York, NY) 1998;280:867–73. doi: 10.1126/science.280.5365.867. [DOI] [PubMed] [Google Scholar]
  • 37.Kalichman SC, Stevenson LY. Psychological and social factors associated with histories of risk for human immunodeficiency virus infection among African-American inner-city women. J Womens Health. 1997;6:209–17. doi: 10.1089/jwh.1997.6.209. [DOI] [PubMed] [Google Scholar]
  • 38.Thompson MA, Mugavero MJ, Amico KR, Cargill VA, Chang LW, Gross R, et al. Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Ann Intern Med. 2012;156:817–33. doi: 10.7326/0003-4819-156-11-201206050-00419. W-284, W-5, W-6, W-7, W-8, W-9, W-90, W-91, W-92, W-93, W-94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Giordano TP, Guzman D, Clark R, Charlebois ED, Bangsberg DR. Measuring adherence to antiretroviral therapy in a diverse population using a visual analogue scale. HIV clinical trials. 2004;5:74–9. doi: 10.1310/JFXH-G3X2-EYM6-D6UG. [DOI] [PubMed] [Google Scholar]
  • 40.Simoni JM, Huh D, Wang Y, Wilson IB, Reynolds NR, Remien RH, et al. The validity of self-reported medication adherence as an outcome in clinical trials of adherence-promotion interventions: Findings from the MACH14 study. AIDS Behav. 2014;18:2285–90. doi: 10.1007/s10461-014-0905-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Altice FL, Kamarulzaman A, Soriano VV, Schechter M, Friedland GH. Treatment of medical, psychiatric, and substance-use comorbidities in people infected with HIV who use drugs. Lancet. 2010;376:367–87. doi: 10.1016/S0140-6736(10)60829-X. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Langebeek N, Gisolf EH, Reiss P, Vervoort SC, Hafsteinsdottir TB, Richter C, et al. Predictors and correlates of adherence to combination antiretroviral therapy (ART) for chronic HIV infection: a meta-analysis. BMC medicine. 2014;12:142–1453408941291432. doi: 10.1186/s12916-014-0142-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Ferro EG, Weikum D, Vagenas P, Copenhaver MM, Gonzales P, Peinado J, et al. Alcohol use disorders negatively influence antiretroviral medication adherence among men who have sex with men in Peru. AIDS Care. 2015;27:93–104. doi: 10.1080/09540121.2014.963013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Sheehan DV, Lecrubier Y, Harnett Sheehan K, Janavs J, Weiller E, Keskiner A, et al. The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. European Psychiatry. 1997;12:232–41. [Google Scholar]
  • 45.Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied psychological measurement. 1977;1:385–401. [Google Scholar]
  • 46.O’Donnell W, DeSoto C, Desoto JRD. The Neuropsychological Impairment Scale (NIS) manual. Los Angeles, USA: Western Psychological Services; 1994. [Google Scholar]
  • 47.Zweben A, Barret D, Carty K, McRee B, Morse P, Rice C. Strategies for facilitating protocol compliance in alcoholism treatment research. Bethesda, MD: National institute on Alcohol Abuse and Alcoholism; 1998. [Google Scholar]
  • 48.Carroll KM, Nich C, Rounsaville BJ. Contribution of the therapeutic alliance to outcome in active versus control psychotherapies. Journal of Consulting and Clinical Psychology. 1997;65:510–4. doi: 10.1037//0022-006x.65.3.510. [DOI] [PubMed] [Google Scholar]
  • 49.Copenhaver MM, Lee IC, Baldwin P. A randomized controlled trial of the community-friendly health recovery program (CHRP) among high-risk drug users in treatment. AIDS and behavior. 2013;17:2902–13. doi: 10.1007/s10461-013-0539-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Snapinn SM. Noninferiority trials. Current controlled trials in cardiovascular medicine. 2000;1:19–21. doi: 10.1186/cvm-1-1-019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Muthén LK, Muthén BO. Mplus user’s guide. 7. Los Angeles, CT: Muthén & Muthén; 1998–2015. [Google Scholar]
  • 52.R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2014. [Google Scholar]
  • 53.Tingley D, Yamamoto T, Hirose K, Keele L, Imai K. Mediation: R package for causal mediation analysis. Journal of Statistical Software. 2014;59:1–38. [Google Scholar]
  • 54.Imai K, Keele L, Yamamoto T. Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science. 2010:51–71.
  • 55.Imai K, Tingley D, Yamamoto T. Experimental designs for identifying causal mechanisms. Journal of the Royal Statistical Society: Series A (Statistics in Society) 2013;176:5–51. [Google Scholar]
  • 56.Rubin DB. Formalizing Subjective Notions About the Effect of Nonrespondents in Sample Surveys. Journal of the American Statistical Association. 1977;72:538–43. [Google Scholar]
  • 57.Rubin DB. Multiple imputation for nonresponse in surveys. New York, USA: John Wiley & Sons; 2009. [Google Scholar]
  • 58.Schafer JL. Analysis of incomplete multivariate data. New York, USA: CRC press; 2010. [Google Scholar]
  • 59.Song DL, Altice FL, Copenhaver MM, Long EF. Cost-Effectiveness Analysis of Brief and Expanded Evidence-Based Risk Reduction Interventions for HIV-Infected People Who Inject Drugs in the United States. PLoS ONE. 2015;10:e0116694. doi: 10.1371/journal.pone.0116694. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES