Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: Arch Sex Behav. 2019 Oct 19;49(1):195–209. doi: 10.1007/s10508-019-01515-x

Stimulant use and study protocol completion: Assessing the ability of men who have sex with men to collect dried blood spots for laboratory measurement of HIV viral load

Richard A Teran 1, Adam W Carrico 2, Keith J Horvath 3, Martin J Downing Jr 4, Mary Ann Chiasson 1,5, Suzan M Walters 6, Sabina Hirshfield 7
PMCID: PMC7018572  NIHMSID: NIHMS1541300  PMID: 31630286

Abstract

Stimulant use is associated with higher HIV viral load (VL) and sexual HIV transmission risk among men who have sex with men (MSM) living with HIV. There is little research on willingness of drug users living with HIV to fully participate in studies, especially those involving self-collection of biomarker data. This study presents findings from an at-home dried blood spot collection study measuring laboratory quantified VL among US HIV-positive MSM who reported high-risk sexual behavior and/or suboptimal antiretroviral therapy (ART) adherence to assess the association between drug use behavior and (1) ability to complete a study protocol and (2) VL outcomes. Among recruited participants (n=766), 35% reported stimulant drug use (amphetamines, cocaine, crack, crystal meth, ecstasy, or a combination of stimulant drugs), 39% reported using other drugs (heroin, marijuana, prescription opioids, and others), and 27% reported no drug use in the past 3-months. In all, 61% of enrolled participants completed the study protocol. Stimulant drug users were less likely (ARR, 0.84; 95% CI, 0.72 – 0.98) to complete the protocol than other drug users. Furthermore, other drug users were significantly less likely than non-other drug users (ARR, 0.52; 95% CI, 0.28 – 0.97) to have an HIV VL result ≥1,500 copies/mL. This study provides important estimates regarding the likelihood of participation in biomedical research activities among HIV-positive MSM with varying drug use behaviors, showing that it is feasible to conduct such biomedical studies with drug-using MSM who report high-risk sexual behavior and struggle with their ART adherence.

Keywords: HIV-1, Viral Load, Dried Blood Spot Testing, Men Who Have Sex with Men, Stimulant drug use

Introduction

The prevalence of stimulant drug use, including crystal methamphetamine (crystal meth), is substantially elevated among men who have sex with men (MSM) in the United States (Buchacz et al., 2005; Colfax et al., 2005; Mansergh et al., 2006; McCarty-Caplan, Jantz, & Swartz, 2014; Santos et al., 2013). For MSM living with HIV, stimulant use is associated with significant disengagement in steps across the HIV care continuum. Specifically, studies have reported that HIV-positive MSM stimulant users are less likely to be engaged in HIV care (Horvath et al., 2013), less likely to be adherent to antiretroviral therapy (ART) (Carrico et al., 2011; Mayer, Skeer, O’Cleirigh, Goshe, & Safren, 2014; Stall & Purcell, 2000), and less likely to have suppressed HIV viral load (VL) (Carrico et al., 2019; Ellis et al., 2003; Morin et al., 2007). Given the evidence suggesting that HIV-positive individuals who are on ART and achieve and maintain an undetectable viral load cannot transmit the virus to others (Eisinger, Dieffenbach, & Fauci, 2019; McCray & Mermin, 2017; The Lancet HIV, 2017), it is imperative that all MSM living with HIV, including stimulant users, achieve viral suppression. Several studies also report increased sexual risk behaviors such as receptive condomless anal sex (CAS) and exchange sex among stimulant-using MSM (Gamarel, Woolf-King, Carrico, Neilands, & Johnson, 2015; Shoptaw & Reback, 2007; Walters et al., 2018) – which further magnify the concerns about continued HIV transmission resulting from individuals with unsuppressed viremia in this population.

Monitoring progress along the HIV care continuum – and more specifically, the achievement of viral suppression among stimulant-using HIV-positive MSM is critical. Biomedical, clinical, and observational research provide an optimal setting to study factors associated with achieving (or not achieving) viral suppression among this population. However, HIV-positive MSM who use stimulants face personal and structural barriers to participating in biomedical research. Stimulant-using MSM living with HIV experience multiple, overlapping psychosocial health problems and experience impairments related to stimulant use disorders that may partially explain higher rates of attrition in clinical research studies (Colfax & Shoptaw, 2005; Fleury, Grenier, Bamvita, Perreault, & Caron, 2015). In addition, active substance users or those with substance use disorders are frequently excluded from HIV biomedical research protocols due to concerns about participation and attrition (Cohen et al., 2011; Grant et al., 2010). Many studies have documented attrition issues among substance users (Brown et al., 2006; Cattie et al., 2015; Claus, Kindleberger, & Dugan, 2002; Cottler, Compton, Ben-Abdallah, Horne, & Claverie, 1996); however, there is little research on study activity behaviors among drug users living with HIV, particularly in biomedical research. Furthermore, home-based collection of biomarker data, such as dried blood spots (DBS), has been found to be feasible among MSM and many have proposed that these methods may address potential retention and engagement barriers in prevention research and HIV care (Gilbert et al., 2013; Hall, Ricca, Khosropour, & Sullivan, 2017; Sharma, Stephenson, White, & Sullivan, 2014; Sharma, Sullivan, & Khosropour, 2011). In addition, studies have also shown a willingness among drug users to provide biological specimens (e.g., hair, nails urine) and researchers have recommended that all drug use-related epidemiological studies include biomarker screening, when feasible (Cappelle et al., 2015; Li, Janulis, & Mustanski, 2019; Palamar, Salomone, Cleland, & Sherman, 2018).

This study presents findings from an at-home DBS collection study for laboratory (lab) quantification of VL among an online sample of US HIV-positive MSM with a history of suboptimal ART adherence and CAS with serodiscordant partners. The goals for this study were to (1) assess whether drug use (i.e., stimulant drug users, other drug users, and non-drug users) affects how an individual participates in biomedical research involving at-home DBS collection and (2) assess differences in DBS specimen HIV-1 RNA results by drug use groups.

Methods

Study recruitment

Participants for The Mailed Spot (M-Spot) study were recruited following their completion of the Sex Positive! study (parent study), a national online behavioral intervention. The parent study protocol has been described previously (Hirshfield et al., 2016). Eligible participants in the parent study self-reported having CAS with an HIV-negative or unknown status male partner in the past six months and being: (a) biologically male and identified as male or genderqueer; (b) ≥18 years of age; (c) white, black, or Hispanic; (d) able to read and respond in English; (e) a U.S. resident; (f) HIV-positive; (g) not virally suppressed (≥200 copies/mL) in the past year or reported past-month suboptimal ART adherence. Only white, black, and Hispanic MSM were recruited for the parent study as these three racial/ethnic groups comprise the majority of HIV infections in the U.S. After completing the parent study’s 12-month survey, men received a study recruitment email about the M-Spot study. Men recruited from the parent study were further screened and those diagnosed with hemophilia, or who were currently taking anticoagulation medication, were excluded.

Study procedures

The M-Spot study assessed the feasibility of home self-collection of DBS specimens for lab quantification of VL. Procedures for the M-Spot study have previously been reported (Hirshfield et al., 2018). To summarize, between September 2016 and February 2017, consenting participants were mailed a package containing a DBS kit, collection materials, and a return envelope with postage. After using a lancet and applying a blood sample on the DBS kit (herein referred to as a ‘DBS specimen’), men completed a brief online survey (herein referred to as the ‘M-Spot survey’), which collectedrecent HIV care information. Men mailed their DBS specimen to a research lab. Upon lab receipt, DBS specimens were stored for up to 4 months at 4 degrees Celsius before testing. If a DBS kit was half-filled with blood or not filled at all, the DBS specimen was deemed untestable. If a participant returned a DBS specimen with enough blood to be tested, they were deemed to have completed the study protocol. Specimens were tested in batches corresponding to receipt date. Previous studies have found high correlation between VL results from DBS specimens and plasma samples (Alvarez-Muñoz et al., 2005; Cassol et al., 1997; Garrido et al., 2009), and that HIV serological markers in samples collected using Hemaspot kits are stable to varying temperatures and protected from specimen degradation (Manak et al., 2018). Participants received a $15 electronic Amazon gift code for returning the DBS specimen and another $15 code for completing the M-Spot survey.

Viral load results from DBS specimens fell into one of three categories: (1) ‘Not Detected’ if no HIV-1 RNA was detected in the specimen by the assay, indicating an undetectable viral load; (2) a qualitative ‘≤832 copies/mL’ result when fewer than or equal to 832 copies/mL of HIV-1 RNA were detected; and (3) a quantitative result when HIV-1 RNA was detectable above 832 copies/mL. Of note, a qualitative result was used to describe specimens with detectable HIV-1 RNA that was ≤832 copies/mL because the amount of viremia in these specimens was outside the 95% confidence limit of the assay and could not be accurately quantified. Further, a lower limit of quantification was not reported by the manufacturer for the research assay used as there is a low probability of reproducibility when specimens have ≤832 copies/mL.

Ethics Statement

The Institutional Review Board at Public Health Solutions approved all study procedures. The Institutional Review Board at Johns Hopkins University approved all lab-related procedures. Participants provided consent by clicking a button at the end of an online consent form to indicate that they had read the consent page and agreed to participate. A Certificate of Confidentiality was obtained from the National Institute of Mental Health to protect the privacy of HIV-positive participants enrolled in this study.

Study Measures

Data from the M-Spot survey were merged with data from the parent study’s screener, 9-month, and 12-month follow-up surveys (herein referred to as ‘parent screener,’ ‘9-month parent survey’ and ‘12-month parent survey,’ respectively). Demographic measures were primarily collected from the parent screener; HIV care and adherence measures for this analysis were primarily collected from the 12-month parent survey. If a participant reported having an HIV care visit in the time between completing the 12-month parent survey and enrolling in the M-Spot study, HIV care data on that care visit was collected from the M-Spot survey. Specimen collection data, feasibility, and process-related data were collected from the M-Spot Survey.

Participant characteristics

The parent screener included questions to assess a participant’s age, race and ethnicity, gender identity, and sex at birth. The 12-month parent survey assessed level of education, employment status, and annual income. A “prefer not to answer” response option was available for questions on race and ethnicity, gender identity, and sex at birth. Participants selecting this response option on the parent screener survey were excluded from the study per eligibility criteria. All “prefer not to answer” or “I don’t know” responses for measures described below on the 12-month survey and M-Spot survey were treated as missing.

Alcohol use

Participants completed the 3-item version of the Alcohol Use Disorders Identification Test in the 12-month parent survey. The screening instrument identifies individuals who are heavy drinkers or have active alcohol use disorders (Bush et al., 1998). Each item was measured on a 5-point scale (0–4). Responses were totaled across the 3 items, with a score of ≥4 indicating a possible alcohol use disorder. Participants responding “prefer not to answer” to any of the 3 items were coded as missing and responses were not totaled across the 3 items.

Sexual Risk Behavior

Participants were asked in the 12-month parent survey about their number of male anal sex partners for the past 3 months. Pull-down menus listed 0 through 100 partners, 101+ partners, I don’t know, and prefer not to answer. Responses were categorized to 0 partners, 1–4 partners, and ≥5 partners.

Among participants reporting one or more male anal sex partners, serodiscordant anal sex with a male partner was assessed using additional encounter-level questions for up to the last three partners in the past 3 months. For each partner, participants were asked to report their partner’s serostatus at the last encounter. If participants reported more than 3 partners, they were asked to indicate how many of these men were HIV-negative or of unknown status using a pull down a pull-down menu listed 0 through 100 partners, 101+ partners, I don’t know, and prefer not to answer. We defined ‘no serodiscordant partners’ as having only HIV-positive male anal sex partners in the past 3 months. Men reporting known HIV-negative or unknown status male anal sex partners were defined as having ‘confirmed/unknown serodiscordant partners.’

Condomless anal sex was assessed among men reporting confirmed or unknown serodiscordant partners. Encounter-level CAS for the three most recent partners in the past 3 months was assessed by several items which asked the participant about insertive and receptive anal sex with their partner and whether or not a condom was used. If participants reported more than 3 partners, they were asked to indicate how many men they had insertive anal sex without a condom using a pull-down menu that listed 0 through 100 partners, 101+ partners, I don’t know, and prefer not to answer. Men reporting no condom use in the encounter-level CAS questions, or in the pull-down menu if they had 4 or more partners were defined as having serodiscordant CAS with a male partner in the past 3 months.

Sexually transmitted infections (STIs), other than HIV, occurring in the past 3 months were assessed in the 12-month parent survey. Participants were asked “in the last 3 months, were you diagnosed with any of the following?”. Participants responded Yes or No to a list of infections, including chancroid(s), Chlamydia, Gonorrhea, Hep A, Hep B, Hep C, Herpes-Genital, HPV – Genital/Anal Warts, Lymphogranuloma Venereum, non-Gonococcal urethritis, and Syphilis. Any participant responding yes to any of the infections were classified as having an STI diagnosis in the past 3 months.

Antiretroviral therapy adherence

Participants were asked in the 12-month parent survey about their current use of antiretroviral therapy (ART) medications (yes, no). Among participants on treatment, past 30-day adherence to ART was assessed using a 3-item scale (Wilson, Lee, Michaud, Fowler, & Rogers, 2016). Participants were asked: “In the last 30 days, on how many days did you miss at least one dose of any of your HIV medicines?” (pull down 0–30 days, I don’t know, prefer not to answer); “In the last 30 days, how good a job did you do at taking your HIV medicines in the way you were supposed to?” (very poor, poor, fair, good, very good, excellent, prefer not to answer); and “In the last 30 days, how often did you take your HIV medicines in the way you were supposed to?” (never, rarely, sometimes, usually, almost always, always). Participants responding “I don’t know” or “prefer not to answer” to any of the items were coded as missing. Responses for participants with complete adherence data were linearly transformed to a 0–100 scale and averaged across all three items for a final adherence score (0–100), with a score <90 indicating suboptimal ART adherence.

HIV Care

To assess engagement in HIV care, men were asked in the 12-month parent survey if they had a doctor, nurse, or another medical provider who oversees their overall HIV health care. Response options included no, yes, and prefer not to answer. Participants were also asked in the 12-month parent survey when was the last time they had a health care appointment with their HIV care provider (Last 3 months, 3–6 months ago, 6–9 months ago, 9–12 months ago, More than a year ago, I don’t know, and Prefer not to answer).

The M-Spot and 12-month parent survey included items to measure self-reported VL status. Participants indicating an HIV care visit since the 12-month parent survey were asked if they had a VL test. Participants reporting ‘My viral load was detectable’ or ‘I don’t know – but I think I was detectable’ were categorized as having a self-reported detectable viral load status. Participants reporting ‘My viral load was undetectable’ or ‘I don’t know – but I think I was undetectable’ were categorized as having a self-reported undetectable VL status.

Data on self-reported VL status from participants who did not report an HIV care visit between the 12-month parent survey and the M-Spot study was obtained from the 12-month parent survey. Using the same strategy as in the M-spot survey, responses were dichotomized (detectable, undetectable). Date of last viral load test was not collected in the 12-month parent survey.

Drug use

The 9- and 12-month parent surveys measured past 3-month use of cocaine (snorted/smoked, injected), crack (snorted/smoked, injected), crystal meth (snorted/smoked, injected), heroin (snorted, injected), ketamine (snorted/smoked, injected), methadone (snorted/smoked, injected), synthetic drugs (swallowed or injected bath salts, synthetic marijuana), prescription opioids (injected or swallowed oxycontin, Percocet, Vicodin), amphetamines (injected or swallowed amphetamine, Adderall, Dexedrine), downers (injected or swallowed valium, Ativan, Klonopin, Xanax), Ecstasy (MDMA, or molly), erection medication (Viagra, Cialis), GHB (gamma-hydroxybutyrate), hallucinogens (LSD, PCP, Peyote, Mushrooms), poppers (nitrite inhalants), and/or marijuana. Participants were presented with a full list of drugs and were asked to check any drug they had used in the past 3 months. Participants who had sex in the past 3 months were presented with the same set of questions to indicate if they had used any of these drugs within 2 hours before or during sex.

Participants self-reporting past 3-month use of crystal meth, cocaine, crack, ecstasy or amphetamines were defined as current stimulant users. Other studies of MSM living with HIV have similarly classified ecstasy as a stimulant drug (Carrico et al., 2014; Lim et al., 2012) and according to the National Institute on Drug Abuse, ecstasy’s chemical composition is similar to stimulants and hallucinogens (National Institute on Drug Abuse, 2018). Individuals reporting both stimulant and other drug use were classified as stimulant users. Participants reporting past 3-month use of drugs other than stimulants were defined as other drug users. Participants reporting no past 3-month drug use were classified as non-drug users.

We primarily report drug use data from the 12-month parent survey as these data best represent current drug use behaviors at the time of recruitment to the M-Spot study. Drug use data from the 9-month parent survey was used to assess drug use history for the 6 months before enrollment in the M-Spot study. The M-Spot survey did not collect drug use data as the survey was designed only to collect up-to-date HIV care data.

Statistical Methods

Data cleaning and analyses were performed using SAS version 9.4 (Cary, North Carolina). Descriptive statistics were computed for all variables to evaluate frequency and distribution of data and to examine missing data. Chi-square tests were conducted to assess demographic, socioeconomic, sexual risk behavior, and HIV care differences between the three drug use categories (Table 1). Demographic and socioeconomic differences by protocol completion status (i.e., the participant did or did not provide a testable specimen) (Table 2) and DBS HIV-1 RNA result (Table 4) were also assessed. Participation in the M-Spot study was assessed at multiple stages: if the participant consented and enrolled; if they attempted to collect their blood using the DBS kit; if the lab received their specimen by mail; and if the participant provided a testable DBS specimen. Relative risk regression analyses assessed the crude and adjusted risk estimates assessing the association between participants’ past 3-month drug-use status and M-Spot study protocol completion (Table 3). We chose relative risk models for these analyses because study participation was high and traditional logistic regression would not approximate a risk ratio, which is a more appropriate measure (Zhang & Yu, 1998). All adjusted models controlled for age, race, employment status, and income. Among participants with a testable DBS specimen (n=337), VL results were classified into 3 categories: (1) undetectable; (2) a qualitative result (≤832 copies/mL) or a quantitative result ranging between 833– 1,499 copies/mL; and (3) ≥1,500 copies/mL (quantitative result). Several studies have assessed the viral load threshold in which an individual becomes infectious to others; findings indicate that risk of onward HIV transmission occurs when viral load is above 1,500 copies/mL (Attia, Egger, Muller, Zwahlen, & Low, 2009; Marks et al., 2015; Quinn et al., 2000). We calculated crude and adjusted risk ratios using relative risk to assess the likelihood of providing a DBS specimen with HIV-1 RNA results ≥1,500 copies/mL, by past 3-month drug use status (Table 5).

Table 1.

Sociodemographic, HIV care, and behavioral characteristics of recruited participants, by drug use status (n=765 a)

Total n=765 N (%) Non-drug users n=208 n (%) Other drug users n=291 n (%) Stimulant drug users n=266 n (%) p value b
Age, n=764 c 0.130
 18 – 29 years 155 (20.3) 48 (23.1) 53 (18.3) 54 (20.3)
 30 – 39 years 232 (30.4) 63 (30.3) 85 (29.3) 84 (31.6)
 40 – 49 years 221 (28.9) 57 (27.4) 78 (26.9) 86 (32.3)
 ≥ 50 years 156 (20.4) 40 (19.2) 74 (25.5) 42 (15.8)
Race, n=765 <0.001
 White 521 (68.1) 124 (59.6) 210 (72.2) 187 (70.3)
 Black 124 (16.2) 51 (24.5) 45 (15.5) 28 (10.5)
 Hispanic 120 (15.7) 33 (15.9) 36 (12.4) 51 (19.2)
Education, n=762 c 0.290
 Less than a college graduate 382 (50.1) 104 (50.2) 136 (46.9) 142 (53.6)
 College graduate or higher 380 (49.9) 103 (49.8) 154 (53.1) 123 (46.4)
Employment status, n=754 c <0.001
 Not working full time 311 (41.3) 69 (34.2) 98 (33.8) 144 (55.0)
 Working full time 443 (58.8) 133 (65.8) 192 (66.2) 118 (45.0)
Income, n=742 c 0.002
 < $20,000 225 (30.3) 54 (27.7) 71 (24.7) 100 (38.5)
 $20,000 - $59,999 319 (43.0) 88 (45.1) 124 (43.2) 107 (41.2)
 ≥ $60,000 198 (26.7) 53 (27.2) 92 (32.1) 53 (20.4)
Alcohol use disorder, n=638 c 0.163
 No 380 (59.6) 96 (64.4) 148 (55.4) 136 (61.3)
 Yes 258 (40.4) 53 (35.6) 119 (44.6) 86 (38.7)
Past 3 month injection drug use, n=557 c -- d
 No 451 (81.0) 0 (0.0) 291 (100.0) 160 (60.2)
 Yes 106 (19.0) 0 (0.0) 0 (0.0) 106 (39.8)
Past 3 month male anal sex partners, n=754 c <0.001
 0 partners 115 (15.3) 56 (27.2) 40 (13.8) 19 (7.3)
 1 – 4 partners 415 (55.0) 117 (56.8) 168 (58.1) 130 (50.2)
 ≥ 5 partners 224 (29.7) 33 (16.0) 81 (28.0) 110 (42.5)
Past 3 month male anal sex with SD partner, n=637 c 0.136
 No 108 (16.9) 29 (19.6) 33 (13.2) 46 (19.2)
 Confirmed / Unknown 529 (83.1) 119 (80.4) 216 (86.8) 194 (80.8)
Past 3 month CAS with male SD partner, n=514 c <0.001
 No 41 (8.0) 19 (16.7) 14 (6.6) 8 (4.2)
 Yes 473 (92.0) 95 (83.3) 197 (93.4) 181 (95.8)
Past 3 month STI diagnosis, n=756 c <0.001
 No 622 (82.3) 179 (87.8) 246 (85.1) 197 (74.9)
 Yes 134 (17.7) 25 (12.2) 43 (14.9) 66 (25.1)
Engaged in HIV care, n=757 c 0.634
 No 69 (9.1) 19 (9.5) 23 (7.9) 27 (10.2)
 Yes 688 (90.9) 182 (90.6) 268 (92.1) 238 (89.8)
Last HIV care visit, n=687 c 0.079
 < 6 months 627 (91.3) 169 (92.9) 250 (93.4) 208 (87.8)
 6 – 12 months 50 (7.3) 9 (5.0) 17 (6.3) 24 (10.1)
 > 12 months 10 (1.5) 4 (2.2) 1 (0.4) 5 (2.1)
ART adherence, n=762 c <0.001
 Not currently on ART 60 (7.9) 18 (8.7) 13 (4.5) 29 (10.9)
 Suboptimal adherence 361 (47.4) 81 (39.1) 126 (43.5) 154 (58.1)
 High adherence 341 (44.8) 108 (52.2) 151 (52.1) 82 (30.9)
Self-reported HIV viral load, n=672 c <0.001
 Undetectable (<=200 copies/ml) 616 (91.7) 168 (92.8) 258 (95.9) 190 (85.6)
 Detectable (>200 copies/ml) 56 (8.3) 13 (7.2) 11 (4.1) 32 (14.4)

Abbreviations: SD, serodiscordant; CAS, condomless anal sex; ART, antiretroviral therapy.

Notes: Column percentages presented; percentages may exceed 100% due to rounding.

a

A total of 766 participants were recruited. One participant with missing drug use data was excluded from all analyses.

b

Chi-square test.

c

Denominators vary due to missing data or survey skip pattern.

d

Test statistic not computed due to 0 values in multiple categories.

Table 2.

Drug use and sociodemographic characteristics of participants, by study completion status (n=553 a)

Total n=553 N (%) Completed protocol n=336 n (%) Did not complete protocol b) n=217 n (%) p value c)
Drug use, n=553 0.018
 Non-drug users 136 (24.6) 73 (21.7) 63 (29.0)
 Other drug users 220 (39.8) 149 (44.4) 71 (32.7)
 Stimulant drug users 197 (35.6) 114 (33.9) 83 (38.3)
Age, n=552 d 0.637
 18 – 29 years 111 (20.1) 69 (20.5) 42 (19.4)
 30 – 39 years 181 (32.6) 115 (34.2) 65 (30.1)
 40 – 49 years 159 (28.8) 91 (27.1) 68 (31.5)
 ≥ 50 years 102 (18.5) 61 (18.2) 41 (19.0)
Race, n=553 0.071
 White 380 (68.7) 238 (70.8) 142 (65.4)
 Black 83 (15.0) 41 (12.2) 42 (19.4)
 Hispanic 90 (16.3) 57 (17.0) 33 (15.2)
Employment, n=545 d 0.706
 Not working full time 225 (41.3) 140 (41.9) 85 (40.3)
 Working full time 320 (58.7) 194 (58.1) 126 (59.7)
Income, n=541 d 0.897
 < $20,000 173 (32.0) 107 (32.5) 66 (31.1)
 $20,000 - $59,999 233 (43.1) 142 (43.2) 91 (42.9)
 ≥ $60,000 136 (25.0) 80 (24.3) 55 (25.9)

Notes: Column percentages presented; percentages may exceed 100% due to rounding.

a

A total of 554 participants were enrolled. One participant with missing drug use data was excluded from all analyses.

b

Individuals categorized in the ‘Did not complete protocol’ group include individuals enrolled, but who did not attempt blood collection, who attempted blood collection but did not mail a blood specimen to the lab, and individuals who mailed a specimen with insufficient blood.

c

Chi-square test.

d

Denominators vary due to missing data.

Table 4.

Drug use and sociodemographic characteristics of participants, by DBS HIV-1 RNA result, n=336 a

HIV-1 RNA Results
Total n=336 N (%) Undetectable b n=159 n (%) ≤ 832–1,499 copies/mL c n=120 n (%) ≥ 1,500 copies/mL n=57 n (%) p-value d
Drug use, n=336 0.271
 Non-drug users 73 (21.7) 30 (18.8) 27 (22.5) 16 (28.1)
 Other drug users 149 (44.4) 76 (47.8) 55 (45.8) 18 (31.6)
 Stimulant drug users 114 (33.9) 53 (33.3) 38 (31.7) 23 (40.4)
Age, n=336 0.398
 18 – 29 years 69 (20.5) 35 (21.9) 26 (21.7) 8 (14.0)
 30 – 39 years 116 (34.4) 61 (38.1) 35 (29.2) 20 (35.1)
 40 – 49 years 91 (27.0) 36 (22.5) 35 (29.2) 20 (35.1)
 ≥ 50 years 61 (18.1) 28 (17.5) 24 (20.0) 9 (15.8)
Race, n=336 0.720
 White 239 (70.9) 114 (71.3) 87 (72.5) 38 (66.7)
 Black 41 (12.2) 19 (11.9) 12 (10.0) 10 (17.5)
 Hispanic 57 (16.9) 27 (16.9) 21 (17.5) 9 (15.8)
Employment, n=335 e <0.001
 Not working full time 141 (42.1) 58 (36.5) 43 (36.1) 40 (70.2)
 Working full time 194 (57.9) 101 (63.5) 76 (64.9) 17 (29.8)
Income, n=330 e 0.003
 < $20,000 107 (32.4) 46 (29.1) 34 (28.8) 27 (50.0)
 $20,000 - $59,999 142 (43.0) 68 (43.0) 50 (42.4) 24 (44.4)
 ≥ $60,000 81 (24.6) 44 (27.9) 34 (28.8) 3 (5.6)

Notes: Column percentages presented; percentages may exceed 100% due to rounding.

a

A total of 337 participants provided a testable specimen. One participant with missing drug use data was excluded from all analyses.

b

An undetectable HIV-1 RNA result indicates that no HIV-1 RNA was detectable in the DBS specimen.

c

Individuals categorized in the ‘≤832–1,499 copies/mL’ group include individuals who had a qualitative (≤832 copies/mL) DBS HIV-1 RNA result and a quantitative DBS HIV-1 RNA result that ranged between 833 – 1,499 copies/mL.

d

Chi-square test.

e

Denominators vary due to missing data.

Table 3.

Crude and adjusted relative risk estimates assessing the association between past 3-month drug use status and completing study protocol

Completing protocol a
Crude Estimates Adjusted Estimate
RR (95% CI) p-value ARR (95% CI) p-value
Drug use
 Non-drug users (Reference) 1.00 1.00
 Other drug users b 1.26 (1.05 – 1.51) 0.012 1.24 (1.03 – 1.49) 0.023
 Stimulant drug users b 1.08 (0.89 – 1.31) 0.453 1.04 (0.85 – 1.27) 0.736
 Stimulant drug users vs. other drug users c 0.85 (0.74 – 0.99) 0.040 0.84 (0.72 – 0.98) 0.024
Age d 0.99 (0.99 – 1.00) 0.361 0.99 (0.98 – 1.00) 0.233
Race
 White (Reference) 1.00 1.00
 Black 0.80 (0.63 – 0.99) 0.044 0.79 (0.62 – 0.99) 0.045
 Hispanic 1.01 (0.85 – 1.21) 0.901 0.99 (0.83 – 1.18) 0.892
Employment
 Not working full time (Reference) 1.00 1.00
 Working full time 0.97 (0.85 – 1.11) 0.705 0.94 (0.80 – 1.11) 0.485
Income
 < $20,000 (Reference) 1.00 1.00
 $20,000 - $59,999 0.99 (0.84 – 1.15) 0.853 0.97 (0.82 – 1.15) 0.745
 ≥ $60,000 0.96 (0.80 – 1.15) 0.646 0.99 (0.80 – 1.23) 0.919

Abbreviations: RR, risk ratio; CI, confidence interval; ARR, adjusted risk ratio.

a

Individuals who did not complete the study protocol served as the reference group for the model outcome.

b

Reference category for categorical variable comparison is ‘Non-drug users’.

c

Reference category was changed from ‘Non-drug users’ to ‘Other drug users” in order to obtain RR and ARR estimates when comparing ‘Stimulant drug users’ to ‘Other drug users’.

d

Age modeled as a continuous variable in all crude and adjusted relative risk regression models.

Table 5.

Crude and adjusted relative risk estimates assessing the association between past 3-month drug use status and DBS HIV-1 RNA ≥1,500 copies/ml

Specimen with DBS HIV-1 RNA ≥ 1,500 copies/ml a
Crude Estimates Adjusted Estimate
RR (95% CI) p-value ARR (95% CI) p-value
Drug use
 Non-drug users (Reference) 1.00 1.00
 Other drug users b 0.55 (0.31 – 0.98) 0.041 0.52 (0.28 – 0.97) 0.040
 Stimulant drug users b 0.87 (0.52 – 1.47) 0.602 0.78 (0.46 – 1.34) 0.367
 Stimulant drug users vs. other drug users c 1.58 (0.92 – 2.71) 0.100 1.49 (0.88 – 2.52) 0.134
Age d 1.01 (0.99 – 1.03) 0.308 1.01 (0.99 – 1.03) 0.382
Race
 White (Reference) 1.00 1.00
 Black 1.37 (0.77 – 2.43) 0.281 1.15 (0.64 – 2.06) 0.639
 Hispanic 0.99 (0.53 – 1.86) 0.984 0.88 (0.46 – 1.66) 0.684
Employment
 Not working full time (Reference) 1.00 1.00
 Working full time 0.35 (0.21 – 0.56) <0.001 0.45 (0.25 – 0.83) 0.011
Income
 < $20,000 (Reference) 1.00 1.00
 $20,000 - $59,999 0.71 (0.45 – 1.11) 0.134 1.09 (0.65 – 1.82) 0.746
 ≥ $60,000 0.18 (0.06 – 0.55) 0.003 0.33 (0.10 – 1.10) 0.071

Abbreviations: RR, risk ratio; CI, confidence interval; ARR, adjusted risk ratio.

a

Individuals with an undetectable DBS HIV-1 RNA result served as the reference group for the model outcome.

b

Reference category for categorical variable comparison is ‘Non-drug users’.

c

Reference category was changed from ‘Non-drug users’ to ‘Other drug users” in order to obtain RR and ARR estimates when comparing ‘Stimulant drug users’ to ‘Other drug users’.

d

Age modeled as a continuous variable in all crude and adjusted relative risk regression models.

Results

Recruited participants

A total of 766 men were sent an invitation link to participate in the M-Spot study (Figure 1, Box A), of which 765 men provided complete drug use data during the 12-month parent survey. Men with complete drug-use data were mostly white (68%), employed full-time (58.8%), had a yearly income between $20,000 - $59,999 (43%), and half were college-educated (50%) (Table 1). Median age was 39 (range: 19–77). A majority (92%) of participants self-reported having an undetectable VL based on their last HIV care visit; however, almost half (47%) were suboptimally adherent to their ART based on a 30-day ART adherence scale (Wilson et al., 2016).

Figure 1.

Figure 1

M-Spot Study Flow Diagram

Drug use

Among participants with drug use data (n=765/766), 266 (35%) reported cocaine, crack, crystal meth, amphetamine, and/or ecstasy use in the past 3 months in their 12-month parent survey. Nearly all of these stimulant drug users (245/266; 92%) also reported using multiple stimulants or stimulants and other drugs. Among the 266 stimulant-using participants, 106 (40%) reported injection drug use in the previous 3 months.

A total of 291 of 765 (38%) men reported exclusive use of one or more non-stimulant drugs (i.e., other drug users) in the past 3 months in their 12-month parent survey. None of these participants reported injection drug use. Furthermore, 208 of 765 (27%) men reported no drug use in their 12-month parent survey. In order to more fully assess the drug use history of M-Spot participants, we examined drug use data from the 9-month parent survey for men who reported no drug use in the 12-month parent survey. Among the 208 men reporting no past 3-month drug use in the 12-month parent survey, 52 (25%) reported past 3-month drug use in the 9-month parent survey – indicating some drug use in the 6 months before recruitment to the M-Spot Study. There were no differences between men reporting no drug use in both 9- and 12-month parent surveys and men reporting some drug use in the 9-month parent survey and no drug use in the 12-month survey [data not shown]. Therefore, drug use groups are based on reported drug use in the 12-month parent survey.

Across the three drug groups, race, employment status, income, ART adherence status, and self-reported viral load were associated with participant drug use (Table 1). A lower proportion of black men (11% vs. 16% vs. 25%; P < 0.001) reported stimulant drug use compared to other drug users and non-drug users, respectively. A greater proportion of individuals without full-time employment (55% vs. 34% vs. 34%; P < 0.001), and those with incomes <$20,000 (39% vs. 25% vs. 28%; P = 0.002) reported any stimulant drug use compared to other drug users and non-drug users. Also, a greater proportion of men self-reporting suboptimal ART adherence (58% vs. 44% vs. 49%; P < 0.001), and a detectable viral load (14% vs. 4% vs. 7%; P < 0.001) reported stimulant drug use compared to other drug users and non-drug users. Meanwhile, although not statistically significant, a greater proportion of other drug users reported an alcohol use disorder (45% vs. 39% vs. 36%; P = 0.163) than stimulant drug users and non-drug users.

Drug use and sexual risk behavior

Distinct differences were found across the three drug groups regarding recent sexual risk behavior and STI history. A greater proportion of stimulant drug users than other drug users and non-drug users reported 5 or more male anal sex partners in the past 3 months (43% vs. 28% vs. 16%; P < 0.001). Among participants who reported any male anal sex partners in the past 3 months, a greater proportion of other drug users reported anal sex with a known or unknown male serodiscordant partner (87% vs. 81% vs. 80%; P = 0.136). Among men who reported anal sex with a male serodiscordant partner, more stimulant drug users reported CAS with the serodiscordant partner(s) (96% vs. 93% vs. 83%; P < 0.001) compared to other drug users and non-drug users. Finally, more stimulant drug users reported a past 3-month STI diagnosis (25% vs. 15% vs. 12%; P < 0.001) than other drug users and non-drug users.

Study completion

Among recruited men (n=766), 663 (87%) opened the email and clicked on the M-Spot recruitment link (Figure 1, Box B). In total, 568 men were eligible to participate (Figure 1, Box C), and 554 (72% of those recruited) enrolled in the study (Figure 1, Box D). There were no statistically significant differences in the M-Spot study sample based on the parent study’s intervention randomization condition. After enrollment, 439 (79% of those enrolled) attempted blood collection (Figure 1, Box E), 413 (75% of those enrolled) returned a DBS specimen to the lab (Figure 1, Box F) and 337 (61% of those enrolled) provided a testable specimen to the lab (Figure 1-Box G). Individuals who provided a testable specimen to the lab (n=337) were considered to have successfully completed the study protocol, while participants who did not attempt to collect a blood specimen, those who did not return a specimen to the lab, or who returned a kit with no blood or insufficient blood were considered as not completing the study protocol (n=217).

Across drug use categories, 54% (73/136) of enrolled non-drug users, 68% (149/220) of enrolled other drug users, and 58% (114/197) of enrolled stimulant users were able to complete the M-Spot Study protocol. The majority of individuals completing the study protocol were other drug users (44%), between the ages of 30–39 years (34%), White (71%), employed full-time (58%), and had an annual income between $20,000 - $59,999 (43%) (Table 2). Other drug users were significantly more likely (adjusted relative risk (ARR), 1.24; 95% confidence interval (CI), 1.03 – 1.49) to complete the study protocol and provide a testable DBS specimen, compared to non-drug users. Furthermore, stimulant drug users had a lower probability of (ARR, 0.84; 95% CI, 0.72 – 0.98) completing the study protocol and providing a testable DBS specimen than other drug users (Table 3).

Viral Load Results

A total of 336 participants returned a testable DBS specimen and reported complete drug use data in the 12-month parent survey. In total, over half (177/336; 53%) had detectable HIV-1 RNA while 47% (159/336) of participants returned a sample with no detectable HIV-1 RNA (Table 4). Of the DBS specimens classified as having a detectable VL, a total of 99 (29% of testable specimens) DBS specimens had a qualitative result of ‘≤832 copies/ mL (≤2.92 log copies).’ An additional 21 (6% of testable specimens) DBS specimens had a quantitative result between 833 – 1,499 copies/mL (median: 1,096 copies/mL; IQR: 912 – 1,230 copies/mL). Participants returning a DBS specimen with a qualitative HIV-1 RNA result of ≤832copies/mL or a quantitative HIV-1 RNA result between 833–1,499 copies/mL were categorized into one group (n=120) (Table 4). Of significance, 57 (17% of testable specimens) specimens had viremia ≥1,500 copies/mL (median: 8,128 copies/mL; IQR: 2,570 – 44,668 copies/mL) – a threshold indicative of an increased probability of further HIV transmission (Table 4). The majority of participants with DBS results ≥1,500 copies/mL were stimulant drug users (40%), between the ages of 30–49 (70%), White (67%), not employed full-time (70%), and had an annual income <$20,000 (Table 4).

We conducted additional analyses [data not shown]. We examined CAS and serodiscordant CAS in the past 3 months, to assess whether there were any differences in reported sexual risk behavior across levels of DBS HIV-1 RNA results; no differences were found. Additionally, across the three drug use categories, 21 non-drug using participants (median: 5,623 copies/mL; IQR: 1,513 – 20,893 copies/mL), 27 other drug-using participants (median: 1,950 copies/mL; IQR: 1,096 – 6,310 copies/mL), and 30 stimulant drug-using participants (median: 8,128 copies/mL; IQR: 1,585 – 52,480 copies/mL) returned a DBS specimen with quantifiable HIV-1 RNA. Stimulant drug users who returned a DBS sample with quantitative HIV-1 RNA (samples with viremia ≥833 copies/mL) had significantly higher viremia compared to other drug users (Z=653.5; P = 0.0392). In sensitivity analysis, we observed that stimulant injectors had more than 2-fold higher DBS VL compared to stimulant users who did not inject (14,125 vs. 5,011 copies/mL), but this was not statistically significant (P = 0.1025).

In Table 5 we compared the likelihood of returning a DBS specimen with detectable HIV-1 RNA ≥1,500 copies/mL across different drug user groups among participants who returned a testable DBS specimen. Participants were stratified into 3 different groups based on their DBS specimen HIV-1 RNA result: those who were undetectable, participants with a qualitative viral load result (≤ 832 copies/mL) or a quantitative viral load between 833 – 1,499 copies/mL, and participants with a viral load ≥1,500 copies/mL. Other drug users had a lower probability of (ARR, 0.52; 95% CI, 0.28 – 0.97) providing a DBS specimen with an HIV-1 RNA result ≥1,500 copies/mL, compared to non-drug users. Although not statistically significant, it is worth noting that stimulant drug users had a greater probability of (RR, 1.49; 95% CI, 0.88 – 2.52) providing a DBS specimen with an HIV-1 RNA result ≥1,500 copies/mL compared to other drug users (Table 5). No statistically significant differences were found between stimulant users and non-drug users on high viremia outcomes.

Discussion

Principal findings

In an online U.S. sample of MSM living with HIV who report high-risk sexual behavior, this study is among the first to assess the relationship between drug use and participant activities in an at-home biomarker collection study that entailed self-collecting a DBS specimen and mailing the specimen for lab VL quantification. More than one-third of all participants reported stimulant drug use, and across all drug use categories, most men mailed a testable DBS specimen back to the lab for HIV VL quantification. However, stimulant drug users had a lower probability of completing the study protocol (i.e., provide a testable DBS specimen) compared to other drug users. Among participants who returned a testable DBS specimen, stimulant users had a greater probability of returning a DBS specimen with a quantitative VL result ≥1,500 copies/mL. This finding signals the need for more comprehensive approaches to optimize engagement of stimulant-using MSM in various stages of the HIV care continuum and similar biomedical studies.

This study provides important estimates regarding the likelihood of enrollment, and participation in biomedical research activities among HIV-positive MSM with varying drug use behaviors. When comparing different drug use groups, differences were observed among participants who successfully and unsuccessfully completed the study protocol. Of note, a greater proportion of other drug users completed the protocol and returned a testable DBS specimen compared to stimulant users and non-drug users. Men classified as other drug users included men who reported prescription and non-prescription opioid use, downers, erection medication, and/or marijuana. It is possible that these men had a prescription for these medications and were therefore likely to be engaged in medical care, which could explain their higher likelihood of returning a testable DBS specimen. In addition, while opioid users have medication-assisted treatment (MAT) for drug use, stimulant users do not have an equivalent. This could also explain the differences we observed between stimulant and other (non-stimulant) drug users and suggests that drug treatment is another potential resource for supporting MSM in regimens that include returning specimens (Bruce, Kresina, & McCance-Katz, 2010; Oldfield et al., 2018).

Participant viral suppression

Consistent with prior research, our analyses revealed that a considerable proportion of stimulant users had high levels of HIV viremia, with over 40% of stimulant users returning a specimen with ≥1,500 copies/mL. However, across drug use categories, no statistically significant differences were observed in DBS specimen HIV viremia outcomes (undetectable, ≤832 – 1,499 copies/mL, ≥1,500 copies/mL). It is likely that individuals completing the protocol shared similar HIV care-related characteristics (e.g., retention in care and adherence to ART), which would explain the similar distribution of viral load outcomes across drug use categories seen in Table 4. Since a significant proportion of stimulant users were less likely to complete the protocol (Table 3), it is possible that stimulant users with worse virologic outcomes were among those who did not complete the protocol. Conducting this study on a larger scale, with a larger sample size may expose whether differences do indeed exist. Furthermore, unmeasured differences (e.g., severity of stimulant drug use, neurocognitive impairment) may also account for similar VL outcomes between stimulant users and other drug users who provided a testable specimen.

Future research should also compare self-reported HIV care engagement and viral suppression characteristics with clinical data. Recall bias and social desirability bias may affect the validity of self-reported HIV care covariates; validating these characteristics may improve our understanding of how HIV care and drug use patterns affect participation rates in studies which require participants to self-collect specimens. Furthermore, although a minority of stimulant drug users self-reported a detectable viral load (Table 1), DBS specimen VL testing revealed that over half (54%) of stimulant users returned a specimen with detectable HIV-1 RNA (Table 4). We have previously reported (Hirshfield et al., 2018) on the discrepancies between self-reported viral load status and DBS specimen viral load results. Results from this study further strengthen our call for the field to assure that perceived viral load status is consistent with their actual viral load. Conducting this study in a clinical setting with a larger sample, and with multiple DBS collection points, would enable us to validate self-reported data, assess VL fluctuations in between visits, and assess if VL monitoring in between HIV care visits can improve ART adherence and viral suppression in a non-research setting.

Detectable viral load in context with sexual behavior

The “Undetectable = Untransmittable” campaign has refocused treatment as prevention efforts among MSM living with HIV after the Centers for Disease Control and Prevention and the National Institute of Allergy and Infectious Diseases declared that people on ART who have an undetectable VL have no risk of transmitting the virus to HIV-negative partners (Eisinger et al., 2019; The Lancet HIV, 2017). To prevent further transmission of HIV, it is critical that future research and public health interventions encourage participation among sexually active populations struggling with ART adherence (including those who use drugs) to help them achieve and maintain an undetectable viral load.

Our study results highlight the need to address viral suppression in a population of MSM who engage in sexual transmission risk behavior, given that almost 1 in 5 men who completed the study protocol had a DBS HIV-1 RNA result ≥1,500 copies/mL. To enroll in the parent study, participants had to have reported previous or current suboptimal ART adherence and CAS with serodiscordant partners. During the three months before enrollment in the M-Spot study, 83% of these participants reported continued male anal sex with a serodiscordant partner. Among participants reporting past 3-month serodiscordant anal sex with a male partner, 92% of them reported CAS with the serodiscordant partner (Table 1). Although no differences in sexual risk behavior were identified across levels of DBS HIV-1 RNA in this study, results indicate that sexual transmission risk behaviors occurred regardless of VL level; given the high prevalence of potential transmission risk, the large discrepancy between self-reported VL and actual VL as measured from the DBS specimens is concerning. These results indicate that participant self-report may not be reliable due to social desirability bias or recall bias, or that participants may have been experiencing fluctuating viral load in between HIV care visits.

Impact of drug use on HIV care

We found that 40% (106/266) of stimulant drug-using MSM reported injection drug use in the previous 3 months before enrollment in the M-Spot study. Substance use and injection drug use are important factors contributing to suboptimal medication adherence, detectable viral load, and further HIV transmission (Langebeek et al., 2014). MSM who use substances experience barriers to accessing ART and are more likely to have poor adherence (Shoptaw, 2017). Individual factors such as co-morbidities, social factors such as lack of social supports and community, and structural factors such as poverty, stigma, and discrimination create barriers to care for substance users (Ferro et al., 2015; Malta, Magnanini, Strathdee, & Bastos, 2010).

Injection drug use can exacerbate disengagement in HIV care activities. Stimulant injecting MSM are especially at risk. Research shows that cocaine and amphetamines are associated with more frequent injections (Booth, Watters, & Chitwood, 1993), and amphetamine injectors are more likely to report syringe sharing in the past 30 days, compared to persons who inject drugs other than amphetamines (Braine, Des Jarlais, Goldblatt, Zadoretzky, & Turner, 2005; Rondinelli et al., 2009). As mentioned previously, one barrier that MSM who inject amphetamines experience is that unlike opioid addiction, there currently is no MAT for amphetamine addiction, meaning there is no opportunity to combine drug treatment with HIV treatment (Ballester, Valentine, & Sofuoglu, 2017). This is problematic because combined treatment is most successful for HIV positive substance users (Low et al., 2016; Malta et al., 2010). Research should focus on how to engage stimulant substance users, including injectors, in treatment. Further research is also needed to assess how stimulant injection affects MSM participation in biomedical research, and if there are any significant differences with stimulant drug users who report no injection drug use. Longitudinal studies of drug users (and typologies of use) will help research and intervention communities tailor treatment and adherence among drug-using MSM.

Study limitations

First, although no significant differences were observed when comparing drug use groups at early study stages (i.e., blood collection vs. enrollment, and mailing kit to lab vs. enrollment), about 40% of participants who enrolled did not provide a testable specimen and therefore did not complete the study protocol. When comparing the individuals who provided a testable specimen versus those enrolled, differences were observed between drug-use groups. We have previously reported (Hirshfield et al., 2018) that participants who withdrew or were lost to follow-up in the M-Spot study had lower ART adherence scores. Further research is needed to assess how suboptimal ART adherence and drug use patterns affect a participant’s ability to provide a testable DBS specimen. It is also important to understand how different ways of delivering the study instructions may improve the successful completion of this type of biomedical study that requires participants to self-collect biological specimens. Second, the study population was recruited from a sample of men who had already completed a 12-month online intervention. It is possible that these participants, regardless of drug use behavior, may be more likely to participate in biomedical studies involving DBS specimens. Furthermore, stimulant and other drug use patterns may be different in other populations of MSM living with HIV. Third, a quarter of men reporting past 3-month no drug use in the 12-month parent survey reported past 3-month drug use in the 9-month parent survey. It is possible these men are more similar to participants who reported other or stimulant drug use behavior in the 12-month parent survey; future studies should assess longitudinal substance use and the impact of distal vs. proximal drug use behavior on biomedical study participation. Lastly, the assay used to test the DBS specimens was only able to provide quantitative VL results above 832 copies/mL. The utility of results ≤832 copies/mL is a limitation as the manufacturer has not published the assay’s lower limit of detection. Future research measuring HIV VL from DBS specimens should strive to use assays that can detect lower levels of HIV-1 RNA in order to improve research efforts, limit epidemiologic issues such as data misclassification, and improve the utility of DBS specimens in treatment as prevention strategies.

Conclusions

Findings from this study highlight the feasibility of conducting home-collection of biomarker data for laboratory HIV viral load testing among MSM living with HIV. Despite the feasibility of providing DBS specimens, this study identifies differences in the ability to complete a study protocol based on drug use history, and that drug use profile may be associated with varying degrees of study participation. Further, DBS viral load results indicate various levels of detectable viral load among a population reporting sexual transmission risk behaviors before and during this research study. With the current opioid and stimulant epidemics impacting marginalized populations - including MSM living with HIV - it is critical to engage these populations in future clinical and biomedical interventions as they are likely in greatest need of services and treatment. Strategies to improve particiation and completion of study activities among stimulant users could include contingency management (Landovitz, Fletcher, Shoptaw, & Reback, 2015) and the ability to access study staff via video (e.g., teleconferencing with study personnel to obtain one-on-one coaching on how to self-collect a testable blood specimen). Future studies should also consider personality attributes associated with different types of substance users (e.g., stimulant vs. non-stimulant users), including impulsivity characteristics which may explain specific differences observed in this study among stimulant drug users (de Wit, 2009; Ersche et al., 2013; Verdejo-García, Lawrence, & Clark, 2008). In all, results from this study provide important information that can help researchers better prepare for similar study designs that incorporate self-collected specimens and aim to engage populations experiencing difficulties with ART adherence as well as achieving and maintaining viral suppression.

Acknowledgments

This research was supported by a grant from the National Institute of Mental Health (R01-MH100973-03S1) to Sabina Hirshfield, principal investigator. Richard Teran was supported by a pre-doctoral fellowship in the Global HIV Implementation Science Research training program sponsored by the Mailman School of Public Health, Columbia University, with funding from the National Institute of Allergy and Infectious Diseases (T32 AI114398, PI: Howard). Suzan Walters was supported with funding from the National Institute on Drug Abuse (T32 DA007233-31, PI: Falkin) and Interdisciplinary Research Training Institute on Hispanic Drug Abuse (R25 DA026401; PI Avelardo Valdez). The authors would like to thank Steven Houang, Irene Yoon, and Dayana Bermudez for their contributions to study enrollment, data collection, and retention activities.

Footnotes

Conflict of Interest:

The authors have no conflicts of interest to disclose.

References

  1. Alvarez-Muñoz MT, Zaragoza-Rodríguez S, Rojas-Montes O, Palacios-Saucedo G, Vázquez-Rosales G, Gómez-Delgado A, . . . Muñoz O (2005). High Correlation of Human Immunodeficiency Virus Type-1 Viral Load Measured in Dried-Blood Spot Samples and in Plasma under Different Storage Conditions. Archives of Medical Research, 36(4), 382–386. doi: 10.1016/j.arcmed.2005.03.010 [DOI] [PubMed] [Google Scholar]
  2. Attia S, Egger M, Muller M, Zwahlen M, & Low N (2009). Sexual transmission of HIV according to viral load and antiretroviral therapy: systematic review and meta-analysis. AIDS, 23(11), 1397–1404. doi: 10.1097/QAD.0b013e32832b7dca [DOI] [PubMed] [Google Scholar]
  3. Ballester J, Valentine G, & Sofuoglu M (2017). Pharmacological treatments for methamphetamine addiction: current status and future directions. Expert Review of Clinical Pharmacology, 10(3), 305–314. doi: 10.1080/17512433.2017.1268916 [DOI] [PubMed] [Google Scholar]
  4. Booth RE, Watters JK, & Chitwood DD (1993). HIV risk-related sex behaviors among injection drug users, crack smokers, and injection drug users who smoke crack. American Journal of Public Health, 83(8), 1144–1148. doi: 10.2105/ajph.83.8.1144 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Braine N, Des Jarlais DC, Goldblatt C, Zadoretzky C, & Turner C (2005). HIV risk behavior among amphetamine injectors at US syringe exchange programs. AIDS Education and Prevention, 17(6), 515. [DOI] [PubMed] [Google Scholar]
  6. Brown DM, Thorne JE, Foster GL, Duncan JL, Brune LM, Muñana A, . . . Jabs DA (2006). Factors affecting attrition in a longitudinal study of patients with AIDS. AIDS Care, 18(7), 821–829. doi: 10.1080/09540120500466747 [DOI] [PubMed] [Google Scholar]
  7. Bruce RD, Kresina TF, & McCance-Katz EF (2010). Medication-assisted treatment and HIV/AIDS: aspects in treating HIV-infected drug users. AIDS, 24(3), 331–340. doi: 10.1097/QAD.0b013e32833407d3 [DOI] [PubMed] [Google Scholar]
  8. Buchacz K, McFarland W, Kellogg TA, Loeb L, Holmberg SD, Dilley J, & Klausner JD (2005). Amphetamine use is associated with increased HIV incidence among men who have sex with men in San Francisco. AIDS, 19(13), 1423–1424. doi: 10.1097/01.aids.0000180794.27896.fb [DOI] [PubMed] [Google Scholar]
  9. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA, & for the Ambulatory Care Quality Improvement, P. (1998). The audit alcohol consumption questions (audit-c): An effective brief screening test for problem drinking. Archives of Internal Medicine, 158(16), 1789–1795. doi: 10.1001/archinte.158.16.1789 [DOI] [PubMed] [Google Scholar]
  10. Cappelle D, Yegles M, Neels H, van Nuijs ALN, De Doncker M, Maudens K, . . . Crunelle CLJFT (2015). Nail analysis for the detection of drugs of abuse and pharmaceuticals: a review. 33(1), 12–36. doi: 10.1007/s11419-014-0258-1 [DOI] [Google Scholar]
  11. Carrico AW, Hunt PW, Neilands TB, Dilworth SE, Martin JN, Deeks SG, & Riley ED (2019). Stimulant Use and Viral Suppression in the Era of Universal Antiretroviral Therapy. J Acquir Immune Defic Syndr, 80(1), 89–93. doi: 10.1097/QAI.0000000000001867 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Carrico AW, Riley ED, Johnson MO, Charlebois ED, Neilands TB, Remien RH, . . . Chesney MA (2011). Psychiatric Risk Factors for HIV Disease Progression: The Role of Inconsistent Patterns of Anti-Retroviral Therapy Utilization. Journal of acquired immune deficiency syndromes (1999), 56(2), 146–150. doi: 10.1097/QAI.0b013e318201df63 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Carrico AW, Shoptaw S, Cox C, Stall R, Li X, Ostrow DG, . . . Plankey MW (2014). Stimulant use and progression to AIDS or mortality after the initiation of highly active antiretroviral therapy. J Acquir Immune Defic Syndr, 67(5), 508–513. doi: 10.1097/qai.0000000000000364 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cassol S, Gill MJ, Pilon R, Cormier M, Voigt RF, Willoughby B, & Forbes J (1997). Quantification of human immunodeficiency virus type 1 RNA from dried plasma spots collected on filter paper. Journal of Clinical Microbiology, 35(11), 2795–2801. Retrieved from http://jcm.asm.org/content/35/11/2795.abstract [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Cattie J, Marquine MJ, Bolden KA, Obermeit LC, Morgan EE, Franklin DR, . . . Woods SP (2015). Predictors of attrition in a cohort study of HIV infection and methamphetamine dependence. Journal of Substance Use, 20(6), 407–416. doi: 10.3109/14659891.2014.942397 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Claus RE, Kindleberger LR, & Dugan MC (2002). Predictors of Attrition in a Longitudinal Study of Substance Abusers. Journal of Psychoactive Drugs, 34(1), 69–74. doi: 10.1080/02791072.2002.10399938 [DOI] [PubMed] [Google Scholar]
  17. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, . . . Fleming TR (2011). Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med, 365. doi: 10.1056/NEJMoa1105243 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Colfax G, Coates TJ, Husnik MJ, Huang Y, Buchbinder S, Koblin B, . . . Team ES (2005). Longitudinal patterns of methamphetamine, popper (amyl nitrite), and cocaine use and high-risk sexual behavior among a cohort of San Francisco men who have sex with men. Journal of Urban Health, 82(1), i62. doi: 10.1093/jurban/jti025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Colfax G, & Shoptaw S (2005). The methamphetamine epidemic: implications for HIV prevention and treatment. Curr HIV/AIDS Rep, 2(4), 194–199. [DOI] [PubMed] [Google Scholar]
  20. Cottler LB, Compton WM, Ben-Abdallah A, Horne M, & Claverie D (1996). Achieving a 96.6 percent follow-up rate in a longitudinal study of drug abusers. Drug and Alcohol Dependence, 41(3), 209–217. doi: 10.1016/0376-8716(96)01254-9 [DOI] [PubMed] [Google Scholar]
  21. de Wit H (2009). Impulsivity as a determinant and consequence of drug use: a review of underlying processes. Addict Biol, 14(1), 22–31. doi: 10.1111/j.1369-1600.2008.00129.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Eisinger RW, Dieffenbach CW, & Fauci AS (2019). HIV Viral Load and Transmissibility of HIV infection: Undetectable Equals Untransmittable. JAMA, 321(5), 451–452. doi: 10.1001/jama.2018.21167 [DOI] [PubMed] [Google Scholar]
  23. Ellis RJ, Childers ME, Cherner M, Lazzaretto D, Letendre S, Grant I, & Group HIVNRC (2003). Increased Human Immunodeficiency Virus Loads in Active Methamphetamine Users Are Explained by Reduced Effectiveness of Antiretroviral Therapy. The Journal of Infectious Diseases, 188(12), 1820–1826. Retrieved from http://www.jstor.org/stable/30076778 [DOI] [PubMed] [Google Scholar]
  24. Ersche KD, Jones PS, Williams GB, Smith DG, Bullmore ET, & Robbins TW (2013). Distinctive personality traits and neural correlates associated with stimulant drug use versus familial risk of stimulant dependence. Biol Psychiatry, 74(2), 137–144. doi: 10.1016/j.biopsych.2012.11.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ferro EG, Weikum D, Vagenas P, Copenhaver MM, Gonzales P, Peinado J, . . . Altice FL (2015). Alcohol use disorders negatively influence antiretroviral medication adherence among men who have sex with men in Peru. AIDS Care, 27(1), 93–104. doi: 10.1080/09540121.2014.963013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Fleury M-J, Grenier G, Bamvita J-M, Perreault M, & Caron J (2015). Typology of Individuals with Substance Dependence Based on a Montreal Longitudinal Catchment Area Study. Administration and Policy in Mental Health and Mental Health Services Research, 42(4), 405–419. doi: 10.1007/s10488-014-0581-1 [DOI] [PubMed] [Google Scholar]
  27. Gamarel KE, Woolf-King SE, Carrico AW, Neilands TB, & Johnson MO (2015). Stimulant use patterns and HIV transmission risk among HIV-serodiscordant male couples. J Acquir Immune Defic Syndr, 68(2), 147–151. doi: 10.1097/QAI.0000000000000418 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Garrido C, Zahonero N, Corral A, Arredondo M, Soriano V, & de Mendoza C (2009). Correlation between Human Immunodeficiency Virus Type 1 (HIV-1) RNA Measurements Obtained with Dried Blood Spots and Those Obtained with Plasma by Use of Nuclisens EasyQ HIV-1 and Abbott RealTime HIV Load Tests. Journal of Clinical Microbiology, 47(4), 1031–1036. doi: 10.1128/jcm.02099-08 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Gilbert M, Hottes TS, Kerr T, Taylor D, Fairley CK, Lester R, . . . Ogilvie G (2013). Factors Associated With Intention to Use Internet-Based Testing for Sexually Transmitted Infections Among Men Who Have Sex With Men. J Med Internet Res, 15(11), e254. doi: 10.2196/jmir.2888 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Grant RM, Lama JR, Anderson PL, McMahan V, Liu AY, Vargas L, . . . Glidden DV (2010). Preexposure Chemoprophylaxis for HIV Prevention in Men Who Have Sex with Men. New England Journal of Medicine, 363(27), 2587–2599. doi: 10.1056/NEJMoa1011205 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hall EW, Ricca AV, Khosropour CM, & Sullivan PS (2017). Capturing HIV Incidence Among MSM Through At-Home and Self-reported Facility-based Testing. JAIDS Journal of Acquired Immune Deficiency Syndromes, 75(5), e142–e144. doi: 10.1097/qai.0000000000001338 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Hirshfield S, Downing MJ Jr, Parsons JT, Grov C, Gordon RJ, Houang ST, . . . Chiasson MA (2016). Developing a Video-Based eHealth Intervention for HIV-Positive Gay, Bisexual, and Other Men Who Have Sex with Men: Study Protocol for a Randomized Controlled Trial. JMIR Res Protoc, 5(2), e125. doi: 10.2196/resprot.5554 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Hirshfield S, Teran RA, Downing MJ, Chiasson MA, Tieu HV, Dize L, & Gaydos CA (2018). Quantification of HIV-1 RNA Among Men Who Have Sex With Men Using an At-Home Self-Collected Dried Blood Spot Specimen: Feasibility Study. JMIR Public Health Surveill, 4(4), e10847. doi: 10.2196/10847 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Horvath KJ, Carrico AW, Simoni J, Boyer EW, Amico KR, & Petroll AE (2013). Engagement in HIV Medical Care and Technology Use among Stimulant-Using and Nonstimulant-Using Men who have Sex with Men. AIDS Res Treat, 2013, 121352. doi: 10.1155/2013/121352 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Landovitz RJ, Fletcher JB, Shoptaw S, & Reback CJ (2015). Contingency management facilitates the use of postexposure prophylaxis among stimulant-using men who have sex with men. Open Forum Infect Dis, 2(1), ofu114. doi: 10.1093/ofid/ofu114 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Langebeek N, Gisolf EH, Reiss P, Vervoort SC, Hafsteinsdóttir TB, Richter C, . . . Nieuwkerk PT (2014). Predictors and correlates of adherence to combination antiretroviral therapy (ART) for chronic HIV infection: a meta-analysis. BMC Medicine, 12(1), 142. doi: 10.1186/s12916-014-0142-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Li DH, Janulis P, & Mustanski B (2019). Predictors of correspondence between self-reported substance use and urinalysis screening among a racially diverse cohort of young men who have sex with men and transgender women. Addictive Behaviors, 88, 6–14. doi: 10.1016/j.addbeh.2018.08.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lim SH, Ostrow D, Stall R, Chmiel J, Herrick A, Shoptaw S, . . . Plankey M (2012). Changes in Stimulant Drug Use Over Time in the MACS: Evidence for Resilience Against Stimulant Drug Use Among Men Who Have Sex with Men. AIDS and Behavior, 16(1), 151–158. doi: 10.1007/s10461-010-9866-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Low AJ, Mburu G, Welton NJ, May MT, Davies CF, French C, . . . Vickerman P (2016). Impact of Opioid Substitution Therapy on Antiretroviral Therapy Outcomes: A Systematic Review and Meta-Analysis. Clinical Infectious Diseases, 63(8), 1094–1104. doi: 10.1093/cid/ciw416 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Malta M, Magnanini MMF, Strathdee SA, & Bastos FI (2010). Adherence to Antiretroviral Therapy Among HIV-Infected Drug Users: A Meta-Analysis. AIDS and Behavior, 14(4), 731–747. doi: 10.1007/s10461-008-9489-7 [DOI] [PubMed] [Google Scholar]
  41. Manak MM, Hack HR, Shutt AL, Danboise BA, Jagodzinski LL, & Peel SA (2018). Stability of Human Immunodeficiency Virus serological markers in samples collected as HemaSpot and Whatman 903 Dried Blood Spots. Journal of Clinical Microbiology. doi: 10.1128/jcm.00933-18 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mansergh G, Purcell DW, Stall R, McFarlane M, Semaan S, Valentine J, & Valdiserri R (2006). CDC Consultation on Methamphetamine Use and Sexual Risk Behavior for HIV/STD Infection: Summary and Suggestions. Public Health Reports, 121(2), 127–132. doi: 10.1177/003335490612100205 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Marks G, Gardner LI, Rose CE, Zinski A, Moore RD, Holman S, . . . Giordano TP (2015). Time above 1500 copies: a viral load measure for assessing transmission risk of HIV-positive patients in care. AIDS, 29(8), 947–954. doi: 10.1097/QAD.0000000000000640 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mayer KH, Skeer MR, O’Cleirigh C, Goshe BM, & Safren SA (2014). Factors Associated with Amplified HIV Transmission Behavior Among American Men Who Have Sex with Men Engaged in Care: Implications for Clinical Providers. Annals of Behavioral Medicine, 47(2), 165–171. doi: 10.1007/s12160-013-9527-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. McCarty-Caplan D, Jantz I, & Swartz J (2014). MSM and Drug Use: A Latent Class Analysis of Drug Use and Related Sexual Risk Behaviors. AIDS and Behavior, 18(7), 1339–1351. doi: 10.1007/s10461-013-0622-x [DOI] [PubMed] [Google Scholar]
  46. McCray E, & Mermin J (2017, September 27, 2017). Dear Colleague: September 27, 2017 Retrieved from https://www.cdc.gov/hiv/library/dcl/dcl/092717.html [Google Scholar]
  47. Morin SF, Myers JJ, Shade SB, Koester K, Maiorana A, & Rose CD (2007). Predicting HIV Transmission Risk among HIV-Infected Patients Seen in Clinical Settings. AIDS and Behavior, 11(1), 6–16. doi: 10.1007/s10461-007-9253-4 [DOI] [PubMed] [Google Scholar]
  48. National Institute on Drug Abuse. (2018, June 2018). MDMA (Ecstasy/Molly). Retrieved from https://www.drugabuse.gov/publications/drugfacts/mdma-ecstasymolly [Google Scholar]
  49. Oldfield BJ, Muñoz N, Mcgovern MP, Funaro M, Villanueva M, Tetrault JM, & Edelman EJ (2018). Integration of care for HIV and opioid use disorder: a systematic review of interventions in clinical and community-based settings. AIDS. doi: 10.1097/QAD.0000000000002125 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Palamar JJ, Salomone A, Cleland CM, & Sherman S (2018). Willingness to provide a hair sample for drug testing among electronic dance music party attendees. Subst Abus, 1–8. doi: 10.1080/08897077.2018.1469106 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Quinn TC, Wawer MJ, Sewankambo N, Serwadda D, Li C, Wabwire-Mangen F, . . . Gray RH (2000). Viral load and heterosexual transmission of human immunodeficiency virus type 1. Rakai Project Study Group. N Engl J Med, 342(13), 921–929. doi: 10.1056/NEJM200003303421303 [DOI] [PubMed] [Google Scholar]
  52. Rondinelli AJ, Ouellet LJ, Strathdee SA, Latka MH, Hudson SM, Hagan H, & Garfein RS (2009). Young adult injection drug users in the United States continue to practice HIV risk behaviors. Drug and Alcohol Dependence, 104(1), 167–174. doi: 10.1016/j.drugalcdep.2009.05.013 [DOI] [PubMed] [Google Scholar]
  53. Santos G-M, Coffin PO, Das M, Matheson T, DeMicco E, Raiford JL, . . . Herbst JH (2013). Dose-Response Associations Between Number and Frequency of Substance Use and High-Risk Sexual Behaviors Among HIV-Negative Substance-Using Men Who Have Sex With Men (SUMSM) in San Francisco. JAIDS Journal of Acquired Immune Deficiency Syndromes, 63(4), 540–544. doi: 10.1097/QAI.0b013e318293f10b [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Sharma A, Stephenson RB, White D, & Sullivan PS (2014). Acceptability and intended usage preferences for six HIV testing options among internet-using men who have sex with men. SpringerPlus, 3(1), 109. doi: 10.1186/2193-1801-3-109 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Sharma A, Sullivan PS, & Khosropour CM (2011). Willingness to take a free home HIV test and associated factors among internet-using men who have sex with men. J Int Assoc Physicians AIDS Care, 10. doi: 10.1177/1545109711404946 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Shoptaw S (2017). HIV Positive Gay Men, MSM, and Substance Use: Perspectives on HIV Prevention In Wilton L (Ed.), Understanding Prevention for HIV Positive Gay Men: Innovative Approaches in Addressing the AIDS Epidemic (pp. 75–92). New York, NY: Springer New York. [Google Scholar]
  57. Shoptaw S, & Reback CJ (2007). Methamphetamine use and infectious disease-related behaviors in men who have sex with men: implications for interventions. Addiction, 102(s1), 130–135. doi:doi: 10.1111/j.1360-0443.2006.01775.x [DOI] [PubMed] [Google Scholar]
  58. Stall R, & Purcell DW (2000). Intertwining Epidemics: A Review of Research on Substance Use Among Men Who Have Sex with Men and Its Connection to the AIDS Epidemic. AIDS and Behavior, 4(2), 181–192. doi: 10.1023/a:1009516608672 [DOI] [Google Scholar]
  59. The Lancet HIV (2017). U=U taking off in 2017. Lancet HIV, 4(11), e475. doi: 10.1016/S2352-3018(17)30183-2 [DOI] [PubMed] [Google Scholar]
  60. Verdejo-García A, Lawrence AJ, & Clark L (2008). Impulsivity as a vulnerability marker for substance-use disorders: review of findings from high-risk research, problem gamblers and genetic association studies. Neurosci Biobehav Rev, 32(4), 777–810. doi: 10.1016/j.neubiorev.2007.11.003 [DOI] [PubMed] [Google Scholar]
  61. Walters SM, Rivera AV, Reilly KH, Anderson BJ, Bolden B, Wogayehu A, . . . Braunstein S (2018). Exchange Sex Among Persons Who Inject Drugs in the New York Metropolitan Area: The Importance of Local Context, Gender and Sexual Identity. AIDS and Behavior, 22(9), 2773–2787. doi: 10.1007/s10461-018-2039-z [DOI] [PubMed] [Google Scholar]
  62. Wilson IB, Lee Y, Michaud J, Fowler FJ, & Rogers WH (2016). Validation of a New Three-Item Self-Report Measure for Medication Adherence. AIDS and Behavior, 20(11), 2700–2708. doi: 10.1007/s10461-016-1406-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Zhang J, & Yu KF (1998). What’s the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA, 280(19), 1690–1691. doi: 10.1001/jama.280.19.1690 [DOI] [PubMed] [Google Scholar]

RESOURCES