Abstract
Background
We explore injecting risk and HIV incidence among PWID in New York City (NYC), from 2012 to 2019, when incidence was extremely low, <0.1/100 person-years at risk, and during disruption of prevention services due to the COVID-19 pandemic.
Methods
We developed an Agent-Based model (ABM) to simulate sharing injecting equipment and measure HIV incidence in NYC. The model was adapted from a previous ABM model developed to compare HIV transmission with “high” versus “low” dead space syringes. Data for applying the model to NYC during the period of very low HIV incidence was taken from the “Risk Factors” study, a long-running study of participants entering substance use treatment in NYC. Injecting risk behavior had not been eliminated in this population, with approximately 15 % reported recent syringe sharing. Data for possible transmission during COVID-19 disruption was taken from previous HIV outbreaks and early studies of the pandemic in NYC.
Results
The modeled incidence rates fell within the 95 % confidence bounds of all of the empirically observed incidence rates, without any additional calibration of the model. Potential COVID-19 disruptions increased the probability of an outbreak from 0.03 to 0.25.
Conclusions
The primary factors in the very low HIV incidence were the extremely small numbers of PWID likely to transmit HIV and that most sharing occurs within small, relatively stable, mostly seroconcordant groups. Containing an HIV outbreak among PWID during a continuing pandemic would be quite difficult. Pre-pandemic levels of HIV prevention services should be restored as quickly as feasible.
Keywords: Person who inject drugs (PWID), New York city, Agent Based Modeling (ABM), Risk factors study, COVID-19, Covid
1. Introduction
Calls for “ending the HIV epidemic” among persons who inject drugs (PWID) have been made at the state, national, and international levels (The White House, 2021).
Public-health scale implementation of “combined prevention and care for HIV among persons who inject drugs (PWID),” including syringe access programs, medication assisted treatment for opioid use disorders, and anti-retroviral treatment for HIV seropositive persons, has led to very low HIV incidence—from 1/100 person-years to <1/1000 persons in PWID populations in many high income countries (Des Jarlais et al., 2012) and also in Viet Nam, a middle income country (Des Jarlais et al., 2021, Des Jarlais and Duong, 2018). These very low incidence rates can be considered as an “end of the HIV epidemic” in their local areas (Des Jarlais et al., 2016b).
These very low HIV incidence rates have been achieved despite continuation of injecting risk behaivor (syringe sharing) among moderate numbers of PWID and without complete elimination of HIV transmission in the local PWID population. Even in PWID populations that are well aware of the dangers of becoming HIV positive and have good access to sterile syringes, it is probably not possible to completely eliminate syringe sharing. With many PWID injecting multiple times per day, situations in which groups of PWID do not have enough injection equipment to avoid sharing will occasionally occur. More importantly, there are strong interpersonal dynamics that can promote sharing injection equipment. Sexual partners who do not practice safer sex may believe that they might as well share injection equipment (Dasgupta et al., 2019, de Oliveira Cintra et al., 2006, Morris et al., 2014). Close friends who work together to obtain and use drugs may feel that they already share a common fate, and that blood borne viruses are simply part of that common fate (Eicher et al., 2000, Rhodes et al., 2003).
Despite the stability of very low HIV incidence among PWID in many areas, there have also been outbreaks of HIV transmission among PWID in over 20 high income areas (Des Jarlais et al., 2020; Lyss et al., 2020). Common factors in these outbreaks have been reductions in prevention services, increase in homelessness among PWID and introduction of new injectable drugs with shorter duration of action, thus requiring more sterile injection equipment. We should emphasize, however, that almost all of these should be considered “outbreaks,” with the numbers of excess infections less than a hundred and not as “epidemics” with the numbers of new infections in the multiple hundreds to thousands.
The COVID-19 pandemic has created special concerns for HIV prevention, as the pandemic recreates many of the factors (disruption of prevention services, increased homelessness among PWID, and introduction of new injectable drugs) that generated the pre-COVID-19 outbreaks noted above. For instance, from March 2021 to March 2022, there was an HIV outbreak in Boston, MA, linked to the COVID-19 pandemic with over 60 new HIV cases among PWID (Freyer FJ, 2021; Srikanth A, 2021).
A better understanding of how situations occur with stable low HIV prevalence and stable very low HIV incidence but also with persistent syringe sharing should help inform efforts to “end the HIV epidemic” in many PWID populations. A better understanding of how outbreaks can occur in situations of very low incidence will be critical to maintaining “end of the HIV epidemic” situations. Our ability to conduct research on such situations, however, is necessarily limited. It would neither be practical nor ethical to conduct randomized controlled trials by varying syringe sharing parameters to identify the differences between the situations with stable low HIV transmission versus situations with outbreaks of HIV transmission among PWID. Mathematical modeling of HIV transmission among PWID offers a method for examining situations of stable very low HIV incidence and possible disturbances of such situations (Mumtaz et al., 2018).
Here we use an agent-based model (ABM) to explore injecting risk and HIV incidence among PWID in New York City, during a period (2012–2019) when incidence was extremely low, <0.1/100 person-years at risk (Des Jarlais et al., 2016a). We then extend the model to include consideration of how the COVID-19 pandemic might disturb stable very low HIV incidence and generate an outbreak of HIV among PWID in the city.
1.1. Timeline of the HIV epidemic among PWID in New York City
New York City experienced the first and world’s largest local epidemic of HIV among PWID. A brief timeline of the HIV epidemic among PWID in New York City will provide the historical context for the present modeling study.
1. Mid-1970s to early 1980s: HIV introduced into the PWID population and spread rapidily, reaching 50 % by the early 1980s
2. Mid-1980s to early 1990s: Early risk reduction among PWID, but very limited access to sterile injection equipment; estimated HIV incidence 4–6/100 PY.
3. Mid-1990s to early 2000s: Legalization, funding and expansion of syringe accesss programs. There were large reductions in syringe sharing and the residual sharing was often confined within small stable groups of close friends, sexual partners and relatives, and that persons who knew that they were HIV seropositive avoided “distributive sharing” (passing their used syringes to others) (Perlis et al., 1997). Estimated incidence declined from 4 to 1/100 PY (Des Jarlais et al., 2004).
4. Mid-2000s to 2019: Provision of ART to a “Treatment as Prevention (TasP)” level, with the percentage of HIV seropositive PWID who received ART reaching 80 %. Expansion of buprenorphine treatment. Incidence declined to 0.01/100 PY and remained stable from 2012 to 2019.
5. 2020–2022: Many HIV prevention and treatment services initially reduced by COVID-19 pandemic. Increased homelessness among PWID (Glick et al., 2020). Estimated HIV incidence is uncertain due to the great reduction in HIV testing.
2. Methods
We developed an agent-based model (ABM) to simulate sharing injecting equipment among PWID in NYC.
Agent-based models are microsimulations that reproduce individual behaviors in the context of social networks and account for environmental and individual risks. In our model we simulate communities of PWID where individuals form injecting networks, can share syringes and equipment, and transmit HIV from one person to another. These models allow detailed description of injecting practices where HIV transmission risk is decomposed into individual components including the use of ART and changing injecting networks. Injecting networks in our model have different sizes and connectivity and implement different injecting norms (high and low probability of sharing and sharing with “buddies” vs. “strangers”). Flexibility in network structure allows us to imitate a variety of areas (e.g. urban or rural) when more information about specific structures is known. In model development, we leveraged our past experience with ABMs that simulated HIV transmission among PWID (though none of these prior studies was specific to HIV among PWID in New York City) (Zule and Bobashev, 2009; Zule et al., 2018; Bobashev et al., 2010, 2019; Bobashev et al., 2019; Bobashev and Zule, 2010).
Data for applying the model to New York City was taken from the “Risk Factors” study, a long-running serial cross-sectional study of PWID entering the detoxification and methadone maintenance programs of Mount Sinai Beth Israel Medical Center. The Risk Factors study has monitored HIV prevalence, estimated HIV incidence, and risk behaviors among PWID in New York City since very early in the HIV epidemic (Des Jarlais et al., 1989, Des Jarlais et al., 1994). Measures included demographic characteristics, drug use, and HIV risk behaviors (receptive and distributive syringe sharing, unprotected sex) and size of sharing networks.
Data from the Risk Factors study has tracked closely with data from other HIV studies among PWID in NYC (Murrill et al., 2001, Neaigus et al., 2017, Thomas, 2001). Table 1 presents demographic characteristics and drug use behaviors for Risk Factors participants from 2012 to 2019, a period when HIV incidence was very low (< 0.1/100 person-years) (Des Jarlais et al., 2016a). Most importantly, a cohort study conducted within the Risk Factors study showed close agreement with HIV incidence among PWID based on the surveillance data of the New York City and New York State Departments of Health (data presented below).
Table 1.
N | % | |
---|---|---|
Total | 1009 | 100.0 |
Average Age (SD) | 41(10.4) | |
Gender | ||
Male | 853 | 84.5 |
Female | 153 | 15.2 |
Other | 3 | 0.3 |
Race/ethnicity | ||
White | 366 | 36.3 |
Black | 154 | 15.3 |
Latinx | 449 | 44.5 |
Asian/Pacific Islander | 1 | 0.1 |
Native American | 2 | 0.2 |
Mixed | 22 | 2.2 |
Other | 15 | 1.5 |
Last 6 months heroin injected | ||
Yes | 972 | 96.3 |
No | 35 | 3.5 |
Last 6 months cocaine injected | ||
Yes | 435 | 43.1 |
No | 573 | 56.8 |
Primary drug injecting: heroin | ||
Yes | 717 | 71.1 |
No | 292 | 28.9 |
Primary drug injecting: cocaine | ||
Yes | 43 | 4.3 |
No | 966 | 95.7 |
Daily injection in the last 6 months | ||
Yes | 739 | 73.2 |
No | 270 | 26.8 |
Specific model assumptons are described below:
-
1.
In our model the agents were arranged in clustered networks (i.e., more intense syringe sharing with the members of the same networks (buddies) and only occasional sharing with the members of other networks (strangers)). These strangers represent the common pool of PWID and facilitate the spread of HIV from one network to another. In our model the structure of networks (i.e., who injects with whom) and within-network risk behavior (i.e., how often agents share syringes) was drawn at random from uniform distributions. These networks were of different sizes.
Assumptions 2 – 8 are based on Risk Factors study data. Special data runs were conducted on the 1000 plus Rsik Factors subjects studied from 2012 to 2018; demographic characteristics are presented in Table 1 above.
-
2.
We assume that 85 % of PWID in NYC have good access to syringes from syringe service programs and through pharmacy sales and their sharing rate is very low, about 1 out of 200 injections.
-
3.
An additional 10 % of PWID regularly share syringes (in 50 % of injections) within small, closed groups (e.g. sex partners, close friends and relatives) of size 5 or less. These small groups are typically serosorted with all members HIV seronegative or all seropositive. However, once every five years the groups pick up a new member and lose an old member. We assume that being on ART reduces the likelihood of transmitting HIV on average by two orders of magnitude (99 %). Sharing occurs in half the injections, and random as to who shares with whom for any given injection.
-
4.
The remaining 5 % of PWID are assumed to share within larger (6–15 people) groups with an average of 10 persons per group but with 10 % turnover per month. In these groups, 10 % of injections are shared, with a 1 % chance that the sharing partner is relatively likely to transmit HIV (HIV seropositive, not on ART and willing to share syringes).
-
5.
Based on Risk Factors data, HIV prevalence is assumed to be 7 %; however about 60 % of HIV positive are on ART and thus, only 2.8 % of PWID population are relatively capable of transmitting HIV.
-
6.
Based on Risk Factors data, we assume that individuals inject on average 2 times per day drawn from a distribution ranging between 0 and 4 times a day.
-
7.
We assume that only a fraction (e.g., 0.5) of the individuals in a network are present at a particular injecting episode. In simulation we only track injections that result in sharing syringes and do not consider injections alone.
-
8.
We consider an annual removal rate of 4 % in the population. Individuals may be removed because they die or stop injecting. To keep the population stable and have the same denominator for incidence and prevalence calculations, individuals who leave the population are replaced with HIV-negative individuals on the basis that almost all new PWID are unlikely to have been exposed to the virus.
-
9.
Following Jacquez et al. (Jacquez et al., 1994), and Fiebig et al. (Fiebig et al., 2003) we assume that during the acute stage of being HIV seropositive the probability of transmission increases between 5 and 30 times. For simplicity we use the value of a 10-fold increase. The use of ART, however, reduces the infectivity by a factor of 99 %. It is not reduced to zero because we consider occasions when the use of ART is intentionally or unintentionally interrupted.
-
10.
We assume that everyone in NYC is using low dead-space syringes and powdered heroin. “Dead space” refers to the volume between the plunger and the needle when the plunger is fully inserted into the barrel of the syringe. A “high dead space” syringe has a greater dead space volume, thus increasing the risk that a second user of the syringe will be exposed to infectious material. The dead space syringe assumption is justified by years of ethnographic and epidemiological research (Zule et al., 2010, 2018) that show that in the US the vast majority of syringes are low dead space, and especially in NYC where low dead space syringes have been used for more than a decade. High dead space syringes and unsafe injecting practices are still occasionally found in the United States, but in places where access to syringe exchange is illegal and thus substantially limited (e.g. Outbreak in Scott County, IN in 2014) (Broz et al., 2018). We follow Zule et al., (2009, 2018) (Zule and Bobashev, 2009, Zule et al., 2018) and Bobashev et al., (2010, 2019) (Bobashev et al., 2019, Bobashev and Zule, 2010) and consider estimates of HIV transmission per shared injection after an HIV-infected not on ART person being p = 0.00008.
-
11.
Sexual transmission: Following a CDC report (Centers for Disease Control and Prevention, 2017) we consider sexual HIV incidence among PWID to be 2 per 10,000 person years. We implement that transmission as a random chance for each individual.
Our model does not consider a number of factors that affect the dynamics of HIV spread, such as behavior change immediately after becoming HIV positive or the number of times syringes are reused without sharing. We do, however, consider that after 6 months since becoming HIV positive, an individual starts using ART with a probability of 60 %. A list of model parameters is presented in Table 2, with their sources. Annual and monthly rates in the model are recalculated to represent daily probabilities. For example, the annual rate of 0.04 will result in the daily rate of 1-(1–0.04)1/365 = 0.00011.
Table 2.
Model Parameter | Value | Source |
---|---|---|
Initial HIV prevalence | 0.07 (range 0.03–0.1) | Risk Factors data |
Number of networks | 64 (range 1–200) | Experimental parameter |
Size of network cluster | range: 2–15 | Risk Factors data |
Proportion of people in the cluster participating in sharing | 0.5 (range 0.3–0.8) | Experimental parameter |
Number of times sharing with buddies | High risk: 1 per day Low risk: 1 in 90 days |
Risk Factors data |
Number of times injecting with a stranger | High risk: 10 times per year (range 8–20) Low risk: 1 time per year (range 0–2) |
Experimental parameter |
Removal rate (includes PWID HIV+ all-cause mortality and PWID leaving the population (i.e., stop injecting)). | 0.04 per year (range 0.02–0.06) | Bailey et al. (2007), Bailey et al., (2007), De et al. (2007),De et al. (2007) |
Risk multiplier for an acute stage of HIV infection | 10 (range 5–30) | Jacquez et al. (1994), Jacquez et al. (1994), Fiebig et al. (2003), Fiebig et al. (2003) |
Rate of sexual HIV | 2 * 10−4 per year (range 1 *10−4-3 *10−4) | CDC (2021) (Centers for Disease Control and Prevention, 2017; Grey et al., 2016). Incidence rates through sexual contacts and other sources |
Population | 10000 | Risk Factors data |
Proportion small sharing clusters | 10 % | Risk Factors data |
Proportion large sharing clusters | 5 % | Risk Factors data |
Initial prevalence | 0.07 | Risk Factors data |
HIV+ | Random Binomial with mean 700 | Population size multipled by the prevalence 10000 * 0.07 |
Proportion on ART and greatly reduced likelihood of transmitting | 0.6 | Risk Factors data |
Proportion on ART and greatly reduced likelihood of transmitting | Random Binomial with mean 420 | Number of HIV+ multipled by the ART proportion 700 * 0.6 |
Proportion HIV+ who are HIV seropositive, sharing, and not on ART | 0.4 | Risk Factors data |
Number of those who are infectious (HIV seropositive, sharing, and not on ART) | Random number with mean 280 | Number of HIV+ minus the number of those on ART 700–420 |
Proportion HIV+ who are HIV seropositive, sharing, and not on ART and share within a small cluster | 0.5 | Risk Factors data |
Syringe sharing networks | ||
Small size | 2–5 | Risk Factors data |
Large size | 6–15 | Risk Factors data |
Turnover. Each HIV seropositive PWID randomly moves to a new cluster once in 5 years | 1825 days | Risk Factors data |
Sharing in small group | ||
Proportion of injections shared | 0.5 | Risk Factors data |
Frequency of injection shared | 1/day | Proportion of network participating in sharing multiplied by the number of shares per day 0.5 * 2/day |
Behavior | ||
HIV- injecting frequency | 2/day | Risk Factors data |
Proportion among HIV seronegative PWID | 0.85 | Risk Factors data |
Sharing frequency among not-sharing | 0.01 (1/200 injections or 0.01/day given 2 injections/day or 3 times a year) | Risk Factors data |
+ Annual and monthly rates in the model are re-calculated to represent daily probabilities; for example, an annual rate of 0.04 will result in the daily rate of 1-(1–0/04)1/365 = 0.00011
*“Experimental” parameter means that the value of the parameter varies between different geographic areas, cultures, etc. For illustration purposes, we use a value that is considered reasonable. We use experimental parameters to design “what if experiments” aimed at evaluating intervention strategies and conducting sensitivity analysis.
2.1. Model overview and agent rules in the model
The model consists of 10,000 agents, some of which are linked in “buddy” networks with whom they share syringes and others are called “singletons” and do not belong to any syringe sharing networks. Each agent is assigned an HIV status; HIV seropositive agents are also assigned the stage (acute or not) and ART status. Time is modeled as discrete steps representing one day with the time horizon of 1 year. At each time step, the model loops over networks, and within the networks a sharing event either occurs or not. If sharing occurs, then the model selects individuals involved in sharing and the order at which they share. A person injecting with the same syringe after an infected individual has injected can get infected. It is also checked if there will be sharing with a “stranger” which is a randomly chosen agent form the population of agents. Singletons can only share with a stranger. If a syringe is used after an HIV positive individual it can transmit the virus to an HIV negative person who injects next. At each time step an agent also has a very small chance to contract HIV through sexual transmission. If an agent gets infected after a few months with a certain probability they will received ART treatment. There is a small probability each agent from a large network will move to a different network and get a new set of buddies.
At each time step information is recorded about each agent’s HIV status time when they got infected and to which type of network they belonged. At the end of the year the new HIV prevalence and incidence values are calculated. We thus simulated a number of scenarios summarized in Table 3. The baseline scenario corresponds to our best estimates of the parameter values before COVID-19 pandemic. Because populations can vary in terms of levels of risky behaviors, we considered sensitivity analysis where we considered lower and higher risky behaviors defined in terms of frequencies sharing with strangers, changing networks, sexual incidence and the proprortions of populations in the large sharing networks. For each scenario we ran 500 replications to calculate means and standard deviations for HIV incidence and track incidence variability over time. For each of the 500 runs we calculated the incidence and counted the numbers of runs with incidence higher than 10/10,000 and then divided that number by 500. So if 15 runs produced an HIV incidence >10/10000, the probability will be 0.03. Each scenario started with exactly 7 % HIV prevalence to avoid bias of initial prevalence, but the network structures were randomly generated in each of the runs. The model was programmed in NetLogo to provide interactive visualization of HIV transmission in a community.
Table 3.
Scenario Average parameter values |
Incidence per 10,000 person years mean, range, standard deviation |
Probability of reaching incidence of 10 or more individuals |
---|---|---|
Baseline risk scenario Sharing with strangers: 3 times a year Changing networks once in 5 years HIV incidence through sexual contacts 2 per 10,000 person years Proportion of individuals in large sharing groups 0.05 |
4.6 (range 0 – 11) std. 2.3 | 0.03 |
Low risk scenario Sharing with strangers: 1 time a year Changing networks once in 5 years HIV incidence through sexual contacts 1 per 10,000 person years Proportion of individuals in large sharing groups 0.05 |
2.0 (range 0 – 8) std. 1.6 |
<0.01 |
High risk scenario (without COVID-19 Sharing with strangers: 5 times a year Changing networks once a year HIV incidence through sexual contacts 2 per 10,000 person years Proportion of individuals in large sharing groups 0.1 |
5.6 (range 0 – 16) std. 2.4 | 0.06 |
COVID-19 baseline scenario Sharing with strangers: 4 times a year Changing networks once in 1 years Injecting 3 times a day HIV incidence through sexual contacts 2 per 10,000 person years Proportion of individuals in large sharing groups 0.15 |
7.5 (range 1–20) std. 2.8 | 0.23 |
COVID-19 high risk scenario Sharing with strangers: 7 times a year Changing networks once in 0.5 years Injecting 3 times a day HIV incidence through sexual contacts 2 per 10,000 person years Proportion of individuals in large sharing groups 0.15 |
8.5 (range 2–22) std. 2.9 | 0.33 |
*Sensitivity analysis shows that one of the most influential parameters is non-surprisingly sexual transmission of HIV because it is bringing HIV independently of injecting contacts. This factor is also interacting with the spread parameters such as the size of large sharing groups and the number of times a person shares with a stranger. Sharing with a stranger also has a direct effect on incidence because most of the random “strangers” are seronegative, and with more of them a seropositive individual shares syringes, the higher is the chance to spread. The size of the high-sharing group itself did not have much effect on the incidence because the proportion of such groups remains small, and even an outbreak remains localized. Similarly, higher changing of groups by a small number of highly contagious individuals contributes not much because they end up mostly in small groups of individuals and again the spread remains localized. Most of the transmission occurs during the acute phase before an individual gets tested and gets ART.
The COVID-19 pandemic was likely to alter the behaviors of PWID and we adjusted our model to reflect scenarios that are likely to arise from these situations. For COVID-19 scenarios we assumed that access to sterile syringes is reduced, so that the percentage of PWID who never share falls from 85 % to 70 %. The percent of those who share in small, stable groups increases to 15 %, and the percentage who share in large, unstable groups increases to 15 %. The proportion of shared injections within each of these two sharing groups increases by half. We also consider that during the COVID-19 pandemic the mean frequency of daily drug injections increases from 2 to 3, and HIV testing and access to ART is disrupted, so that rather than newly HIV positive persons getting on ART in 6 months, the period before ART increases to 12 months. These changes correspond to the studies of pre-COVID-19 HIV outbreaks in other cities (Des Jarlais et al., 2020) and early data on the pandemic effects in NYC (Pinto and Park, 2020).
2.2. Outcomes
We considered two main outcomes: the annual HIV incidence per 10,000 PWID and probabilities of annual HIV incidence rising from the pre-COVID-19 annual rate of 4 per 10,000 person-years to an outbreak level of 10 or more cases per 10,000 person-years.
3. Results
3.1. HIV incidence outcomes
Table 3 shows different scenarios with different HIV incidence rates generated by the model. The first three risk scenarios are our “baseline” scenario and “low” and “high” risk baseline scenarios. The second and third scenarios were sensitivity analyses to examine the effect on HIV of varying the frequency of injecting with strangers and the rate of the frequency with which individual PWID change sharing networks. Under the base case scenario, the probability of reaching an incidence of 10 cases per 10,000 person years was 0.03; in high-risk scenario it was 0.06. In the baseline COVID-19 scenario, however, it reached 0.23 and with an additional combination of high risk and COVID-19 it reached 0.33.
The primary baseline analyses and the two sensitivity analyses (for sharing with strangers and for group turnover) all generated very low HIV incidence rates, all below the 50/10,000 “end of an HIV epidemic” level.
For each of the 500 replications of the baseline simulation we tracked daily prevelaence over the year. Fig. 1 shows a sample of 200 trajectories of the 500 runs illustrating how variability in prevalence grows over time. The heavy line represents the mean of the runs. There is little variation in the modeled HIV prevalence among individual runs, mostly due to the large size of the PWID community, where the large majority is at very low risk. However, for smaller subpopulations the range could be much higher when an HIV outbreak occurs. For example, for a network of 8 people seroconversion of 2 individuals will add 25 % to the prevalence.
Table 4 shows the HIV incidence observed in three recent empirical studies of PWID in NYC.
Table 4.
Study title/methods | Dates | HIV incidence |
---|---|---|
Risk Factors Cohort | 2011–2014 | 0.00 per 10,000 PY (95 % CI = 0.00/10,000 PY – 9.6/10,000 PY) |
NYS SEA HIV Surveillance | 2011–2012 | 2/10,000 PY (95 % CI = 0.01/10,000 PY – 3.0/10,000 PY) |
NYC HIV Surveillance | 2011–2019 | Range from 8.0 to 2.3/10,000 PY |
*Risk Factors cohort study and New York State Department of HIV Surveillance Serologic Algorithm Estimation (Des Jarlais et al., 2016a) and New York City Department of Health and Mental Hygiene HIV Surveillance (New York City Department of Health (NYCDOHMH), 2020). The Risk Factors study included a modest percent (5 %) of PWID who reported men-with-men sexual behavior, and the two surveillance studies do not include the MSM-IDU transmission risk category.
All of the non-COVID-19 model generated incidence rates (primary analysis and two sensitivity analyses) fell within the 95 % confidence bounds of all of the empirically observed HIV incidence rates among PWID in the city.
We note that the non-COVID-19 modeled HIV incidence rates were obtained using the Risk Factors study prevalence and injecting risk behavior without any recalibration of the model to improve the fit with the observed empirical data on HIV incidence.
4. Discussion
The model does describe how a moderate percentage of the PWID population (15 %) may engage in syringe sharing, with the patterns of sharing leading to a very low but non-zero rate of HIV infections in the PWID population. Mean HIV incidence in these sensitivity analyses varied from 2.0/10,000 person-years to 5.6/10,000, compared to the model baseline mean incidence of 4.6/10,000 person-years and the empirically observed incidence of 4/10,000 person-years at risk among PWID in the city (Des Jarlais et al., 2016a).
4.1. Key components of maintaining very low incidence
In our assessment, there are two primary factors for the very low pre-COVID-19 HIV incidence observed in NYC. First, as a percentage of the total active injecting population, the percentage of PWID who are highly likely to transmit HIV—who are HIV seropositive, not on ART and engage in syringe sharing—is very low, less than 1 % of the PWID population. Thus, even if an HIV seronegative PWID were to share syringes with another PWID, the chance that this other PWID would be capable of transmitting HIV is quite low. This very low percentage can be achieved through a combination of multiple factors: a very low HIV seroprevalence (7 %), high ART coverage among HIV seropositive PWID (60 %), very low rates of distributive syringe sharing by HIV seropositive PWID and a generally low probability of transmission per sharing (0.00008) in a population where 90 % share on average 3 times a year and 10 % share 60 times a year. A second factor is that most of the syringe sharing that does occur happens within small, relatively stable groups of sexual partners, very close friends or relatives. This pattern of syringe sharing has been noted previously in empirical studies (Des Jarlais et al., 2004) and in previous modeling efforts (Vickerman et al., 2006). These small groups are likely to be seroconcordant (mostly HIV seronegative concordant) so that sharing within these groups would not lead to HIV transmission.
A “back of the envelope” calculation assuminig 10 % of high risk and 90 % low-risk individuals in a population of 10,000 would result in corresponding incidence of 0.07 * (1–0.6)* 3 * 0.00008 * 9000 + 0.07 * (1–0.6)* 60 * 0.00008 * 1000 = 2.1 per year per 10,000. With the addition of sexual transmission of 1/10,000, the crude estimate is about 3/10,000 per year. In reality, of course, variability of behavior, network structures, acute infectivity due to late testing, mortality, and other factors play an important role in modifying this crude estimate and adding variability to HIV incidence.
The above back of the envelope example provides a linear (at the log scale) and mean-field interpretation of the epidemic that does not account for a potentially clustered structure. For example, if there are large clusters of sharing networks, then if HIV non-attenuated by ART arrives to these clusters, we would expect an outbreak. In the model we specifically track who acquired HIV from whom and which subgroups contributed most to the HIV incidence. For example, singletons acquire HIV through injection by only sharing with strangers, but at the same time, there are many more singletons than individuals in sharing groups, so many risk factors are working linearly in this subpopulation. In large sharing networks the potential for infection is higher, but only a small percent of the population belongs to these networks. We would not expect a large epidemic or unusually large outbreaks just because the vast majority of the PWID population are low risk, and occasional outbreaks are within the stochastic variability. We observed a somewhat artificial interaction between the increase of sexual risk and network cluster size, which is somewhat expected. Sexual HIV risk in our model is acting as a random chance variable to acquiring HIV and equally (linearly) affects all agents, but if more individuals become HIV positive in large clusters, we would expect an outbreak.
4.2. Implications of COVID-19 disruptions of HIV prevention and care services
In the two COVID-19 disruption analyses, the probability of an outbreak of HIV among PWID—defined as incidence rising to 10 cases per 10,000 person-years—increased from 0.03 to between 0.25 and 0.33. Such an outbreak would not resemble the incidence of 4/100 person-years that existed prior to the implementation of syringe service programs in the city (Des Jarlais et al., 2005), rather it would be similar in size to most of the pre-COVID-19 outbreaks in multiple North American, Western European, and Middle Eastern cities (Des Jarlais et al., 2020; Lyss et al., 2020). However, the efforts needed to contain those pre-COVID-19 outbreaks—increased HIV testing and contact tracing, genotyping, assisting HIV seropositive PWID to ART, greatly increasing supplies of sterile injection equipment, increasing access to substance use treatment and housing, would be quite difficult to implement when public health personnel are already stressed by an ongoing COVID-19 pandemmic. Thus, the adverse public health consequences of an outbreak of HIV among PWID during a continuing COVID-19 pandemic would likely be considerably worse than those that occurred during the pre- COVID-19 era.
4.3. General implications for HIV prevention
There are several direct implications from this model for HIV prevention and care for PWID. First is value of redundancy in large-scale prevention programs for achieving very low numbers of PWID who are highly likely to transmit HIV to others. This number can be limited through (1) providing very good access to sterile injection equipment to reduce the need for syringe sharing, (2) providing medication for opioid use disorder (MOUD) to reduce injecting, and (3) providing ART to reduce infectiousness. As noted in the introduction, NYC has not achieved the 90–90–90 ART treatment goals for PWID—less than 40 % of PWID who are on ART are at viral suppression (New York City Department of Health, 2019). However, with the good access to sterile injecting equipment (with large syringe service programs and low-cost, non-discriminatory pharmacy sales) and the ready availability of MOUD (no waiting lists for entry into treatment), HIV incidence has remained very low since the early 2010s
A second implication is that sharing within small, relatively stable, injecting networks does not lead to widespread HIV transmission among PWID. This suggests that HIV prevention efforts should not focus on trying to eliminate sharing within such networks, but rather should address factors that may disrupt these networks, leading to PWID sharing syringes within much larger networks with more rapid turnover. Incarceration, economic insecurity, and homelessness would be factors that could disrupt the small, relatively stable sharing networks.
5. Limitations
Several limitations of the analyses reported here should be noted. First, the standard limitations that occur with all mathematical modeling of epidemics—that the model is a simplification of complex reality, that the model is dependent upon assumptions and on the quality of the input data—would apply to this model (Metcalf and Lessler, 2017).
A second limitation concerns social desirability effects on the PWID reported rates of risk behaviors. We suspect that syringe sharing by HIV seropositive PWID is under-reported in our data.
Third, we modeled injecting related HIV transmission in the PWID population of NYC as a whole. We did not attempt separate models for different demographic groups within the PWID population because the data for several of our key variables were too sparse for separate models. For example, less than 1 % of the Risk Factors sample of over 1000 PWID met the criteria for the key variable of “PWID likely to transmit HIV” (HIV seropositive, not on ART, and engaged in distributive sharing). This is consistent with the very low HIV incidence rate in the model and in the empirical studies (Table 3), but does not permit meaningful disaggregation by demographic subgroups. We believe that our results are likely to hold for major demographic subgroups (sex, race/ethnicity) but should not be applied to smaller demographic subgroups.
Fourth, we did not model sexual risk behaviors—particularly men-who-have-sex-with-men (MSM) sexual risk behaviors--that would generate sexual transmission of HIV among PWID. Rather we assigned an arbitrary incidence of 1/10,000 new sexual transmissions per year (Centers for Disease Control and Prevention, 2017). We did not have the data needed for including sexual behavior as a route of HIV transmission among PWID. The percentage of MSM-PWID in our Risk Factors study was quite modest, approximately 5%. We believe that sexual transmission, particularly among MSM-PWID will require a separate model. However, we would note that the low seroprevalence and the high percentage of HIV seropositives who were on ART would serve to limit sexual transmission.
Fifth, true validation is possible by observing the data over a period of time and evaluating the probability of observed numbers given the prediction. Predictive modeling techniques only illustrate the range of possible scenarios and probabilities of extreme events. The observed reality is a single realization of the underlying process and likely contains some randomness. So even the comparison of future observed trajectories with model predictions should be viewed as a comparison of two random processes. A number of models could produce similar results. Our justification is that we used (1) a well developed agent based model for HIV transmission among PWID, (2) the best-available data for the local NYC situation, (3) our extensive knowledge of the HIV situation among PWID in NYC, and (4) observed close matching between the model estimates of HIV incidence and the empirical data on HIV incidence among PWID in the city. This close matching was observed without having to revise either the behavioral rules for the PWID agents or the input data parameters.
Finally, we would again note the difficulties of precisely measuring very low rates of HIV incidence. For example, even with the large sample sizes in our cohort studies and in the NYCDOHMH and NYSDOH surveillance data (New York City Department of Health and Mental Hygiene, 2020; New York State Department of Health, 2019), the ranges for 95 % CIs for the observed HIV incidence were over an order of magnitude.
Despite these limitations we observed consistency between the modeled incidence to the empirical studies of HIV incidence and prevalence in New York City. The true validation fo the projected results could be only done when the future data is collected as was done, for example, in (Eaton et al., 2015) to evaluate 10 HIV forecasting models in Africa. We evaluated the steady state (i.e. situation when the incidence and prevalence does not change over time), degree of variability/uncertainty, and conducted sensitivity analysis to evaluate the impact of key parameters on the upward and downward trends.
We would also note that the COVID-19 pandemic clearly presents a challenge to maintaining HIV prevention services in the city (Glick et al., 2020).
6. Conclusions
We applied a previously developed agent-based model of HIV transmission in PWID populations to the current situation of very low HIV incidence in New York City. Drug injecting and risk behavior (syringe sharing) data were taken from the Risk Factors study, a large, long-running study of PWID in the city. The HIV incidence generated by the model closely matched the three available empirical estimates of HIV incidence in the city. The primary factors in the very low HIV incidence were the extremely small numbers of PWID likely to transmit HIV (PWID who are HIV seropositive, not on ART, and engaging in syringe sharing) and that the great majority of HIV seronegative PWID who are sharing syringes are doing so within small, relatively stable groups that would prevent long chains of HIV transmission. The model indicates that the COVID-19 pandemic greatly increases the likelihood of an outbreak of HIV among PWID, so that efforts to restore services should be of the highest priority.
Funding source
This work was supported through grant 5R01DA003574 from the US National Institute on Drug Abuse. The funding agency had no role in the design, conduct, data analysis or report preparation for the study.
CRediT authorship contribution statement
DDJ conceived the original research question for the study and wrote the first draft of the paper; GB performed modeling analysis and provided statistical support; JF and CM provided information on the Risk Factors data and contributed to literature review, referencing, and editing of the manuscript. All authors contributed to and have approved the final manuscript.
Declaration of Competing Interest
No conflict of interest declared.
References
- Bailey S.L., Ouellet L.J., Mackesy-Amiti M.E., Golub E.T., Hagan H., Hudson S.M., Latka M.H., Gao W., Garfein R.S. Perceived risk, peer influences, and injection partner type predict receptive syringe sharing among young adult injection drug users in five U.S. cities. Drug Alcohol Depend. 2007;91(Suppl. 1):S18–S29. doi: 10.1016/j.drugalcdep.2007.02.014. [DOI] [PubMed] [Google Scholar]
- Bobashev G., Mars S., Murphy N., Dreisbach C., Zule W., Ciccarone D. Heroin type, injecting behavior, and HIV transmission. A simulation model of HIV incidence and prevalence. PloS One. 2019;14(12) doi: 10.1371/journal.pone.0215042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bobashev G.V., Zule W.A. Modeling the effect of high dead‐space syringes on the human immunodeficiency virus (HIV) epidemic among injecting drug users. Addiction. 2010;105(8):1439–1447. doi: 10.1111/j.1360-0443.2010.02976.x. [DOI] [PubMed] [Google Scholar]
- Broz D., Zibbell J., Foote C., Roseberry J.C., Patel M.R., Conrad C., Chapman E., Peters P.J., Needle R., McAlister C., Duwve J.M. Multiple injections per injection episode: high-risk injection practice among people who injected pills during the 2015 HIV outbreak in Indiana. Int. J. Drug Policy. 2018;52:97–101. doi: 10.1016/j.drugpo.2017.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Centers for Disease Control and Prevention . In: HIV Risk and Prevention Estimates, in: CDC. Atlatna G.A., editor. 2017. [Google Scholar]
- Dasgupta S., Tie Y., Lemons A., Wu K., Burnett J., Shouse R.L. Injection practices and sexual behaviors among persons with diagnosed HIV infection WHO inject drugs—United States, 2015–2017. Morb. Mortal. Wkly. Rep. 2019;68(30):653. doi: 10.15585/mmwr.mm6830a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- De P., Cox J., Boivin J., Platt R., Jolly A. The importance of social networks in their association to drug equipment sharing among injection drug users: a review. Addiction. 2007;102:1730–1739. doi: 10.1111/j.1360-0443.2007.01936.x. [DOI] [PubMed] [Google Scholar]
- Des Jarlais D.C., Duong H.T. Ending HIV epidemics among people who inject drugs in LMICs. Lancet. 2018;392(10149):714–716. doi: 10.1016/S0140-6736(18)31721-5. [DOI] [PubMed] [Google Scholar]
- Des Jarlais D.C., Friedman S.R., Novick D.M., Sotheran J.L., Thomas P., Yancovitz S., Mildvan D., Weber J., Kreek M.J., Maslansky R., Bartelme S., Spira T., Marmor M. HIV-1 infection among intravenous drug users in Manhattan, New York City, from 1977 through 1987. JAMA. 1989;261:1008–1012. doi: 10.1001/jama.261.7.1008. [DOI] [PubMed] [Google Scholar]
- Des Jarlais D.C., Friedman S.R., Sotheran J.L., Wenston J., Marmor M., Yancovitz S.R., Frank B., Beatrice S., Mildvan D. Continuity and change within an HIV epidemic: injecting drug users in New York City, 1984 through 1992. JAMA. 1994;271(2):121–127. [PubMed] [Google Scholar]
- Des Jarlais D.C., Perlis T., Arasteh K., Hagan H., Milliken J., Braine N., Yancovitz S., Mildvan D., Perlman D.C., Maslow C., Friedman S.R. “Informed altruism” and “partner restriction” in the reduction of HIV infection in injecting drug users entering detoxification treatment in New York City, 1990-2001. J. Acquir Immune Defic. Syndr. 2004;35(2):158–166. doi: 10.1097/00126334-200402010-00010. [DOI] [PubMed] [Google Scholar]
- Des Jarlais D.C., Perlis T., Arasteh K., Torian L.V., Beatrice S., Milliken J., Mildvan D., Yancovitz S., Friedman S. HIV incidence among injection drug users in New York City, 1990 to 2002: use of serologic test algorithm to assess expansion of HIV prevention services. Am. J. Public Health. 2005;95(8):1439–1444. doi: 10.2105/AJPH.2003.036517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Des Jarlais D.C., Feelemyer J.P., Modi S.N., Arasteh K., Mathers B.M., Degenhardt L., Hagan H. Transitions from injection-drug-use-concentrated to self-sustaining heterosexual HIV epidemics: patterns in the international data. PLoS One. 2012;7(3) doi: 10.1371/journal.pone.0031227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Des Jarlais D.C., Arasteh K., McKnight C., Feelemyer J., Campbell A.N., Tross S., Smith L., Cooper H.L., Hagan H., Perlman D. Consistent estimates of very low HIV incidence among people who inject drugs: New York City, 2005-2014. Am. J. Public Health. 2016;106(3):503–508. doi: 10.2105/AJPH.2015.303019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Des Jarlais D.C., Kerr T., Carrieri P., Feelemyer J., Arasteh K. HIV infection among persons who inject drugs: ending old epidemics and addressing new outbreaks. AIDS. 2016;30(6):815–826. doi: 10.1097/QAD.0000000000001039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Des Jarlais D.C., Sypsa V., Feelemyer J., Abagiu A.O., Arendt V., Broz D., Chemtob D., Seguin-Devaux C., Duwve J.M., Fitzgerald M. HIV outbreaks among people who inject drugs in Europe, North America, and Israel. Lancet HIV. 2020;7(6):e434–e442. doi: 10.1016/S2352-3018(20)30082-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Des Jarlais D.C., Arasteh K., Huong D.T., Oanh K.T.H., Feelemyer J.P., Khue P.M., Giang H.T., Thanh N.T.T., Vinh V.H., Le S.M., Vallo R., Quillet C., Rapoud D., Michel L., Laureillard D., Moles J.P., Nagot N. Using large-scale respondent driven sampling to monitor the end of an HIV epidemic among persons who inject drugs in Hai Phong, Viet Nam. PLoS One. 2021;16(11) doi: 10.1371/journal.pone.0259983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eicher A., Crofts N., Benjamin S., Deutschmann P., Roger A. A certain fate: spread of HIV among young in jecting drug users in Manipur, North-East India. AIDS Care. 2000;12(4):497–504. doi: 10.1080/09540120050123891. [DOI] [PubMed] [Google Scholar]
- Eaton J.W, Bacaër N., Bershteyn A., Cambiano V., Cori A., Dorrington R.E., Fraser C., et al. Assessment of epidemic projections using recent HIV survey data in South Africa: a validation analysis of ten mathematical models of HIV epidemiology in the antiretroviral therapy era. The Lancet Global Health. 2015;3(10):e598–e608. doi: 10.1016/S2214-109X(15)00080-7. [DOI] [PubMed] [Google Scholar]
- Fiebig E.W., Wright D.J., Rawal B.D., Garrett P.E., Schumacher R.T., Peddada L., Heldebrant C., Smith R., Conrad A., Kleinman S.H. Dynamics of HIV viremia and antibody seroconversion in plasma donors: implications for diagnosis and staging of primary HIV infection. Aids. 2003;17(13):1871–1879. doi: 10.1097/00002030-200309050-00005. [DOI] [PubMed] [Google Scholar]
- Freyer F.J. Health officials strugge to contain Boston HIV outbreak. Boston Globe. Boston MA. 2021 [Google Scholar]
- Glick S.N., Prohaska S.M., LaKosky P.A., Juarez A.M., Corcorran M.A., DC D.J. The impact of COVID-19 on syringe services programs in the United States. AIDS Behav. 2020:1. doi: 10.1007/s10461-020-02886-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grey J.A., Bernstein K.T., Sullivan P.S., Purcell D.W., Chesson H.W., Gift T.L., Rosenberg E.S. Estimating the population sizes of men who have sex with men in US states and counties using data from the American Community Survey. JMIR Public Health Surveill. 2016;2(1) doi: 10.2196/publichealth.5365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jacquez J., Koopman J., Simon C., Longini I. Role of the primary infection in epidemic HIV infection of gay cohorts. J. Acquir. Immunodefic. Syndr. 1994;7:1169–1184. [PubMed] [Google Scholar]
- Lyss S.B., McClung B.K., Asher R.P., Oster AM A. Responding to clusters and outbreaks of HIV infections among persons who inject drugs — United States, 2016–2019. Recent Exp. Lessons Learn. Clin. Infect. Dis. 2020 doi: 10.1093/infdis/jiaa112. [DOI] [PubMed] [Google Scholar]
- Metcalf C.J.E., Lessler J. Opportunities and challenges in modeling emerging infectious diseases. Science. 2017;357(6347):149–152. doi: 10.1126/science.aam8335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morris M.D., Evans J., Montgomery M., Yu M., Briceno A., Page K., Hahn J.A. Intimate injection partnerships are at elevated risk of high-risk injecting: a multi-level longitudinal study of HCV-serodiscordant injection partnerships in San Francisco, CA. PLoS One. 2014;9(10) doi: 10.1371/journal.pone.0109282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mumtaz G.R., Awad S.F., Feizzadeh A., Weiss H.A., Abu-Raddad L.J. HIV incidence among people who inject drugs in the Middle East and North Africa: mathematical modelling analysis. J. Int. AIDS Soc. 2018;21(3) doi: 10.1002/jia2.25102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murrill C.S., Prevots D.R., Miller M.S., Linley L.A., Royalty J.E., Gwinn M. Incidence of HIV among injection drug users entering drug treatment programs in four US cities. J. Urb. Health. 2001;78(1):152–161. doi: 10.1093/jurban/78.1.152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Neaigus A., Reilly K.H., Jenness S.M., Hagan H., Wendel T., Gelpi-Acosta C., Marshall D.M.I. Trends in HIV and HCV risk behaviors and prevalent infection among people who inject drugs in New York City, 2005–2012. JAIDS J. Acquir. Immune Defic. Syndr. 2017;75:S325–S332. doi: 10.1097/QAI.0000000000001407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- New York City Department of Health . NYCDOHMH. HIV Epidemiology Program; 2019. Care and clinical status of people newly diagnosed with HIV and people living with HIV in New York City, 2018. [Google Scholar]
- New York City Department of Health (NYCDOHMH), 2020. HIV Surveillance Mid-Year Report, 2018, in: Program, H.E. (Ed.). New York City Department of Health.
- New York City Department of Health and Mental Hygiene , 2020. HIV Surveillance Annual Report, 2019, in: NYCDOHMH (Ed.). NYCDOHMH, New York City.
- New York State Department of Health, 2019. New York State HIV/AIDS Annual Report 2018, in: NYSDOH (Ed.). NYSDOH, Albany.
- de Oliveira Cintra A., Caiaffa W., Mingoti S., II, P.A.-B Characteristics of male and female injecting drug users of the AjUDE-Brasil II Project. Cad. Saude Publica. 2006;22(4):791–802. doi: 10.1590/s0102-311x2006000400018. [DOI] [PubMed] [Google Scholar]
- Perlis T.E., Friedman S.R., Rockwell R., Des Jarlais D.C. Distributive syringe sharing among HIV+ IDUs in N.Y.C. Am. Public Health Assoc. Indianap. 1997 [Google Scholar]
- Pinto R.M., Park S. COVID-19 pandemic disrupts HIV continuum of care and prevention: implications for research and practice concerning community-based organizations and frontline providers. AIDS Behav. 2020;24(9):2486–2489. doi: 10.1007/s10461-020-02893-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rhodes T., Mikhailova L., Sarang A., Lowndes C.M., Rylkov A., Khutorskoy M., Renton A. Situational factors influencing drug injecting, risk reduction and syringe exchange in Togliatti City, Russian Federation: a qualitative study of micro risk environment. Soc. Sci. Med. 2003;57(1):39–54. doi: 10.1016/s0277-9536(02)00521-x. [DOI] [PubMed] [Google Scholar]
- Srikanth A. HIV outbreak in Boston connected to COVID-19 pandemic. Hill. 2021 [Google Scholar]
- The White House . National HIV/AIDS Strategy for the United States 2022-2025. 2021. [Google Scholar]
- Thomas P. 25 years of HIV in New York City: lessons from surveillance. J. Urb. Health. 2001;78(4):669–678. doi: 10.1093/jurban/78.4.669. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vickerman P., Hickman M., Rhodes T., Watts C. Model projections on the required coverage of syringe distribution to prevent HIV epidemics among injecting drug users. J. Acquir Immune Defic. Syndr. 2006;42(3):355–361. doi: 10.1097/01.qai.0000219788.73539.47. [DOI] [PubMed] [Google Scholar]
- Zule W., Bobashev G. High dead-space syringes and the risk of HIV and HCV infection among injecting drug users. Drug Alcohol Depend. 2009;100(3):204–213. doi: 10.1016/j.drugalcdep.2008.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zule W.A., Pande P.G., Otiashvili D., Bobashev G.V., Friedman S.R., Gyarmathy V.A., Des Jarlais D.C. Options for reducing HIV transmission related to the dead space in needles and syringes. Harm Reduct. J. 2018;15(1):3. doi: 10.1186/s12954-017-0207-5. [DOI] [PMC free article] [PubMed] [Google Scholar]