Abstract
ISUM (“I’ll show you mine”) was a randomized controlled trial in which 272 transgender women and men who have sex with men in New York, NY (NYC) and San Juan, Puerto Rico (SJU) were assigned to an intervention group (n=136), in which they had access to free HIV self-testing (ST) kits, or to a control group (n=136). The trial aimed to determine whether the intervention group would use ST to screen sexual partners and have fewer condomless anal intercourse (CAI) occasions with serodiscordant or unknown status partners than the control group. The intervention group had on average 10 (32%) fewer CAI occasions; though clinically relevant, this difference fell short of statistical significance. In NYC (n=166) intervention participants had significantly fewer CAI occasions, whereas in SJU (n=106) they reported non-significantly more CAI occasions. Two devastating hurricanes hit SJU during the study and may have impacted results in unmeasured ways.
Keywords: HIV, self-testing, MSM, transgender women, prevention
RESUMEN
Te lo (“Te lo enseño”) fue un ensayo aleatorio controlado en el cual 272 mujeres transgénero y hombres que tienen sexo con hombres en la ciudad de Nueva York (NYC) y San Juan, Puerto Rico (SJU) fueron asignados a un grupo de la intervención (n=136), en el cual fueron provistos del autotest de VIH (ST) gratis, o a un grupo de control (n=136). El ensayo fue diseñado para determinar si los participantes del grupo de intervención usarían ST para testear a parejas sexuales potenciales y tendrían menos ocasiones de sexo anal sin condones (CAI) con parejas serodiscordantes o con un estatus desconocido comparados con los participantes en el grupo de control. El grupo de la intervención tuvo como promedio 10 (32%) ocasiones menos de CAI que el grupo de control; sin embargo, esta diferencia no tuvo significancia estadística. En NYC (n=166) los participantes del grupo de la intervención tuvieron significativamente menos ocasiones de CAI que el grupo de control, mientras que en SJU (n=106), reportaron un número de ocasiones de CAI mayor, aunque la diferencia no tuvo significación estadística. Dos huracanes devastadores afectaron a SJU durante el estudio y esto pudo haber impactado los resultados de la intervención de maneras no medidas.
INTRODUCTION
In New York City (NYC), HIV prevalence among ethnic minority men who have sex with men (MSM) has reached proportions comparable to sub-Saharan Africa (1), with estimates varying from 26.1% to 43.0% for African American MSM and from 33.5% to 34.9% for Latino MSM (2). Puerto Rico has the second highest rate of HIV infection among US territories and twice that of the 50 states and D.C., with males accounting for 65% of new infections and, among them, male-to-male sexual contact being associated with 30% of new infections (3). Furthermore, Puerto Ricans, who are US citizens, represent 32% of NYC’s Latino community (4) and often travel back and forth between NYC and the island, making it important to conduct prevention studies in both sites. In addition, a recent systematic review estimated prevalence of HIV infection among transgender women (TGW) to be 14.1% overall, with rates as high as 44.2% among racial minority TGW (5). These statistics demonstrate that HIV prevention interventions to date have not been successful at curbing the epidemic among TGW and ethnic minority MSM, and that new strategies are needed.
One such strategy could be HIV self-testing (ST), which entails collecting one’s own oral fluid or blood specimen, performing an HIV test, and interpreting test results without the need for technical training. Diagnostic accuracy of rapid ST is high, and self-testers can achieve similar results to health-care workers (6). The WHO recommends the use of ST as one strategy for achieving the first of the 90-90-90 targets for global control of HIV: making 90% of all people living with HIV aware of their HIV status (7). Furthermore, ST kits are progressively becoming available worldwide, and numerous international studies have documented their high acceptability among key populations (8-10).
ST can also be used as a harm-reduction tool in interpersonal situations. Although consistent condom use or pre-exposure prophylaxis (PrEP) can prevent HIV transmission, many individuals cannot or will not use condoms correctly or ingest systemic medication to prevent HIV infection (11-15). For them, the possibility of assessing a partner’s status using ST can decrease the risk intrinsic to relying on intuition or partners’ reports on HIV status (16-24). Although there is a window period during which a very recent infection cannot be detected, technological developments have shortened this window period to only a few weeks (25). Furthermore, mathematical modelling has shown that if MSM engage in anal intercourse without condoms following a non-reactive ST result, they have lower chances of becoming infected by someone still in the window period than by following heuristics and using condoms inconsistently (26-27).
Previous studies conducted by our group have shown that MSM are willing to use rapid ST to screen their potential partners (21) and that use of ST prevents HIV exposure in key populations (22). Yet, these prior studies did not include a control group to determine whether high-risk exposure is lower among individuals who use ST compared to those who do not. The study reported here is the first to measure such an outcome. The primary specific aim of our study was to determine if high-risk MSM and TGW who have access to ST to screen potential sexual partners engage in less sexual risk behavior --number of condomless anal intercourse (CAI) occasions with serodiscordant or unknown status partners-- than MSM and TGW who do not use ST as a screening device.
METHODS
The study’s field name was ISUM (“I’ll show you mine”), a pun on the idea of potential sexual partners showing each other their ST results. It was a 5-year, randomized controlled trial exploring the effectiveness of ST as a risk reduction tool for populations at high-risk of HIV infection. The study took place in NYC and San Juan, Puerto Rico (SJU), and was conducted by bilingual (Spanish/English) and bicultural staff in both sites. By eligibility criteria, participants had to be 18 years of age or older, HIV-negative, not taking oral PrEP at the time of recruitment (as this could have influenced their interest in testing partners), identify as an MSM or TGW, and report two or more sexual partners and three or more occasions of CAI with serodiscordant or unknown status partners in the prior three months.
Study Design
Fig. 1 shows the study design. Participants responded to a brief pre-screening survey by phone, e-mail, or in-person. Those who qualified were invited to an in-person screening (Visit 1) in which they signed the screening informed consent, completed a baseline behavioral questionnaire via an online computer administered self-interview (CASI), completed a rapid ST (OraQuick® In-Home HIV Test) following printed instructions without making mistakes, and received a confirmatory capillary (fingerstick) whole blood test (Alere Determine™ HIV-1/2 Ag/Ab Combo Test) administered by staff. Those who were eligible based on the questionnaire and test results were invited to return for enrollment within one week.
At Visit 2, participants signed the enrollment informed consent and were randomized to either the intervention or the control group. The randomization procedure was stratified by gender identity, to ensure that transgender participants were randomized into both groups in roughly equal numbers. The intervention group received ten rapid oral ST kits to take home and viewed an instructional video (https://www.youtube.com/watch?v=uq6Qb4BJLdM) that included key points for consideration when using the tests to screen sexual partners or clients (e.g., if and when to propose testing to a partner, the need to respect a partners’ decision not to be tested, and partners’ potential reactions that could include violence); these considerations were also included in an information card given to participants that instructed them to exercise their best judgment when deciding which partners to ask to test. The control group was neither given ST kits nor shown the video at Visit 2.
Both intervention and control group participants were offered condoms. All participants were also trained to use the short message service computer-assisted self-interview (SMS-CASI) system to reply to daily messages (see measures below), including the explanation of the abbreviations used in the messages to increase confidentiality, the incentive system for completed sessions, the follow-up call by staff that participants would receive if they did not respond to the system for three days, and the availability of research staff for troubleshooting problems with the SMS-CASI system if needed (see Brown et al. 2019 (28) for more details on the SMS-CASI system).
After three months, participants returned for a follow-up visit (Visit 3), in which they were tested for HIV using the Alere Determine™ HIV-1/2 Ag/Ab Combo Test and completed a follow-up online CASI. In addition, a subsample of participants in the intervention group was selected for an in-depth interview. Given that the primary aim of the study was to compare CAI behavior between the groups during the three-month intervention period, at Visit 3 follow-up those in the control group were given six ST kits to take home, were shown the video about ST use with partners, and finished study participation. The intervention group participants continued follow-up for three additional months (with no refill of ST kits). The objective of this follow-up was to assess changes in sexual risk behavior among intervention group participants following lack of facilitated access to ST –this being the secondary specific aim of the study. Intervention group participants continued reporting their sexual behavior through SMS-CASI and returned for a final visit (Visit 4) in which they were retested for HIV and completed a final online CASI assessment; those who had been interviewed in-depth at Visit 3 also completed a follow-up final in-depth interview during Visit 4.
Measures
All measures were available in English and Spanish. The daily SMS-CASI system was used to assess our main outcome: CAI with serodiscordant or unknown status partners (24). The SMS-CASI questions followed the script that appears in Fig. 2. Participants were first sent a question asking them about their readiness to report and to enter their password. Then, they were asked to report the number of CAI occasions they had had since their last report, the number of those occasions that took place with a seronegative partner, the number of occasions in which they saw a partner’s negative test results, and the number of unused test kits in their possession. Of note, by asking about number of CAI occasions “since last report,” participants were allowed to provide cumulative information for days in which they had missed sending reports. This helped to reduce missing data.
The baseline online CASI questionnaire included items on demographics, sexual behavior (including sex in exchange for money or other goods or services), motivations to remain HIV-uninfected, substance use, likelihood of use of a rapid ST, and perceived effectiveness and difficulty of discussing ST with partners. The follow-up online CASI administered at Visit 3 included items on sexual behavior, use of rapid ST, substance use and HIV testing, perceived effectiveness and difficulty of storing and using ST with partners, future ST use, and motivations to remain HIV-uninfected. Finally, the follow-up online CASI administered at Visit 4 for the intervention group participants included sexual behavior, use of rapid ST, future use of ST, and motivations to remain HIV-uninfected.
The in-depth interview probed about participants’ experiences using the tests with partners and clients, whether any partner tested positive, decision-making surrounding whom participants decided to test, participants’ assessments of future ST use for screening sexual partners, and important considerations for ST use among transgender women and sex workers. Qualitative study results will be reported elsewhere.
Procedures
A detailed description of recruitment strategies has been previously published (29). In brief, participants were recruited in-person at LGBT non-profit organizations, clubs, bars, or gay marches; online via social media and dating sites/apps; by email or phone using registries of participants from prior studies; and via word-of-mouth through other participants, who were given a $10 incentive for referring friends who enrolled in the study, for a maximum of $30.
Participants were compensated between $30 and $50 for study visits. They also received $1 per day per completed SMS-CASI session and a 50% bonus if at least six SMS-CASI per week were completed._All procedures were approved by the Institutional Review Boards at the New York State Psychiatric Institute [Protocol # 6854] and the University of Puerto Rico Medical Sciences Campus [Protocol # 0400115].
Data Monitoring and Cleaning
The daily SMS-CASI responses were monitored on a daily basis to identify errors or missed reports. SMS-CASI data were cleaned by a research assistant who marked and coded all errors. The data cleaning process was spot-checked by a second research assistant to ensure no errors were missed. These procedures are described in detail elsewhere (28).
Data Analysis
Descriptive statistics for sample characteristics at baseline were generated and compared between intervention and control groups using t-tests for continuous variables and Chi-square tests for categorical variables. The primary analysis used generalized linear model (GLM) with log link function to compare the number of CAI occasions with serodiscordant or unknown status partners following the intent-to-treat principle. An over-dispersion parameter was also included to account for between-subject heterogeneity. As we stratified randomization of participants to the groups based on site and gender identity, the primary analysis included both variables as covariates. The primary analysis model is of the form logμ =α0+ α1X1+α2X2+βΙ, where X1 denotes site indicator for NYC (vs. SJU), X2 is gender identity indicator for MSM (vs. transgender women), I is the group indicator for the intervention (vs. control). The regression coefficient β corresponding to the intervention group indicator estimates the logarithm of the two study-group population rate ratios (i.e., the ratio of the mean number of CAI occasions with serodiscordant or unknown status partners for intervention vs. control), and thus represents the effect of intervention on the primary outcome.
Note that while there were two sources to obtain the number of CAI occasions (i.e., through SMS-CASI and online CASI), following the statistical analytic plan stated in the grant proposal, the primary outcome was the one collected from SMS-CASI that, given its frequency, was expected to have less recall bias than a 3-month retrospective assessment. The online CASI self-report data showed no significant difference in any participant characteristics including the number of CAI occasions at baseline, and the randomization preserved the validity of the above analytic approach. In secondary analyses, we checked for a group-by-site and group-by-gender interaction to examine potential effect modifiers. Such effect modifiers could help “interpret” an observed significant/non-significant coefficient β. If found, we would further investigate whether cultural differences (such as participants’ likelihood to use rapid ST, perceived difficulty in discussing ST with partners, and perceived effectiveness of ST) could “explain” the site/gender difference in intervention effects. If not, then we would report the separate intervention effects (by site or gender) as our primary findings. Sensitivity analyses were also conducted using non-parametric methods to evaluate how the skewness of data impacted the study findings. While the inclusion of TGW provided greater generalizability of our study findings, the potential difference in partner type between MSM and TGW might impact our evaluation of the effect of the intervention. To address this, we would conduct additional sensitivity analysis by excluding TGW to understand whether findings would be comparable to the analysis used for the whole sample. Three participants had missing SMS-CASI primary outcome data. Following the protocol, we imputed the missing primary outcome with the self-report data from online CASI for two participants, and used the multiple imputation method to impute the remaining participant’s outcome in the absence of both SMS and online CASI outcome data. We report rate ratio (RR), ratio of two rate ratios (Ratio of RR) and their corresponding 95% confidence intervals and p-values. Findings with corresponding p-values no greater than 0.05 were considered as statistically significant. Analyses were conducted using SPSS version 25 (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY).
Qualitative data management, analysis and results from this study are reported in other publications (30-39).
RESULTS
Screening and Enrollment
The CONSORT diagram in Fig. 3 shows participant screening, enrollment, and retention during the study.
By study design, our intention was to recruit 300 participants, half in NYC and the other half in SJU. However, the devastation suffered by Puerto Rico due to Hurricanes Irma and María in 2017 (31, 40) affected our recruitment goals, resulting in 106 participants enrolled in SJU (n=53 intervention and n=53 control) and 166 in NYC (n=83 intervention and n=83 control), for a total of 272 participants.
Table 1 describes their demographic characteristics. Participants average age was 34 years old and had on average some college studies. They were mainly ethnic minorities, identifying mainly as men and gay. Nevertheless, 10% of the participants were transgender women. Two thirds were employed and close to one fifth were students.
Table 1.
Demographics | Intervention (n=136) |
Control (n=136) |
Full Sample (n=272) |
|
---|---|---|---|---|
Mean (SD) | Mean (SD) | t (df) p | Mean (SD) | |
Age | 33.85 (11.12) | 34.09 (11.16) | .180 (269) .857 | 33.97 (11.12) |
Level of educationa | 4.36 (1.26) | 4.46 (1.22) | .686 (270) .493 | 4.41 (1.24) |
Annual income | $24,668 (29,876) | $22,566 (26,087) | −.797 (242) .426 | $23,617 (28,007) |
N (%) | N (%) | χ2 (df) p | N (%) | |
Hispanic/Latino | 76 (56%) | 78 (57%) | 0.06 (1) .807 | 154 (57%) |
White | 39 (29%) | 43 (32%) | 0.24 (1) .625b | 82 (30%) |
Black/African-American | 64 (47%) | 44 (32%) | 108 (40%) | |
Asian | 3 (2%) | 3 (2%) | 6 (2%) | |
Native American | 1 (1%) | 2 (2%) | 3 (1%) | |
Other/More than one | 28 (21%) | 44 (32%) | 72 (27%) | |
Man | 123 (90%) | 122 (90%) | 0.04 (1) .839 | 245 (90%) |
Woman/Transgender | 13 (10%) | 14 (10%) | 27 (10%) | |
Gay/Homosexual | 102 (75%) | 110 (81%) | 1.37 (1) .242c | 212 (78%) |
Bisexual | 26 (19%) | 16 (12%) | 42 (15%) | |
Straight/Heterosexual | 4 (3%) | 3 (2%) | 7 (3%) | |
Other | 4 (3%) | 7 (5%) | 11 (4%) | |
Employed | 94 (70%) | 86 (63%) | 1.24 (1) .265 | 180 (66%) |
Student | 22 (16%) | 27 (20%) | 0.62 (1) .430 | 49 (18%) |
Education was measured on a scale where 1=eighth grade or lower, 2=partial high school, 3=high school graduate/GED, 4=partial college, 5=college graduate, 6=partial graduate school, 7=graduate school degree
White vs. Other
Gay vs. Other
Intervention Outcome
Table 2 shows the main outcome. Overall, the intervention group participants reported 32% fewer CAI occasions with serodiscordant or unknown status partners than the control group participants, but this difference fell short of statistical significance. We evaluated the intervention effect separately by site: In NYC, intervention group participants reported statistically significant fewer CAI occasions than control group participants; in SJU, intervention group participants reported a statistically non-significant greater number of CAI occasions than control group participants. Not shown in Table 2, the difference of intervention effect between the two sites was statistically significant (RRNYC/RRSJU=0.23, 95% CI=0.11, 0.47, p<.001).
Table 2.
CAI occasions w/ serodiscordant or unknown status partners |
Intervention Mean |
Control Mean |
Rate Ratio | 95% Confidence Intervals |
p |
---|---|---|---|---|---|
Total | 20.86 | 30.91 | 0.68 | 0.45-1.05 | .08 |
NYC | 14.12 | 39.58 | 0.38 | 0.22-0.64 | <.001 |
SJU | 31.42 | 17.49 | 1.64 | 0.99-2.69 | .06 |
There were no baseline site differences in terms of likelihood of using an over-the-counter ST to test a sexual partner and likelihood of raising the idea of using a rapid ST with a partner. We did find that, compared to the NYC site, there was a greater proportion of participants in SJU who reportedly would trust blood test results more, as opposed to trusting oral tests more or both equally (NYC 45% vs PR 58%, p= .047). Given that all three variables (likelihood of using ST kits, raising the idea of using a rapid ST with a partner, and trusting blood test results more) could be potential moderators of the intervention effects and help to explain the intervention by site interaction, we conducted further analyses. Yet, adjusting for any of those three variables or adjusting for all variables was insufficient to explain the site differences in terms of intervention effect, which remained significantly larger in NYC than in PR.
No significant group-by-gender interactions (RRwomen/RRman=1.15, 95% CI=0.40, 3.35, p=.80) were observed. Findings from sensitivity analysis using non-parametric methods, analysis of CASI data, and analysis excluding TGW showed similar results. The lack of overall significance of behavior change during the intervention precluded the analysis of maintenance of behavior change originally intended for V4. A separate publication will explore how the changes in behavior observed in the NYC cohort during the intervention may have been affected post intervention (V4) after free provision of ST kits was suspended.
Personal ST Use
During the three-month intervention period, 100 (78%) participants in the intervention group used the OraQuick® In-Home HIV test to test themselves. Although only intervention group participants had been provided with self-test kits, 17 (13%) control group participants also self-tested, having obtained tests from a community-based organization or HIV testing provider (8, 47%), a friend (5, 29%), another participant in the research study (2, 12%), a sexual partner (2, 12%) or by purchasing it (1, 6%) -- participants could choose all options that applied.
ST Use with Potential Sexual Partners
During the three-month intervention period, 114 (88%) participants in the intervention group used the OraQuick® In-Home HIV test to screen at least one potential sexual partner, compared to 8 (6%) in the control group. Three (38%) in the control group reported obtaining the tests to test their partners by purchasing them, three (38%) from a friend, two (25%) from a sexual partner, one (13%) from a community-based organization or HIV testing provider, and one (13%) from another participant in the research study. Characteristics of partners who were invited to self-test will be reported elsewhere.
Not every participant used self-tests with partners or used them with all partners: Sixteen (12%) participants in the intervention group did not use the test with a potential sexual partner (of the 16 participants, only two had no sex partners). Seventy-nine (62%) participants reported not discussing the rapid ST with some of their partners (Mean=10 partners, Mdn=6, SD=15, range = 1-100). Reasons given for not asking these partners to test included: not having the test kit on hand (n=31, 39%), believing the partner was HIV negative (n=28, 35%), feeling uncomfortable bringing up the test (n=24, 30%), not wanting to risk ending the sexual encounter (n=20, 25%), knowing they were not going to engage in anal intercourse (n=19, 24%), feeling that the partner might react negatively (n=15, 19%), being too drunk or high (n=10, 13%), and not wanting to go through the hassle of testing (n=10, 13%). Participants could also write in “other” reasons; those who did (n=16, 20%) included seeing alternative test results that proved negative results, partner saying they were HIV+ but undetectable, the partner being a steady one, deciding to use a condom, or forgetting to ask the partner to test. More details on sexual partners tested by our participants are presented in a separate manuscript.
Proposing ST to Potential Sexual Partners
Via Chat, Text, or Call
We asked participants in the intervention group about their experiences bringing up the ST and using it with sexual partners. Seventy-one (55%) participants proposed using the test to at least one potential sex partner when they were not with them in person, such as through a chat, call, text, or similar method. Among them, 16% always proposed the test through any of these methods, 23% often, 45% sometimes, and 17% rarely. Participants reported that in general many agreed to use the test, although some refused. Few got angry or upset when asked to use a test, and few just stopped responding/ended the communication. Finally, a few partners disclosed they were HIV positive.
In-Person
One hundred and eleven (86%) participants proposed using the test to at least one potential sex partner in person. Of these, 65% asked at least one partner whom they had just met and were going to have sex with for the first time; 41% asked at least one partner whom they already knew with whom they were going to have sex for the first time, 24% asked at least one lover or primary partner, and 72% asked at least one partner with whom they had had sex before. Of those who reported proposing test use to a partner, eleven people (10%) reported proposing the test to a partner who gave them money or gifts in exchange for sex. Among those who asked a partner to use a test, 94% reported at least one partner took the test, while 37% had at least one partner who refused to test, 34% had at least one partner who got angry or upset, and 6% had at least one partner who got physically violent. Nevertheless, cases of physical violence were infrequent: Out of 870 partners who were asked in person to use the test, only 16 (2%) got physically violent. Details of these experiences will be reported elsewhere. Finally, 11% had at least one partner disclose they were HIV positive.
Sexual Activity Following ST Use
Among 41 participants who had at least one partner who refused to use the test, 30 (73%) reported engaging in sexual activity with at least one of such partners. Among these participants, 25 (83%) had anal intercourse with at least one partner who refused to test, and 18 (60%) had CAI with at least one partner who refused to test. We did not assess respondent’s role (insertive, receptive, or both) in these CAI occasions.
Among participants whose partners tested themselves in front of the participants, 98% had at least one partner who received HIV negative results, 14% had at least one partner who received HIV positive results, and 2% had a partner who received inconclusive results. Among those who had a partner test negative, 100% had sex (oral or anal), 94% had anal sex, and 86% had CAI with at least one of them. Among those who had a partner test positive, 64% had sex (oral or anal), 50% had anal sex, and 36% had CAI with at least one of them. Finally, among the two participants who each had one partner with inconclusive HIV test results, one of them reported having sexual activity with the partner but no anal intercourse.
On average, participants also reported using the test with 1.85 (SD=4.44) people who were not potential sex partners.
Reactive ST Results
There were 24 potential partners who received HIV positive test results and detailed information was collected on 22 of them. Of those 22, 45% were individuals who participants had just met and with whom they were going to have sex with for the first time, 9% were people they knew but with whom were going to have sex with for the first time, 45% were people with whom they had had sex before, and none were lovers or primary partners. The average approximate age of these partners was 31 years old. Testing occurred at the participant’s home/apartment (n=10, 45%), at the partner’s home/apartment (n=9, 41%), in a hotel (n=2, 9%), and in a car (n=1, 5%). In 6 (27%) of the 22 cases, there was an attempt to contact study personnel, the hotline number in the HIV test kit, or other resources to help them deal with the situation. Participants reported that, to the best of their knowledge, among the 22 partners who tested positive, at least 12 (55%) of them sought follow-up confirmatory testing with a healthcare provider or at a clinic, while they were unsure whether seven (32%) of them sought confirmatory testing, and they believed that three (14%) of them did not seek confirmatory testing. On a scale of 1 (not stressful at all) to 10 (extremely stressful), participants gave an average rating of 6.4 (SD=2.6) regarding how stressful they found dealing with the partner receiving an HIV positive results. On a scale of 1 (not good at all) to 10 (very good), participants reported an average rating of 7.6 (SD=2.8) regarding how they felt about how they handled the experience.
Four participants (3%) received HIV positive test results while testing themselves. Three of them were testing themselves in front of a potential sexual partner at the time. One of them reported contacting study personnel, the hotline number in the kit, or accessing other resources while dealing with their HIV positive result. None of them sought confirmatory testing at a clinic or saw a healthcare provider.
Experiences with Testing Procedures
Seven participants (5%) reported having any difficulties performing the test, one (1%) reported difficulties interpreting the results, four (3%) reported calling the hotline number shown in the test kit, and one (1%) reported contacting study personnel due to problems using the test. Most common difficulties included the test kits getting damaged or not working properly. Most common reasons for calling the study hotline or contacting study personnel included getting a positive result or needing more tests.
DISCUSSION
The results of our randomized controlled trial show that participants in the intervention group had on average 10 (32%) fewer CAI occasions with serodiscordant or unknown status partners than the control group. Although this decrease may be clinically relevant, this difference fell short of statistical significance. We detected a significant group-by-site interaction. In NYC (total N=165), intervention group participants reported statistically significantly fewer CAI occasions than the control group. This difference (Mean CAIintervention=14.12, Mean CAIcontrol=39.58) may also have clinical significance. By contrast, in SJU (total N=107), the intervention group reported statistically non-significant greater number of CAI occasions. Consistent (significant) positive findings from analyses using various data sources (i.e., SMS-CASI or online CASI) and analytic approaches (i.e., parametric or non-parametric methods) provided evidence of beneficial intervention effect for the NYC site. Concerning SJU, it is not clear why participants in the intervention group reported having more (although not statistically significant) CAI occasions than the control participants, as sensitivity analysis excluding subjects with an exceptionally high number of partners or those who reported having received money or other goods in exchange for sex did not change the direction of intervention effect. Moderation analysis conducted with variables assessed at baseline concerning likelihood to use ST with partners or raise the idea with them, as well as trusting more blood than oral tests, did not explain intersite differences.
Hurricanes Irma and Maríia fell on Puerto Rico in 2017, with the latter reaching Category 5 intensity (41), bringing incredible devastation to the island (42) and leaving many residents without electricity, water and shelter at times for many months (43). This happened while our study was in the field, both for participant recruitment and follow-up. Although our analysis of the SMS-CASI data before and after the hurricanes struck the island showed that all participants were able to communicate sexual behavior and HIV testing via SMS-CASI within 30 days following María, that re-engagement within 30 days after the hurricane was 100%, and that retention of active participants was an amazing 100% (31), the extraordinary circumstances that affected Puerto Rican residents in the midst of our study cannot be underestimated. These include psychological and social reactions to stress, hardships, isolation, forced migration of network members to mainland US (44), heightened awareness of colonial status of the territory (45), and other unknown circumstances that may have impacted our study’s intervention effect in ways not captured by our measures. In fact, research on HIV risk behavior in the aftermath of stressful or traumatic life events, including hurricanes and terrorist attacks, has shown associated increases in both the numbers of sexual partners and the number of unprotected sexual occasions (46-48). This may also have been in the case in Puerto Rico following hurricanes Irma and María.
An alternative hypothesis is that cultural differences between the two study sites may be at play. As expected, 99% of participants in SJU identified as Latino, much fewer (30%) in NYC. Prior scientific literature has reported that Latino survey respondents, especially those responding to surveys in Spanish, are more likely to acquiesce than non-Latino European Americans (49), defined as the systematic selection of agreeable (“strongly agree”) or affirmative (“yes”) responses to survey items, regardless of item content or directionality (50). It is possible that respondents at the SJU site, who overwhelmingly responded to CASI in Spanish, might have stated at baseline that they would be very likely both to use an over-the-counter ST to test a sexual partner and to raise the idea with a partner due to acquiescence; yet, at the time of the sexual encounter, SJU respondents may have been less inclined to avoid CAI with partners whose HIV status they could not verify. However, our study was not designed to study acquiescence as a potential confound, an issue that should be considered for future studies.
Beyond the mixed results of our RCT, several other results merit attention. We succeeded in reaching key populations at high risk in both sites, as evidenced by their reported CAI with multiple partners and the number of individuals who believed at pre-screening that they were uninfected but found otherwise when we tested them prior to enrollment (28/368, 7.61%). This documents that concerted efforts to test at-risk individuals should continue, as this is key to link them to treatment to help achieve viral suppression, which is essential to remain healthy and prevent further viral transmission. Two-thirds of participants were ethnic minority transgender women and men who have sex with men, who are among the populations most highly affected by HIV in the US (51). Our study showed that not only can these individuals be reached, but also that they can be engaged in programs with very little attrition at 3 or 6 months.
Of the 1,419 potential study participants pre-screened, only 51 (<4%) said they were unlikely to use ST and were ineligible to attend visit 1. Of the 368 individuals who completed visit 1, only 6 (<2%) would not test themselves or their partners, therefore being ineligible to enroll in the trial. This shows that acceptability of ST was extremely high in our population of interest. Furthermore, study participants showed high motivation to use self-tests on themselves, given that not only 78% of those in the intervention group used the kits to ST, but also 13% of participants in the control group found a way to procure self-test kits to test themselves, despite the fact that all participants had tested themselves and received a confirmatory rapid HIV test with negative results at baseline. This shows that making ST kits available and affordable to at-risk individuals needs to be a key component for all campaigns trying to reach 90% HIV-status assessment. Furthermore, 76% of participants in the intervention group succeeded in getting a potential sexual partner to self-test. This shows that dissemination of ST through peer networks has an important potential to complement and potentially increase the reach of official DOH testing clinics and outreach programs.
Attention must be paid to the circumstances in which, despite having the ST available, some participants decided against asking their partners to test themselves. For example, the issue of not having the test kit on hand could be resolved if ST kits were more portable. The OraQuick® In-Home HIV Test approved by the FDA for marketing in the US, which we used in our study, comes packaged in a plastic box that measures 6 in x 7 in x 2 in (16 cm x 18 cm x 5 cm). The same OraQuick® ST is available outside the US in a much smaller refillable package that improves portability and could decrease the number of individuals who do not use ST due to not having one on hand. Other reasons for not asking a partner to use ST could be managed by improving self-efficacy and capacity-building among individuals at risk.
Interestingly, more than half of the participants in the intervention group chose to discuss the issue of using ST online with potential sexual partners. This is a positive sign, because it may reduce uncomfortable situations in face-to-face encounters with individuals not forewarned about the intended mutual use of ST. Nevertheless, the fact that 85% of participants proposed the use of ST in-person to a variety of potential partners demonstrates that they felt self-assured in using ST in interpersonal situations. Furthermore, some participants who engaged in commercial sex proposed the use of ST to their clients, a topic that deserves further exploration. Very few partners reacted violently to the proposed ST, which shows that this infrequent occurrence should not be considered a major barrier to mutual ST. Training on how to suggest ST to a partner (or when to abstain from doing so) and how to handle different potential reactions should be made widely available.
Of note, 18 participants had CAI with someone who refused to take the test. Prior studies (23, 52-54) show that trusting intuition or feeling reluctant to stop the progression of a sexual encounter may lead individuals to engage in potentially risky situations that they may later regret. More qualitative research is needed to determine the psychological, interpersonal and contextual situations that may lead individuals to engage in CAI with someone whose status is unknown and also refuses to take the test.
Participants who found that their potential sexual partners were HIV infected reported that the experience was only mildly stressful and that, overall, they felt good about how they handled it. This is important as one of the questions often raised to the use of ST in interpersonal situations is whether and how an individual will be able to handle a partner’s positive test result. Furthermore, equal proportions of positive ST results were obtained among new potential sexual partners and former sexual partners; this should raise the alert for individuals who believe that former sexual partners and the degree of familiarity achieved with them somehow mitigate infection risk. The experience of participants whose partners had reactive (positive) ST results and those who found themselves to be infected, as well as those of participants who decided to engage in CAI despite a partner’s positive result, will be reported in a separate manuscript that will include details on the in-depth interviews of such participants. Our study was not powered to assess the potential effect of the intervention on seroconversion.
Finally, very few participants had difficulties with testing procedures and almost none had problems with interpreting test results. The ease of use of ST technology can contribute to popularize its use.
Limitations
The major limitation of our study is the occurrence of two devastating hurricanes in Puerto Rico in 2017 while the study was on the field. This may have affected the study’s outcome in unanticipated ways. The remarkably efficient work of our team members in SJU to recontact participants as soon as possible following the catastrophe, to offer assistance, and to reengage them in the study prevented attrition and loss to follow up; yet, we were unable to systematically collect data on psychosocial effects of the hurricanes on our study population.
Second, participants had access to free ST during the study, and they could receive additional kits if they so requested. The current market cost of ST, which is around 40 USD in the US, would clearly be prohibitive for many people, especially considering that two test kits would be required for mutual testing. There is a need to explore how to decrease costs or provide free kits to individuals at high-risk of HIV infection.
Third, our study relied on self-reports. Although participants were thoroughly trained on the use of SMS-CASI, and the daily messages should have helped minimized recall bias, we have no objective means to determine whether the reported sexual behavior and ST use actually took place as described.
Finally, our participants were not randomly chosen. Results may not be generalizable to other MSM and TGW.
Conclusions
Our study shows that transgender women and men who have sex with men at high risk of HIV infection, including ethnic minorities, can be reached and engaged in HIV prevention studies with very low attrition for up to 6 months. Our findings demonstrate that they can be motivated to use ST with partners, succeed in convincing their partners to use ST, encounter very limited violent reactions, and identify previously undetected infected individuals. In such cases, individuals are able to deal with partners’ positive ST results satisfactorily. Furthermore, the results of the NY site indicate that using ST in this cohort resulted in a statistically significant decrease of CAI as compared to the control group, and the ST use may be a valuable risk reduction tool. More studies are needed to determine the extent to which ST use to screen sexual partners may decrease CAI and contribute to decrease transmission of HIV.
ACKNOWLEDGMENTS
We would like to thank the study participants for their time and effort. This work is funded by the Eunice Kennedy Shriver Institute of Child Health and Human Development and the National Institutes of Health [R01 HD076636] PI: Carballo-Diéguez]. The HIV Center for Clinical and Behavioral Studies is funded by an NIMH center grant [P30-MH43520 PI: Remien]. William Brown III was supported by the National Library of Medicine (NLM) [grant numbers R01-LM012355 PI: Schillinger, T15-LM007079 PI: Hripcsak, R01-LM013045 PI: Lyles], the National Institute on Minority Health and Health Disparities (NIMHD) [grant number P60-MD006902 PI: Bibbins-Domingo], the Agency for Healthcare Research and Quality (AHRQ) [grant number K12-HS026383], and the National Center for Advancing Translational Sciences (NCATS) of the NIH [UCSF-CTSI grant number KL2-TR001870] during various stages of the research and/or preparation of the article. Sarah Iribarren was supported by the National Institute of Nursing Research (NINR) [grant numbers T32 NR014205 PI: Stone, K23NR017210 PI: Iribarren] during various stages of the research and/or preparation of the article. The content is solely the responsibility of the authors and does not necessarily represent the official views of NICHD, NIMH, NLM, NIMHD, or the NIH.
Footnotes
Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.
REFERENCES
- 1.El-Sadr WM, Mayer KH, Hodder SL. AIDS in America – Forgotten but not gone. New Engl J Med. 2010. March;362(11):967–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Jenness SM, Neaigus A, Murrill CS, Gelpi-Acosta C, Wendel T, Hagan H. Recruitment-adjusted estimates of HIV prevalence and risk among men who have sex with men: Effects of weighting venue-based sampling data. Public Health Reports. 2011. September;126(5):635–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Miranda S, Lopez B, García-Rivera EJ, Rangel M, Hernandez AL, Espinoza L, An Q, Song R, Zhang R, Myles Z. Incidence and diagnoses of HIV infection-Puerto Rico, 2006. Morbidity and Mortality Weekly Report. 2009;58(21):589–91. [PubMed] [Google Scholar]
- 4."Hispanic or Latino by Type: 2010 Census Summary File 1". US. Census Bureau 2011. Archived from The original on December 1, 2011. Available at: https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=CF. Accessed October 25, 2019.
- 5.Becasen JS, Denard CL, Mullins MM, Higa DH, Sipe TA. Estimating the prevalence of HIV and sexual behaviors among the US transgender population: A systematic review and meta-analysis, 2006–2017. Am J Public Health. 2019. January;109(1):e1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Figueroa C, Johnson C, Ford N, Sands A, Dalai S, Meurant R, Prat I, Hatzold K, Urassa W, Baggaley R. Reliability of HIV rapid diagnostic tests for self-testing compared with testing by health-care workers: a systematic review and meta-analysis. Lancet HIV. 2018. June 1;5(6):e277–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Johnson C, Baggaley R, Forsythe S, Van Rooyen H, Ford N, Mavedzenge SN, Corbett E, Natarajan P, Taegtmeyer M. Realizing the potential for HIV self-testing. AIDS and Behavior. 2014. July 118(4):391–5. [DOI] [PubMed] [Google Scholar]
- 8.Krause J, Subklew-Sehume F, Kenyon C, Colebunders R. Acceptability of HIV self-testing: a systematic literature review. BMC Public Health. 2013. December;13(1):735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Figueroa C, Johnson C, Verster A, Baggaley R. Attitudes and acceptability on HIV self-testing among key populations: a literature review. AIDS Behav. 2015. November 1;19(11):1949–65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ng OT, Chow AL, Lee VJ, Chen MI, Win MK, Tan HH, Chua A, Leo YS. Accuracy and user-acceptability of HIV self-testing using an oral fluid-based HIV rapid test. PLoS One. 2012. September 17;7(9):e45168. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Carballo-Diéguez A, Bauermeister J. ‘Barebacking’ Intentional Condomless Anal Sex in HIV-Risk Contexts. Reasons for and Against It. J Homosex. 2004. May 19;47(1):1–6. [DOI] [PubMed] [Google Scholar]
- 12.Bauermeister JA, Carballo-Diéguez A, Ventuneac A, Dolezal C. Assessing motivations to engage in intentional condomless anal intercourse in HIV risk contexts (“bareback sex”) among men who have sex with men. AIDS Educ Prev. 2009. April;21(2):156–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Balán IC, Carballo-Diéguez A, Ventuneac A, Remien RH. Intentional condomless anal intercourse among Latino MSM who meet sexual partners on the Internet. AIDS Educ Prev. 2009. February;21(1):14–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Whitfield TH, John SA, Rendina HJ, Grov C, Parsons JT. Why I quit pre-exposure prophylaxis (PrEP)? A mixed-method study exploring reasons for PrEP discontinuation and potential re-initiation among gay and bisexual men. AIDS Behav. 2018. November 1;22(11):3566–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Namey E, Agot K, Ahmed K, Odhiambo J, Skhosana J, Guest G, Cornell A. When and why women might suspend PrEP use according to perceived seasons of risk: implications for PrEP-specific risk-reduction counselling. Cult Health Sex. 2016. September 1;18(9):1081–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Maman S, Murray KR, Mavedzenge SN, Oluoch L, Sijenje F, Agot K, Thirumurthy H. A qualitative study of secondary distribution of HIV self-test kits by female sex workers in Kenya. PloS One. 2017. March 27;12(3):e0174629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Thirumurthy H, Masters SH, Mavedzenge SN, Maman S, Omanga E, Agot K. Promoting male partner HIV testing and safer sexual decision making through secondary distribution of self-tests by HIV-negative female sex workers and women receiving antenatal and post-partum care in Kenya: a cohort study. Lancet HIV. 2016. June 13(6):e266–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Masters SH, Agot K, Obonyo B, Mavedzenge SN, Maman S, Thirumurthy H. Promoting partner testing and couples testing through secondary distribution of HIV self-tests: a randomized clinical trial. PLoS Med. 2016. November 8;13(11):e1002166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Matovu JK, Makumbi FE. Expanding access to voluntary HIV counselling and testing in sub-Saharan Africa: alternative approaches for improving uptake, 2001–2007. Trop Med Int Health. 2007. November;12(11):1315–22. [DOI] [PubMed] [Google Scholar]
- 20.Gichangi A, Wambua J, Gohole A, Mutwiwa S, Njogu R, Bazant E. Provision of oral HIV self-test kits triples uptake of HIV testing among male partners of antenatal care clients: results of a randomized trial in Kenya. In 21st International AIDS conference 2016 Jul 18 (pp. 18–22). [Google Scholar]
- 21.Carballo-Diéguez A, Frasca T, Dolezal C, Balan I. Will gay and bisexually active men at high risk of infection use over-the-counter rapid HIV tests to screen sexual partners?. J Sex Research. 2012. July 1;49(4):379–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Carballo-Diéguez A, Frasca T, Balan I, Ibitoye M, Dolezal C. Use of a rapid HIV home test prevents HIV exposure in a high risk sample of men who have sex with men. AIDS Behav. 2012. October 1;16(7):1753–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Balán IC, Carballo-Diéguez A, Frasca T, Dolezal C, Ibitoye M. The impact of rapid HIV home test use with sexual partners on subsequent sexual behavior among men who have sex with men. AIDS Behav. 2014. February 1;18(2):254–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Frasca T, Balan I, Ibitoye M, Valladares J, Dolezal C, Carballo-Diéguez A. Attitude and behavior changes among gay and bisexual men after use of rapid home HIV tests to screen sexual partners. AIDS Behav. 2014. May 1;18(5):950–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Factsheet: What is the window period for HIV testing? NAM Aidsmap. 2019. June Available at: https://www.aidsmap.com/page/1322978/. Accessed 1 July 2019.
- 26.Ventuneac A, Carballo-Diéguez A, Leu CS, Levin B, Bauermeister J, Woodman-Maynard E, Giguere R. Use of a rapid HIV home test to screen sexual partners: an evaluation of its possible use and relative risk. AIDS Behav. 2009. August 1;13(4):731–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Leu CS, Ventuneac A, Levin B, Carballo-Diéguez A. Use of a rapid HIV home test to screen sexual partners: a commentary on Ventuneac, Carballo-Dieguez, Leu et al. 2009. AIDS Behav. 2012. January 1;16(1):1–4. [DOI] [PubMed] [Google Scholar]
- 28.Brown W 3rd, Sheinfil A, Lopez-Rios J, Giguere R, Dolezal C, Frasca T, Lentz C, Balán IC, Rael C, Cruz Torres C, Crespo R, Febo I, Carballo-Diéguez A. Methods, system errors, and demographic differences in participant errors using daily text message-based short message service computer-assisted self-interview (SMS-CASI) to measure sexual risk behavior in a RCT of HIV self-test use. MHealth. 2019. June 18; 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Iribarren SJ, Ghazzawi A, Sheinfil AZ, Frasca T, Brown W, Lopez-Rios J, Rael CT, Balán IC, Crespo R, Dolezal C, Giguere R. Mixed-method evaluation of social media-based tools and traditional strategies to recruit high-risk and hard-to-reach populations into an HIV prevention intervention study. AIDS Behav. 2018. January 1;22(1):347–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Carballo-Diéguez A, Giguere R, Balán IC, Dolezal C, Brown W 3rd, Lopez Rios J, Sheinfil A, Frasca T, Rael CT, Lentz C, Crespo R, Cruz Torres C, Leu C-S, Febo I. Few aggressive or violent incidents are associated with the use of HIV self-tests to screen sexual partners among key populations. Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Brown W 3rd, Lopez Rios J, Sheinfil A, Frasca T, Cruz Torres C, Crespo R, Dolezal C, Giguere R, Lentz C, Balán I, Rael C, Febo I, Carballo-Diéguez A. Text messaging and disaster preparedness aids engagement, re-engagement, retention and communication among Puerto Rican participants in an HIV self-testing study after Hurricanes Irma and Maria. Disaster Medicine and Public Helath Preparedness. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Giguere R, Lopez Rios J, Frasca T, Lentz C, Balán IC, Dolezal C, Rael CT, Brown W 3rd, Sheinfil AZ, Cruz Torres C, Crespo R, Febo I, Carballo-Diéguez A. Use of HIV self-testing kits to screen clients among transgender female sex workers in New York and Puerto Rico. Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rael CT, Giguere R, Lopez Rios J, Lentz C, Balán IC, Sheinfil A, Dolezal C, Brown W 3rd, Frasca T, Cruz Torres C, Crespo R, Iribarren S, Leu C-S, Febo I, Carballo-Diéguez. Transgender women’s experiences using a home HIV-testing kit for self- and partner-testing. Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Balán IC, Lopez Rios J, Giguere R, Lentz C, Dolezal C, Cruz Torres C, Brown W 3rd, Crespo R, Sheinfil A, Rael CT, Febo I, Carballo-Diéguez A. Then we looked at his results: What happens when a sexual partner’s HIV self-test result is positive? Under review. [Google Scholar]
- 35.Sheinfil AZ, Giguere R, Dolezal C, Lopez Rios J, Iribarren S, Brown W 3rd, Rael C, Lentz C, Balán I, Frasca T, Cruz Torres C, Crespo R, Febo I, Carballo-Diégeuz A. Applying the Information-Motivation-Behavioral Skills model to predict HIV serostatus awareness among a population of high-risk men who have sex with men and transgender women. Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Lentz C, et al. Broaching the topic of HIV self-testing kit use with potential sexual partners among men who have sex with men (MSM) and transgender women (TGW) in New York and Puerto Rico. In progress. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Dolezal C, et al. Substance use and testing partners using HIV rapid home tests. In progress. [Google Scholar]
- 38.Iribarren S, et al. HIV self-test considerations: Preferences, cost considerations, issues using, and future plans for use in high-risk populations. In progress. [Google Scholar]
- 39.Febo I, Giguere R, Dolezal C, Brown W 3rd, Balán IC, Lopez Rios J, Sheinfil A, Lentz C, Crespo R, Cruz Torres C, Carballo-Diéguez A. Opportunities for HIV prevention among men who have sex with men and transgender women at high risk of infection in Puerto Rico. In progress. [Google Scholar]
- 40.The New York Times. Hurricane Maria Updates: In Puerto Rico, the Storm ‘Destroyed Us’ [newspaper on the Internet]. 2017. September 21 Available at: https://www.nytimes.com/2017/09/21/us/hurricane-maria-puerto-rico.html. Accessed 1 July 2019.
- 41.Scott M Hurricane Maria’s devastation of Puerto Rico, [newspaper on the Internet]. 1 August 2018. Available at: https://www.climate.gov/news-features/understanding-climate/hurricane-marias-devastation-puerto-rico. Accessed 1 July 2019.
- 42.Fink S Nearly a year after Hurricane Maria, Puerto Rico revises death toll to 2,975. [newspaper on the Internet]. 28 August 2018. Available at: https://www.nytimes.com/2018/08/28/us/puerto-rico-hurricane-maria-deaths.html. Accessed 1 July 2019.
- 43.The New York Times. Photos: Hurricane Maria, and Puerto Rico one year later, [newspaper on the Internet]. 22 September 2018. Available at: https://www.nytimes.com/2018/09/22/us/photos-hurricane-maria-puerto-rico.html. Accessed 1 July 2019.
- 44.Meléndez E, Hinojosa J. Estimates of Post-Hurricane Maria Exodus from Puerto Rico. Hunter College, Center for Puerto Rican Studies: Research Brief. [newsletter on the Internet]. October 2017. Available at: https://centropr.hunter.cuny.edu/sites/default/files/RB2017-01-POST-MARIA%20EXODUS_V3.pdf. Accessed 17 July 2019.
- 45.Rodríguez-Díaz CE. Maria in Puerto Rico: Natural disaster in a colonial archipelago. Am J Public Health. 2018. January;108(1):30–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Chiasson MA, Hirshfield S, Humberstone M, DiFilippi J, Koblin BA, Remien RH. Increased high risk sexual behavior after September 11 in men who have sex with men: An Internet survey. Arch Sex Behav. 2005. October;34(5):527–35. [DOI] [PubMed] [Google Scholar]
- 47.Reisner SL, Mimiaga MJ, Safren SA, Mayer KH. Stressful or traumatic life events, post-traumatic stress disorder (PTSD) symptoms and HIV sexual risk taking among men who have sex with men. AIDS Care. 2009. November;12:1481–9. [DOI] [PubMed] [Google Scholar]
- 48.Robinson WT. Impact of Hurricane Katrina on the Louisiana HIV/AIDS epidemic: A socio-ecological perspective In: Petrucci O, ed. Natural Disasters – Multifaceted Aspects in Management and Impact Assessment. INTech Open; 2013. DOI: 10.5772/55472. Available at: https://www.intechopen.com/books/natural-disasters-multifaceted-aspects-in-management-and-impact-assessment/impact-of-hurricane-katrina-on-the-louisiana-hiv-aids-epidemic-a-socio-ecological-perspective. Accessed 17 July 2019. [DOI] [Google Scholar]
- 49.Davis, et al. Influence of item characteristics.
- 50.Davis RE, Johnson TP, Lee S, Werner C. Why Do Latino Survey Respondents Acquiesce? Respondent and Interviewer Characteristics as Determinants of Cultural Patterns of Acquiescence Among Latino Survey Respondents. Cross-Cultural Research. 2019. February;53(1):87–115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Baumgartner H, Steenkamp JB. Response styles in marketing research: A cross-national investigation. Journal of marketing research. 2001. May;38(2):143–56. [Google Scholar]
- 52.HIV and AIDS in the United States of America. Avert. 2019. July Available at: https://www.avert.org/professionals/hiv-around-world/western-central-europe-north-america/usa. Accessed 17 July 2019.
- 53.Bauermeister JA, Giguere R, Carballo-Dieguez A, Ventuneac A, Eisenberg A. Perceived risks and protective strategies employed by young men who have sex with men (YMSM) when seeking online sexual partners. J Health Commun. 2010. August 30;15(6):679–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Thorburn S, Harvey SM, Ryan EA. HIV prevention heuristics and condom use among African-Americans at risk for HIV. AIDS Care. 2005. April 1;17(3):335–44. [DOI] [PubMed] [Google Scholar]
- 55.Masaro CL, Dahinten VS, Johnson J, Ogilvie G, Patrick DM. Perceptions of sexual partner safety. Sex Transm Dis. 2008. June 1;35(6):566–71. [DOI] [PubMed] [Google Scholar]