An evaluation of methods to improve the reporting of adherence in a placebo gel trial in Andhra Pradesh, India

INTRODUCTION

Vaginal microbicides are being developed as female-initiated products to prevent male-to-female transmission of HIV and other sexually transmitted infections (STIs) (1, 2). One of the key challenges in microbicide trials is obtaining accurate self-reports from participants about product use and sexual behavior. The overreporting of microbicide use and underreporting of unprotected sex have serious consequences for determining efficacy; scientists may conclude that a candidate microbicide is ineffective when in fact low adherence is at fault (3, 4).

Methodological challenges to assessing adherence have been written about extensively in the literature on patients’ drug-taking behaviors and compliance in clinical trials (5–10). Participants may feel uncomfortable admitting to study staff that they have not used the study product as directed. Additionally, requirements for clinical trial participation, specifically in relation to product and condom use, may disrupt interpersonal relationships and/or challenge existing gender norms, making strict adherence difficult (11, 12). Accurately recalling product use and/or risk behaviors may also prove impossible for many participants. Recognizing these challenges, researchers have been pursuing innovative means to improve the measurement, recording, and validation of self-reports (13–17). Such innovations, however, must be evaluated for feasibility, acceptability and cost-effectiveness before being integrated into a regulatory trial.

This paper reports findings from several methodological experiments embedded within a placebo gel trial; the placebo-only trial was designed to mirror the structure and requirements of a typical microbicide study, although of shorter duration. The purpose of the study was to explore the use of biomarkers and mobile phone technology to improve the reporting of adherence, and actual adherence, to a study product in a clinical trial. The study was conducted among female sex workers (FSWs) in Andhra Pradesh, India. Sex workers were selected as the study population because although overall HIV prevalence among the adult population in India is low (0.36%), among sex workers in the southern part of the country prevalence is estimated to be 50% or higher (18–21). Therefore, if a microbicide trial were to be conducted in India, it would likely be conducted with FSWs.

BACKGROUND

Biomarkers

Biomarkers to detect drug levels in plasma and urine have been used in HIV treatment studies to assess adherence, and are increasingly included in HIV prevention trials to validate self-reports and to provide a more objective assessment of study product use (22).

Biomarkers of unprotected sex (sex without condoms)

Prostate-specific antigen (PSA), which detects protein found in semen up to 48 hours after intercourse, has demonstrated reliability as a means to confirm unprotected sex (23–27). PSA has been subject to the most extensive testing in clinical trials (23), and has been used to investigate discrepancies in self-reported condom use. The majority of published studies comparing PSA results to self-reported condom use have been conducted with sex worker populations. The percentage of female sex workers who test positive for PSA among those who report no unprotected sex has ranged from 15% to 36% (26, 28).

Rapid stain identification of human semen (RSID), developed by Independent Forensics (Lombard, IL), is another biomarker of unprotected sex that can detect the presence of semenogelin (the coagulum formed after ejaculation) in the vagina, indicating that a woman has been exposed to ejaculate in the previous 48 hours (29). RSID for semen detection does not cross-react with saliva, other vaginal fluids, vaginal products or menstrual blood and can detect semenogelin with as little as 1 µL of semen (30). The manufacturer also indicates that the test can detect semenogelin in the ejaculate from vasectomized men. Similar to PSA, RSID can be done on-site by lab technicians and, until recently, RSID was less expensive than PSA. In a recent placebo gel trial in South Africa, RSID was used to validate the accuracy of self-reported condom use among women randomly assigned to face-to-face (FTF) interviews or audio computer-assisted self-interviewing (ACASI). Of those who reported not having sex or sex with a condom, 21% of participants in the FTF arm and 16% in the ACASI arm tested positive for semen (31).

Researchers have hypothesized that informing participants about a biomarker could improve accuracy of self-reports, and/or encourage greater adherence to the study product and consistency in condom use. A study was conducted among sex workers in Mombasa, Kenya, to determine if prior knowledge of PSA improved accuracy of self-reported condom use (32). The results showed no difference in the reports between those with prior knowledge of the biomarker and those who were unaware of the test.

Biomarkers of microbicide use

Because first-generation microbicide products were not systemically absorbed, no true biomarkers, such as drug levels in urine or plasma, were available to confirm product use (33–36). A dye stain assay (DSA), a surrogate marker of gel use, was developed by researchers at the Population Council for a Phase 3 efficacy trial of the candidate microbicide Carraguard, the only non-ARV-based efficacy trial to date with an objective marker of adherence. According to DSA results, gel was estimated to have been used in only 42% of sex acts, although participants reported gel use 96% of the time in self-reports (37). The DSA is inexpensive, fast and easy to use. Applicators are sprayed with an aqueous blue dye solution that adheres to vaginal mucus present on applicators after insertion, producing a grainy, streaky, turquoise pattern. The most recent DSA validation experiment showed predictive values of over 90% (14, 15, 36). The DSA has been proven to be effective at detecting vaginal insertion even when applicators have been stored for long periods and handled extensively (14, 15).

Recall aids

Although the use of biomarkers in HIV prevention studies is increasingly common, most studies continue to conduct behavioral interviews at clinical visits due to the logistics and costs of collecting biomarkers. For example, it is often not feasible to use biomarkers to generate real-time adherence data, and biomarkers generally cannot be used with the placebo arm. To improve accuracy of self-reported behaviors (e.g., frequency of vaginal and anal intercourse, vaginal cleansing practices, menstruation, condom use, and contraceptive use), participants may be provided with a recall aid, such as a coital diary. To eliminate some of the challenges of coital diaries, studies have introduced the use of self-reporting via mobile phones (38). However, although mobile phone usage is becoming more common in developing countries, it is still being validated as a method to improve recall and the reporting of adherence in clinical research (39–41).

Coital diaries

Coital diaries (CDs) vary in design, but typically include tick boxes and pictures, allowing participants to record multiple activities per day. A feasibility study in Tanzania concluded that CDs, combined with staff support, resulted in higher reporting of socially stigmatizing activities (42). In comparing CDs with FTF exit interviews, significant differences were found for vaginal cleansing, sex during menstruation, and sex with a casual partner. Reported frequencies for vaginal sex and anal sex were also higher in the CDs than exit interviews.

CDs are not without limitations, particularly in areas where education levels are low and there are high levels of inhibition surrounding sexual activities (43). Negative aspects of CDs include the time required to provide instruction and to analyze completed diaries (44), participant fatigue, participant’s anxiety that the diaries will be discovered by a third party (42, 45) and, like other self-reporting tools, a tendency to provide socially desirable responses (46, 47). Some researchers believe that many of these potential challenges can be alleviated with increased instruction and staff support (42), although others suggest that the frequency of missing data and the burden on respondents undermines their utility (46–48).

Interactive voice response surveys

In interactive voice response surveys (IVRS), also known as touch-tone data entry, respondents use cell phones to issue reports of behaviors. Similar to ACASI, the participant hears the question and answers using the cell phone keypad. IVRS has demonstrated strong potential for survey research (49), particularly for aiding recall (50, 51) and lowering response bias (50, 52).

METHODS

Study population

Between February and December 2010, FSWs living or working in Nellore, Andhra Pradesh, India, were enrolled in a placebo gel study implemented by YRG CARE, a non-governmental organization based in Chennai, India, at their community clinic in Nellore. Potential participants were first identified via respondent driven sampling (RDS) and a community survey administered in three non-clinical sites in or around Nellore. The survey was designed to identify factors associated with willingness to participate in clinical trials. Details about recruitment and survey results can be found elsewhere (53, 54). Women who completed the survey were invited to the YRG CARE clinic to learn more about the clinical study.

Women who had participated in the community survey were eligible to participate in the clinical study if they were between 18–45 years old, had at least one paying client in the month before screening, were HIV negative and in generally in good health, and were willing and able to give informed consent. Women were excluded if they were pregnant, planning to become pregnant in the next 6 months, or were within four weeks of their last pregnancy outcome, appeared to be under the influence of drugs or alcohol, had abnormal genital findings with epithelial disruption or a Papanicolaou (Pap) smear graded as malignant or severe inflammation that did not improve with treatment, had symptomatic genital herpes, had any other condition that the clinician believed would be jeopardized by participation, or were participating in any other clinical trial or HIV prevention study. Women were invited to return to the clinic two weeks after screening for their STI and Pap smear results and to enroll in the trial if eligible and interested. When clinically indicated, participants were treated for STIs and rescreened after completing treatment. Participants with reproductive tract infections (RTI) were simultaneously treated and enrolled. During the informed consent process, counselors administered a qualitative assessment of participants’ comprehension; participants were required to demonstrate understanding of ten key points regarding study design and trial responsibilities before they were allowed to enroll.

Participants received Rs 200 (approximately US$3.85) for the screening visit, Rs 200 for the enrollment visit, Rs 200 at each monthly visit, and Rs 400 as an incentive for completing all four monthly visits, to cover transport costs and time spent at the clinic. In addition, refreshments were provided. Women who returned for an unscheduled visit were reimbursed up to Rs 100 for transportation costs.

The protocol was approved by the Institutional Review Boards of the Population Council, New York, and YRG CARE, Chennai, India, and approved by the Health Ministry Screening Committee (HMSC) through the Indian Council of Medical Research (ICMR). All participants gave informed consent in accordance with established guidelines and ethical standards for research with human subjects.

Study design

The study entailed three experiments, described in detail below; the first experiment was conducted among all FSWs screened for the study, and the second and third experiments were implemented among the enrolled population.

Experiment 1: Advance Knowledge of RSID

The first experiment was designed to determine if prior knowledge of RSID, which can detect unprotected sex, improves accuracy of condom use reports. At screening, prior to undergoing a behavioral interview and a clinical exam, participants were randomized to know about the RSID test (“KNOW RSID”) or not to know about the RSID (“DON’T KNOW RSID”). Women in the “KNOW RSID” group were fully informed of the process and purpose of the RSID test, including that vaginal swabs collected during the clinical exam would be tested for semen, and that researchers could use the results to assess whether a participant had unprotected sex in the past two days. Those in the “KNOW RSID” group signed an additional informed consent form indicating they had been told about the RSID test. Women in the “DON’T KNOW RSID” group were told that vaginal swabs collected during the clinical exam would be subjected to further testing, but were not told explicitly about the RSID test.

To determine whether prior knowledge encouraged more accurate reports, RSID test results were compared to responses provided to two questions in the behavioral interview; (1) “in the past 48 hours, have you had vaginal sex,” and (2) “in the past 48 hours, did you use a condom every time you had vaginal sex.” Reporting was considered "accurate" if RSID test results were negative (no semenogelin detected) and a woman reported no sex or using a condom each time she had sex in the 48 hours prior to screening, or if RSID test results were positive (semenogelin detected) and a woman reported sex without a condom in the previous 48 hours. Reports were considered "inaccurate" if RSID test results were positive, yet a woman reported no unprotected sex in the previous 48 hours, or if RSID test results were negative yet a woman reported that she did have sex without using a condom in the previous 48 hours. The timeframe was based on the manufacturer’s documentation of the length of time required for semen clearance from the vagina. Recent investigations, however, suggest that semen may stay in the vagina longer than previously estimated (55).

Experiment 2: Advance Knowledge of DSA

The second experiment sought to determine if prior knowledge of the DSA, which validates insertion of vaginal applicators, leads to more accurate reports of gel use. Participants randomized to the “KNOW DSA” were told that returned opened applicators would be tested to see if they had been inserted into the vagina and that results from the DSA would be compared to participants’ self-reports of gel use. Participants randomized to the “DON’T KNOW DSA” groups were informed that returned applicators would be saved for future testing for researchers to learn more about microbicides. Those in the “DON’T KNOW DSA” groups were not told about the DSA or that DSA results would be compared to answers provided in the behavioral interview.

To assess if prior knowledge produced more accurate reports, self-reported product use was compared to results from the DSA performed on returned applicators. Adherence was defined as the ratio of inserted applicators, as indicated by the DSA (numerator), to the number of expected insertions, based on number of days in the study (denominator). Two sets of assumptions were made to calculate a minimum and a maximum measure of adherence, by month and for the whole study period. The minimum adherence calculation assumes that applicators distributed but not returned were unused, whereas the maximum adherence calculation assumes all unreturned applicators were inserted.

Experiment 3: IVRS to Improve Accuracy of Self-reported Behaviors

The third experiment was designed to assess the feasibility of IVRS for self-reporting, and to evaluate if daily IVRS calls produce more accurate reports of gel use and sexual activity compared to CDs. At enrollment, participants randomized to the IVRS groups were issued cell phones and received daily calls during a time period of their choice. Participants were asked four questions: if they inserted gel, number of sex acts, number of partners, and number of sex acts with condoms on the previous day. Those in the CD groups received a small paper diary with one page for each day of the month with the same four questions as in IVRS, and were instructed to complete it daily.

To assess whether it was feasible to include IVRS in a clinical trial among this population, the proportion complying and the ratio of logical to illogical data obtained by each reporting mode were examined and compared. The percentage complying for the IVRS and CDs was determined for each participant by calculating the number of days in which there was a successful response (either call answered or page completed), and dividing by the number of days between study visits. Data provided in either mode were determined to be illogical if: (1) the number of partners reported exceeded the number of sex acts or (2) the number of condoms reported exceeded the number of sex acts. The amount of illogical data for each mode was determined by the total number of illogical responses received for all participants divided by the total number of responses received through the course of the trial (calls answered or pages filled).

To determine whether IVRS improves the accuracy of self-reports compared to CDs, the analysis plan, which was dependent on high compliance to the assigned recall aids, called for self-reported gel adherence and condom use to be compared and validated using the DSA and RSID test results.

Can DSA and IVRS increase actual adherence to gel use?

While not a primary objective of this study, data obtained allowed for a comparison of actual gel use for each randomization group (KNOW/DON’T KNOW and IVRS/CD), as validated by the DSA. To determine if prior knowledge of the DSA increased actual gel use, and to assess whether receiving a daily call via IVRS improves actual adherence, average monthly gel use based on the DSA was calculated for the experimental groups and compared.

Procedures

The trial consisted of six clinic visits: screening, enrollment, and four monthly follow-up (FU) visits. At screening, all women were randomized in a 1:1 ratio to one of two groups for the Advance Knowledge of RSID experiment (Experiment 1): “KNOW RSID” or “DON’T KNOW RSID.” At enrollment, all participants were randomized simultaneously to Experiment 2 (Advance Knowledge of DSA) and Experiment 3 (IVRS to Improve Accuracy of Self-reported Behaviors), as follows:

• Group 1: KNOW DSA; CD

• Group 2: KNOW DSA; IVRS

• Group 3: DON’T KNOW DSA; CD

• Group 4: DON’T KNOW DSA; IVRS

At screening and Month 4/Termination visits, participants underwent a physical and pelvic exam, and testing for STIs, including HIV, HSV-2, trichomoniasis, syphilis, gonorrhea, and chlamydia. RTIs (bacterial vaginosis and yeasts) were tested only if symptomatic. Urine pregnancy tests were done at each visit and swabs were collected for the RSID test at screening and at each follow-up visit. Additional details of the exams and testing are described elsewhere (56).

After enrollment and randomization all participants received a monthly supply of applicators filled with hydroxyethylcellulose (HEC) gel and were given a phone or diary, based on randomization group. HEC was selected as the placebo gel for this trial based on its safety profile and lack of in vivo protection against HSV-2 and HIV (57). HEC lacks spermicidal activity, and has been found to be safe and acceptable for vaginal use in a 14-day study (58) and has been used by thousands of women in multiple Phase 2 and 3 efficacy trials (59–64). HEC was produced by Clean Chemical Sweden (CCS, Borlänge, Sweden) and packaged in boxes of 32 pre-filled, single-dose Microlax^®-type applicators. Each applicator was filled with 7 mL of gel, designed to deliver a dose of approximately 4 mL. Participants were given one box of applicators at enrollment and instructed to insert gel once a day, every day, at approximately the same time. Women were given a supply of plastic bags for collecting the used applicators, and cotton swabs, which they were instructed to insert into the tips of the used applicators to prevent leakage during storage.

At enrollment and at each monthly visit, women were counseled to use condoms with each sex act because HEC does not protect against HIV or STIs, nor is it a contraceptive. At each visit, women were instructed to return all applicators dispensed at the previous visit. At all visits (except Month 4/Termination), women were provided with new applicators, cotton swabs, bags, and condoms. Monthly clinic visits included a behavioral interview via ACASI with questions about frequency of sexual acts (oral, vaginal, and anal sex), number of paying and non-paying sexual partners, condom use, drug and alcohol use, sexual and physical violence, product use in the past week, and vaginal practices, such as the insertion of herbs, citrus juices, or lubricants into the vagina. Participants in the CD arm were asked how often they filled in their daily diaries at each study visit. At the final study visit, participants in the IVRS arm were asked to provide reasons for not answering the daily calls. Options included both user errors (e.g., not having the phone with her) and technical errors (e.g., not being able to hear the survey). Additionally, at exit, all participants were asked two questions with the option to agree or disagree with the statements, (1) “researchers could tell if the gel has been inserted into my vagina,” and (2) “researchers could tell if I used a condom the last time I had sex.” DSA and RSID testing were done after participants left the clinic.

Data management

All clinical and laboratory data were entered into paper case report forms (CRFs) at the study site and then sent via DataFax (Data Management Software for Clinical Trials V3.7, Clinical DataFax Systems, Inc., Ontario, Canada) to the Population Council in New York. Screening and monthly behavioral interview responses were captured using a combination of face-to-face interviews (DataFax) and ACASI, both sent daily to the data manager in New York. ACASI data was captured immediately via a secure Access database (Microsoft Office 2007, Seattle, WA). CD data was manually entered into Microsoft Excel (Microsoft Office 2007, Seattle, WA); IVRS calls were administered by automated telephone survey software, which also captured data (TeleSage SmartQ v.5.2, Chapel Hill, NC).

Statistical analyses

To investigate variability in reporting of sexual behavior and adherence, as well as actual adherence to gel use by randomization group separately by visit month and, overall, for the four-month study, two-sample tests were conducted to detect differences in proportions, and t-tests or one-way analysis of variance (ANOVA) were used to compare means. Separate comparisons were made for minimum and maximum adherence. Analyses were conducted using SAS Version 9.2 (SAS Institute, Cary, NC) and Stata Version 11.2 (StataCorp, College Station, TX).

RESULTS

As shown in Figure 1, 551 women were screened for the trial and randomized to the Advance Knowledge of RSID experiment. Of those screened, 33.6% were ineligible for the trial (primarily due to reproductive morbidities), 18% chose not to enroll, and 48.5% were enrolled. At enrollment, participants (n=267) were randomized to four groups. Between 76.1% and 85.1% of participants in each group completed a Month 4/Termination visit, with approximately 50% completing all four study visits. There were 29 adverse events (AEs) reported during the trial, none of which were determined to be related to the HEC gel or participation in the study. The majority of AEs were STIs, and the remaining were related to generalized illnesses (e.g., body aches).

Figure 1.

Participant Flow (CASCADE)

Table 1 and Table 2 provide the background characteristics of the screened and enrolled cohorts, respectively. Mean age was 31.0 years. Most participants were Hindu (approximately 70%), more than two-thirds were currently married, with an average of two children, and the majority (approximately 85%) was using some form of contraception. Nearly two-thirds of women (63%) had not attended any school and almost 80% could not read at all. Approximately 64% of the participants had prior experience with a cell phone (either owning or having used one). A large proportion engaged in high-risk behaviors; approximately 70% reported anal sex with a paying client in the last month, one-third reported ever having used illegal drugs, and half reported that they had ever used alcohol. Approximately half of the participants reported that they had been physically assaulted and more than half reported having been sexually assaulted in the month prior to screening. Approximately 60% of FSWs reported that they always used condoms with clients, and 55% reported always using condoms with non-paying clients. Among those screened for the trial, 20% tested positive for non-viral STIs, 60.7% for HSV-2, and 5.3% for HIV (56).

Table 1.

Background demographics of female sex workers screened into a placebo gel trial by randomization group to the Advance Knowledge of RSID experiment, Andhra Pradesh, India

Characteristic	SCREENED POPULATION
	RSID “know” (n = 276)	RSID “don’t know” (n = 275)	Total (n = 551)	Test statistic comparing know / don’t know
Mean age (range)	30.8 (18–45)	31.0 (18–45)	30.9 (18–45)	t=−0.34; p=0.73
Highest level education completed (n, % per category)
None	170 (63.2)	167 (62.1)	337 (62.6)	z=0.27; p=0.79
Classes 1–5	61 (22.7)	60 (22.3)	121 (22.5)	z=0.10; p=0.92
Class 6 or above	38 (14.1)	42 (15.7)	80 (14.9)	z=−0.48; p=0.63
Currently married [n, (%)]	186 (69.1)	183 (68.0)	369 (68.6)	z=0.28; p=0.78
Religion [n, (%)]
Hindu	188 (69.9)	198 (73.6)	386 (71.8)	z=−0.96; p=0.34
Muslim	37 (13.8)	26 ( 9.7)	63 (11.7)	z=1.47; p=0.14
Christian	44 (16.4)	45 (16.7)	89 (16.5)	z=−0.12; p=0.91
Mean number live births (range)	2.1 ( 0–8)	2.1 ( 0–5)	2.1 ( 0–8)	t=0.41; p=0.68
Could not read [n, (%)]	207 (78.7)	202 (76.5)	409 (77.6)	z=−0.60; p=0.55
Prior experience with cell phone1 [n, (%)]	176 (63.8)	174 (63.3)	350 (63.5)	z=−0.12; p=0.90
Mean number of partners, past month (range)	3.0 (2–24)	3.0 (1–20)	3.0 (1–24)	t=−0.09; p=0.93
Contraceptive method2 [n, (%)]
None	36 (13.7)	37 (13.9)	73 (13.8)	z=0.07; p=0.94
Surgical	194 (73.8)	185 (69.6)	379 (71.6)	z=1.08; p=0.28
Temporary	127 (48.3)	133 (50.0)	260 (49.1)	z=−0.39; p=0.69
Mean coital frequency with paying partners, past month (range)	5.2 (1–43)	5.0 (1–32)	5.1 (1–43)	t=0.42; p=0.67
Frequency of condom use with paying clients [n, (%)]
Always (100%)	145 (56.0)	157 (59.5)	302 (57.7)	z=−0.81; p=0.42
Sometimes (50%)	49 (18.9)	55 (20.8)	104 (19.9)	z=−0.55; p=0.58
Rarely (less than 25%)	29 (11.2)	17 ( 6.4)	46 ( 8.8)	z=1.92; p=0.05
Never (0%)	36 (13.9)	35 (13.3)	71 (13.6)	z=0.21; p=0.83
Frequency of condom use with non-paying partners [n, (%)]
Always (100%)	151 (57.9)	140 (52.6)	291 (55.2)	z=1.21; p=0.23
Sometimes (50%)	44 (16.9)	63 (23.7)	107 (20.3)	z=−1.95; p=0.05
Rarely (less than 25%)	25 ( 9.6)	24 ( 9.0)	49 ( 9.3)	z=0.22; p=0.83
Never (0%)	41 (15.7)	39 (14.7)	80 (15.2)	z=0.33; p=0.74
Had anal sex with client in past month [n, (%)]	141 (54.2)	153 (58.2)	294 (56.2)	z=−0.91; p=0.36
Used condom last AI [n, (%)]	142 (54.6)	161 (60.3)	303(57.5)	z=−1.32; p=0.19
Ever used alcohol [n, (%)]	117 (48.6)	115 (46.4)	232 (47.4)	z=0.48; p=0.63
Ever used illegal drugs [n, (%)]	84 (31.1)	86 (31.7)	170 (31.4)	z=−0.16; p=0.88
Ever forced to have sex, past month [n, (%)]	141 (54.2)	142 (53.2)	283 (53.7)	z=0.24; p=0.81
Ever experienced physical violence, past month [n, (%)]	124 (47.3)	135 (50.6)	259 (49.0)	z=−0.74; p=0.46
Vaginal insertion in past 7 days [n, (%)]	207 (79.0)	226 (84.6)	433 (81.9)	z=−1.68; p=0.09
STI detected at screening [n, (%)]	41 (15.6)	55 (20.7)	96 (18.2)	z=−1.52; p=0.13
HSV-2 detected at screening [n, (%)]	152 (57.8)	169 (63.5)	321 (60.7)	z=−1.35; p=0.18
HIV detected at screening [n, (%)]	12 ( 4.6)	16 ( 6.0)	28 ( 5.3)	z=−0.75; p=0.46

¹Includes women who reported previously owning or using a cell phone.

²More than one choice possible.

Table 2.

Background demographics of female sex workers enrolled in a placebo gel trial by randomization group for the Advance Knowledge of DSA and the IVRS to Improve Accuracy of Self-reported Behaviors experiments, Andhra Pradesh, India

Characteristic	ENROLLED POPULATION
	DSA “know” (n = 133)	DSA “don’t know” (n =134)	Test statistic	IVRS (n = 133)	CD (n = 134)	Test statistic	Total enrolled (n = 267)
Mean age (range)	31.5 (19–44)	30.4 (18–45)	t=1.38; p=0.17	31.1 (18–44)	30.8 (19–45)	t=0.39; p=0.70	31.0 (18–45)
Highest level education completed (n, % per category)
None	84 (63.1)	84 (62.7)	z=0.08; p=0.94	78 (58.6)	90 (67.2)	z=−1.44; p=0.15	168 (62.9)
Classes 1–5	30 (22.6)	27 (20.2)	z=0.48; p=0.63	33 (24.8)	24 (17.9)	z=1.38; p=0.17	57 (21.4)
Classes 6 or above	19 (14.3)	22 (17.2)	z=−0.65; p=0.52	22 (16.5)	20 (14.9)	z=0.36; p=0.72	42 (15.7)
Currently married [n, (%)]	83 (62.4)	102 (76.1)	z=−2.43; p=0.02	90 (67.7)	95 (70.9)	z=−0.57; p=0.57	185 (69.3)
Religion [n, (%)]
Hindu	96 (72.2)	91 (67.9)	z=0.76; p=0.45	96 (72.2)	91 (67.9)	z=0.76; p=0.45	187 (70.0)
Muslim	17 (12.8)	17 (12.7)	z=0.02; p=0.98	17 (12.8)	17 (12.7)	z=0.02; p=0.98	34 (12.7)
Christian	20 (15.0)	26 (19.4)	z=−0.94; p=0.34	20 (15.0)	26 (19.4)	z=−0.94; p=0.34	46 (17.2)
Mean number live births (range)	2.0 (0–8)	2.2 (0–5)	t=−1.19; p=0.23	2.1 (0–5)	2.1 (0–5)	t=−0.21; p=0.83	2.1 ( 0–8)
Could not read [n, (%)]	103 (79.2)	100 (76.2)	z=0.45; p=0.65	98 (75.6)	105 (80.2)	z=−0.82; p=0.41	203 (78.1)
Prior experience with cell phone1 [n, (%)]	82 (61.7)	89 (66.4)	z=−0.81; p=0.42	89 (66.9)	82 (61.2)	z=0.97; p=0.33	171 (64.0)
Mean number of partners, past month (range)	2.8 (2–24)	2.8 (2–12)	t=0.43; p=0.67	2.8 (2–24)	2.8 (2–12)	t=0.15; p=0.88	2.8 (2–24)
Contraceptive method2 [n, (%)]
None	19 (14.3)	21 (15.7)	z=0.32; p=0.75	20 (15.1)	20 (14.9)	z=−0.03; p=0.98	40 (15.0)
Surgical	96 (72.2)	93 (69.4)	z=0.50; p=0.62	94 (70.7)	95 (70.9)	z=−0.04; p=0.97	189 (70.8)
Temporary	71 (53.4)	67 (50.0)	z=0.55; p=0.58	69 (51.9)	69 (51.5)	z=0.06; p=0.95	138 (51.69
Mean coital frequency with paying partners, past month (range)	5.2 (1–24)	4.8 (1–19)	t=1.09; p=0.28	5.4 (1–19)	4.6 (1–24)	t=1.81; p=0.07	5.0 (1–24)
Frequency of condom use with paying clients [n, (%)]
Always (100%)	82 (62.6)	76 (57.6)	z=0.83; p=0.41	79 (60.3)	79 (59.8)	z=0.08; p=0.94	158 (60.1)
Sometimes (50%)	20 (15.3)	31 (23.5)	z=−1.69; p=0.09	19 (19.8)	25 (18.9)	z=0.19; p=0.85	51 (19.4)
Rarely (less than 25%)	12 ( 9.2)	12 ( 9.1)	z=0.02; p=0.98	12 ( 9.2)	12 ( 9.1)	z=0.02; p=0.98	24 ( 9.1)
Never (0%)	17 (13.0)	13 (9.8)	z=0.80; p=0.42	14 (10.7)	16 (12.1)	z=−0.37; p=0.71	30 (11.4)
Frequency of condom use with non-paying partners [n, (%)]
Always (100%)	74 (56.1)	73 (54.9)	z=0.19; p=0.85	67 (50.8)	80 (60.2)	z=−1.54; p=0.12	147 (55.5)
Sometimes (50%)	25 (18.9)	32 (24.1)	z=−1.01; p=0.31	32 (24.2)	25 (18.8)	z=1.08; p=0.28	57 (21.5)
Rarely (less than 25%)	16 (12.1)	9 ( 6.8)	z=1.49; p=0.14	14 (10.6)	11 ( 8.3)	z=0.65; p=0.52	25 ( 9.4)
Never (0%)	17 (12.9)	19 (14.3)	z=−0.33; p=0.74	19 (14.4)	17 (12.8)	z=0.38; p=0.70	36 (13.6)
Had anal sex with client in past month [n, (%)]	79 (60.3)	74 (56.1)	z=0.70; p=0.49	77 (58.8)	76 (57.6)	z=0.20; p=0.84	153 (58.2)
Used condom last AI [n, (%)]	85 (64.4)	74 (56.1)	z=1.38; p=0.17	75 (57.3)	84 (63.2)	z=−0.98; p=0.34	159 (60.2)
Ever used alcohol [n, (%)]	54 (44.6)	65 (56.0)	z=−1.76; p=0.08	53 (45.3)	66 (55.0)	z=−1.49; p=0.14	119 (50.2)
Ever used illegal drugs [n, (%)]	49 (37.4)	38 (28.6)	z=1.53; p=0.13	53 (45.3)	66 (55.0)	z=−0.57; p=0.57	87 (33.0)
Ever forced to have sex, past month [n, (%)]	75 (56.8)	70 (53.0)	z=0.62; p=0.54	41 (31.3)	46 (34.6)	z=−0.87; p=0.39	145 (54.9)
Ever experienced physical violence, past month [n, (%)]	70 (53.0)	68 (51.1)	z=0.31; p=0.76	69 (52.3)	76 (57.6)	z=−0.92; p=0.36	138 (52.1)
Vaginal insertion in past 7 days [n, (%)]	107 (81.1)	116 (87.2)	z=−1.37; p=0.17	65 (49.2)	73 (54.9)	z=−1.04; p=0.30	223 (84.2)
STI detected at screening [n, (%)]	28 (21.1)	26 (19.4)	z=0.34; p=0.74	108 (81.8)	115 (86.5)	z=−1.49; p=0.14	54 (20.2)
HSV-2 detected at screening [n, (%)]	80 (60.2)	76 (56.7)	z=0.57; p=0.57	22 (16.5)	32 (23.9)	z=1.07; p=0.29	156 (58.4)
HIV detected at screening [n, (%)]	0 ( 0.0)	0 ( 0.0)	--	82 (61.7)	74 (55.2)	--	0 ( 0.0)

¹Includes women who reported previously owning or using a cell phone.

²More than one choice possible.

Results of Experiment 1: Advance Knowledge of RSID

The RSID experiment conducted at screening indicated that prior knowledge of the biomarker did not improve the accuracy of self-reported condom use. As shown in Table 3, 78.3% of women in the “KNOW RSID” group (n=253) and 82.9% in the “DON’T KNOW RSID” group (n=257) reported condom use that was consistent with the RSID test results, with no difference between groups (p=0.19). Overall, only 10.6% of women tested positive for RSID, indicating a high level of condom use in the 48 hours prior to screening. However, of those testing positive for RSID, 75.9% (41/54) reported no unprotected sex.

Table 3.

Accuracy of self-reported condom use within the previous 48 hours, as validated by rapid stain identification of human semen (RSID), among female sex workers randomized to the Advance Knowledge of RSID experiment, Andhra Pradesh, India

	Know about RSID (n=253)	Don’t know about RSID (n=257)	Total* (n=510)
Accurate reporters	198 (78.3)	213 (82.9)	411 (80.6)^
No sex/RSID−	58 (22.9)	51 (19.8)	109 (21.4)
Sex with condom/RSID−	134 (53.0)	155 (60.3)	289 (56.7)
Sex without condom/RSID+	6 ( 2.4)	7 ( 2.7)	13 ( 2.6)
Inaccurate reporters	55 (21.7)	44 (17.1)	99 (19.4)
No sex/RSID+	4 ( 1.6)	8 (3.1)	12 ( 2.4)
Sex with condom/RSID+	19 ( 7.5)	10 (3.9)	29 ( 5.7)
Sex without condom/RSID−	32 (12.7)	26 (10.1)	58 (11.4)
TOTAL	253 (49.6)	257 (50.4)	510 (100.0)

^*Missing data from 41 women at screening.

^{^}Two-sample test of differences in proportion, z=−1.32; p = 0.19.

Results of Experiment 2: Advance Knowledge of DSA

Of the 26,464 applicators dispensed over the course of the trial, 20,802 (78.6%) were returned. As shown in Table 4, the Advance Knowledge of DSA experiment conducted among the enrolled population indicated that prior knowledge of the DSA test did not improve the accuracy of self-reported gel use. Using the calculated minimum adherence assumption, 30.2% of applicators in the “KNOW DSA” group (n=129) and 30.1% in the “DON’T KNOW DSA” group (n=130) tested positive for vaginal insertion, with no difference between groups (p=0.98). Similar results are found when using the maximum adherence estimate: 43.1% of applicators from the “KNOW DSA” group and 43.6% in the “DON’T KNOW DSA” group tested positive for vaginal insertion, with no difference between groups (p=0.89).

Table 4.

Adherence to daily gel use by women enrolled in a placebo gel trial by randomization group as validated by the dye stain assay (DSA), by month, Andhra Pradesh, India

	Results by Advance Knowledge of DSA experiment				Results by IVRS to Improve Accuracy of Self-reported Behaviors experiment
	Know (n=129)	Don’t know (n=130)	Total (n=259)*	Test statistic comparing know/ don’t know^	CD (n=134)	IVRS (n=133)	Total (n=267)	Test statistic comparing CD/IVRS^
Minimum adherence
Month 1	38.8%	37.0%	37.9%		33.3%	40.1%	36.9%
Month 2	40.8%	29.1%	34.7%		33.1%	34.2%	33.6%
Month 3	23.3%	31.0%	26.7%		24.2%	28.2%	26.2%
Month 4†	22.7%	24.2%	23.4%		20.5%	25.9%	22.9%
Overall	30.2%	30.1%	30.1%	t=0.03; p=0.98	27.0%	31.9%	29.4%	t=−1.43; p=0.16
Maximum adherence
Month 1	52.0%	53.1%	52.5%		50.9%	56.1%	53.7%
Month 2	51.2%	43.6%	47.2%		47.5%	49.4%	48.5%
Month 3	37.0%	43.8%	40.1%		40.4%	42.0%	41.2%
Month 4†	36.7%	36.6%	36.6%		34.5%	42.2%	37.9%
Overall	43.1%	43.6%	43.3%	t=−0.15; p=0.89	42.1%	47.1%	44.6%	t=−1.47; p=0.14

^*8 participants are omitted from the Know/Don’t know comparison due to protocol deviations with respect to randomization assignment.

^{^}Two-sample t test for the equality of weighted means.

^†A linear regression of adherence on months for each experiment group shows that the observed decline in minimum and maximum. adherence over time is statistically significant. Seven of the eight regressions are jointly significant at the p<0.01 level, and the eighth is significant at the p<0.10 level.

Results of Experiment 3: IVRS to Improve Accuracy of Self-reported Behaviors

Compliance to IVRS was low throughout the trial: overall only 16% of daily IVRS calls were answered (data not shown). The majority of calls were either unanswered or participants hung up before responding to the survey questions. Of responses provided via IVRS, 13% were determined to be illogical. The CD compliance rate was higher than the IVRS rate, but still lower than expected: 71% of all diary pages expected to be completed were actually filled in. The CDs had a lower rate of illogical data (5%) than what was observed for the IVRS group. Although not statistically significant, compliance to reporting via both IVRS and CDs was higher for literate women than illiterate women (data not shown). Literate women answered 18% of IVRS calls while illiterate women answered 15% (p=0.32); literate women completed 79% of diary pages while illiterate women completed 69% (p=0.13).

To better understand challenges to compliance with IVRS, in their final behavioral interview IVRS participants were asked 10 yes/no questions to elicit reasons why they did not answer the daily phone survey. Participants could answer yes to multiple options. The three most commonly selected reasons were being without the phone (96%), the phone was not charged (76%), and the participant was unable to hear the questions (75%).

Low compliance in the IVRS arm made it impossible to compare accuracy in self-reporting between the two modes (CDs and IVRS). Very few participants answered the IVRS call in the two days prior to their clinic visit (at which RSID is conducted), and therefore RSID test results could not be compared to self-reported condom use. Similarly, irregular and infrequent compliance to IVRS prohibited a valid comparison of DSA results to self-reported gel use.

Can the DSA and IVRS increase actual adherence to gel use?

There was no significant difference overall between the “KNOW DSA” and “DON’T KNOW DSA” groups for either minimum (30.2% versus 30.1%, p=0.88) or maximum (43.1% versus 43.6%, p=0.45) adherence, suggesting that prior knowledge of the biomarker does not encourage higher gel use even with repeated counseling. Similarly, the mode used for daily self-reports (CD or IVRS) appears to have no effect on improved adherence to study product. As shown in Table 4, participants randomized to the IVRS group inserted slightly more applicators of gel than those in the CD group, although the difference was not statistically significant.

Self-reported adherence

Table 5 presents the average gel use reported at monthly behavioral interviews, which indicated no significant differences in self-reports by randomization group. However, there were large discrepancies between self-reported adherence in the behavioral interviews (85%, on average) and actual adherence (approximately 40–43%, on average), as measured by the DSA (as shown in Table 3). Data from monthly behavioral interviews via ACASI indicated that at least half of participants did not insert gel because they were concerned about side effects, felt sick, or had a change in routine or were too busy. Additionally, despite instructions to use the gel daily, at least 50% of the participants reported not using the gel because they were not having sex or because they were menstruating (data not shown).

TABLE 5.

Average self-reported gel use reported monthly for the 7 days prior to each visit among female sex workers enrolled in a placebo gel trial randomized to the Advance Knowledge of DSA experiment, Andhra Pradesh, India

	Results by Advance Knowledge of DSA experiment
Self-reported gel use	Know (n=129)	Don’t know (n=130)	Total (n=259)
Month 1	88.7%	88.8%	88.8%
Month 2	90.6%	86.3%	88.4%
Month 3	91.6%	86.9%	89.3%
Month 4	88.8%	90.3%	89.6%
Overall	89.9%	88.1%	89.0%^

^{^}Two-sample t test for the equality of weighted means, t=0.74; p=0.46.

DISCUSSION

Given limited funding for biomedical prevention studies and disappointing trial results, it is imperative to develop and assess mechanisms for improving adherence and the reporting of adherence to maximize available resources for microbicide development. Fully assessing the feasibility and utility of methods such as IVRS and biomarkers require conducting methodological experiments, such as those described in this paper, outside of a regulatory trial. Findings from this study provide several lessons for future microbicide research.

Adherence to gel use was similar to other microbicide studies where the DSA was used. In the Carraguard Phase 3 trial conducted in South Africa with a coitally dependent dosing strategy, adherence was estimated to be 42.1% (37) according to the DSA. In a placebo gel trial conducted in the same sites, also using a coitally dependent regimen, overall average adherence was 44%, with adherence declining over time (31). In this study, using the most liberal assumption that unreturned applicators had been used, the average use was 53% at Month 1, with adherence rates steadily declining throughout the study to an average of 36.6% by Month 4. The behavioral interviews suggest that changes in routine, concerns about side effects, menses, and poor health discouraged daily use. Additionally, participants, and/or their sexual partners, might have found characteristics of the HEC gel undesirable. Perhaps knowing that HEC offered no protective effect, participants were unmotivated to use it, although placebo-only studies conducted elsewhere have not observed lower adherence (31, 63, 64).

The behavioral interview data suggests that participants were still highly motivated to appear adherent to gel use, even when responding to questions via ACASI and not directly to the clinic staff. Women reported using gel on 80% of days, on average, compared to gel use in the 20–40% range according to the DSA. Cultural norms, desire for healthcare, and fear of being withdrawn from the study could have led to inflated reports of product use, even when a biomarker and ACASI were in place. These findings are similar to those noted in a non-sex worker population in South Africa (31), and are suggestive of the degree to which social desirability influences adherence reporting. These findings strengthen the argument for the inclusion of biomarkers to assess actual product use, as self-reports of product adherence are likely to be unreliable.

Although reports of gel use were inflated, overall there was a high level of correlation between self-reported condom use and results of the RSID test. Condom use was high in this cohort, and participant reports typically matched biomarker results. However, among the 11% of participants who tested positive for RSID, approximately three-quarters reported no unprotected sex. This rate of discrepancy is similar to what has been reported in previous studies comparing PSA to self-reported condom use in samples of FSWs (22, 26, 28, 65), although somewhat higher than rates reported in non-sex worker populations (66, 67). Recent investigations have suggested that the time for complete semen clearance may be longer than 48 hours (55). For participants who use condoms inconsistently, this could result in positive RSID tests despite accurately reporting that they had not engaged in unprotected sex in the previous 48 hours. However, very few participants in our study (2.6%) were in this category. Therefore, supported by RSID results, we are inclined to conclude that condom use is high in this specific population, and that self-reported condom use in this study was reasonably accurate. Still, while overall rates of condom use were high, over 20% of participants reported never or rarely using condoms with paying or non-paying partners, suggesting the continued need for targeted condom counseling as part of HIV prevention outreach to FSWs.

Prior knowledge of the RSID test failed to decrease inaccurate reports of condom use in behavioral interviews administered at screening or during the trial. Similarly, prior knowledge of the DSA failed to produce more accurate accounts in monthly behavioral reports of gel use. Although counselors were trained to explain the procedures in easy to understand language, and to gauge comprehension and provide further explanations, if necessary, it is possible that the technical aspect of the RSID or DSA proved to be too confusing for participants. Based on findings from their study, Thomsen et al. (68) have similarly argued that detailed information provided was too complicated for participants to comprehend. Although these results were disappointing, further investigation of the degree to which understanding of the purpose of biomarkers improves the accuracy of self reports, both in populations with higher literacy and among people with greater familiarity with mobile phone technology such as SMS, is merited. In future explorations, it is also advisable that participants are shown an illustration of the tests, if not the test itself; furthermore, ongoing counseling sessions should be supplemented by such visual aids. A full assessment of comprehension should be administered at study initiation and at exit to confirm that the KNOW groups fully understand the biomarkers.

In this study, compliance to completing CDs was significantly higher than responses to the IVRS daily calls, in contrast to previous publications comparing these modes (69, 70). It is unclear, however, when the CDs were completed; nearly half the participants (59%) reported that they did not fill in their CDs daily, as instructed, suggesting the likelihood of “back filling” or “forward filling.” Given the high rates of illiteracy in this population, it is possible that FSWs in our study were unaccustomed to keeping records in their daily lives (e.g., finances or debt), which could have made the CD alien to them, despite the use of graphics and focused counseling.

A study exploring the use of IVRS and SMS for assessing adherence to antiretroviral therapy (ART) was undertaken in Uganda during the same time period, and revealed similar challenges in compliance (38). Only 12.5% of participants in the Haberer study completed calls that resulted in a successfully administered response. We share the researchers’ recommendations that participants should be provided with extensive training, including active practice and confirmation, as well as increased motivation to respond to the calls, particularly as initial enthusiasm may wane over time (38). Furthermore, it is possible that FSWs in this study may have found the daily calls too burdensome to answer. A cross-over study comparing diaries and IVRS conducted by Weiler et al. (48) found that respondents preferred paper dairies, reporting that daily calls were burdensome and took longer to complete than diary entries. There was also more missing data in the IVRS group in the Weiler study (48), similar to our findings. Overall, these studies suggest the importance of further field testing of new technologies in resource-poor areas.

This study had several limitations. First, based on previous research indicating that FTF interviews result in inaccurate behavioral reports (31, 71, 72), ACASI was used throughout the study for the reporting of sensitive behaviors. However, in behavioral data collected in the community survey and during the monthly clinical visits, between 3–12% of women, depending on the study instrument, consistently selected the first option for each question. Although analyses were run with the full population and with these respondents excluded, and no differences were found in any outcomes, it is problematic that a significant proportion of the sample completed the hour-long ACASI interview with illogical answers. Participants may have been trying to speed through the interview, or may not have understood what was being asked of them, indicating the need for pre-testing and ongoing counseling when using ACASI.

Second, the CD did not provide participants with a way to report that they had not engaged in a particular behavior (e.g., inserted gel, had sex, used a condom) on a specific day. Therefore, it was assumed that any diary pages that were completely blank meant that the participant had been non-compliant to completing the entry for that day; however, it could have meant that she actually reported accurately (i.e., she had nothing to report for that day).

Third, because only 16% of all IVRS calls were answered, it was not possible to conduct all of the planned analyses, including a comparison between self-reported behaviors in monthly behavioral interviews with self-reported behaviors via daily IVRS and CD for the four questions: number of sex partners, number of sex acts, gel use, and condom use. In addition, results of RSID testing, responses on the monthly behavioral interviews, and responses on the daily reporting modes (IVRS vs. CD) were to be triangulated, however, due to limited IVRS data, this was impossible.

Finally, limitations to the DSA have been detailed elsewhere (36, 73, 74). It should be noted that the evaluation of sprayed applicators relies on visual inspection, thus training and practice is needed to reduce subjectivity and observer variability. Additionally, the DSA cannot provide information about whether the product was actually expelled into the vagina, the timing of gel use, nor the amount of product used (36).

These limitations encourage us to offer several recommendations for clinical studies. Software used for both ACASI and IVRS should be programmed with consistency checks and mechanisms to prevent participants from being able to exit questions early based on responses. For this study, we did not contact participants who were noncompliant with IVRS as that would invalidate the feasibility experiment, but the real-time data capture of IVRS allows for identification of noncompliant participants, who could then be provided with additional counseling. Studies using IVRS (or SMS) for self-reporting should be attentive to response patterns, and devise plans to ensure or improve compliance. Inaccuracies in self-reporting of product use and condom use underscore the importance of the inclusion of biomarkers in clinical studies. Finally, data obtained from feasibility studies can be strengthened with additional qualitative assessments to fully ascertain participants’ perspectives and understanding. In our study, in-depth interviews might have provided us with information to assess whether the participants in the “KNOW RSID” and “KNOW DSA” groups actually understood the purpose of the biomarkers. We recommend multi-method studies whenever possible.

Although many of the current microbicides undergoing testing contain antiretrovirals (ARVs), which are systemically absorbed, such biomarkers are expensive and logistically difficult to use on an ongoing basis. However, these results suggest that continued work integrating biomarkers in clinical trials is warranted, as self-reports of product use have been shown across a number of studies to be unreliable. While this research did not demonstrate that prior knowledge of a biomarker improves accuracy, this is only the second published study to investigate this, and both were conducted with sex worker populations. Additional explorations with other populations are needed. Further investigations of reporting modes such as SMS and IVRS are also warranted to determine if regular interaction with participants via mobile phones can improve adherence.