Limitations of the Dye-Based Method for Determining Vaginal Applicator Use in Microbicide Trials

Abstract

Background

A dye-based method for determining applicator usage in microbicide trials has been developed to assess whether applicators have been exposed to vaginal fluid. Our objective was to evaluate this method on polypropylene HTI applicators which are being widely used in several effectiveness trials of microbicides.

Methods

Study participants enrolled in a clinical trial assessing SPL7013 (VivaGel) inserted gel intravaginally twice daily for 14 days and returned used and unused applicators. Prior to staining, smears were prepared from each participant-inserted applicator, Gram stained and assessed independently for the presence of vaginal cells and bacteria. Of the 169 participant-inserted applicators, 168 (99%) had vaginal cells identified by Gram stain. Participant-inserted applicators were stained with a 0.05% FD&C Blue Dye No. 1 solution and compared to 70 inserted positive control applicators and 70 unused negative control applicators. Intravaginally inserted applicators should stain turquoise, while unused applicators should not retain any stain. The individual responsible for labeling and preparing the applicators did not serve as an evaluator.

Results

Under optimized conditions, the sensitivity and specificity ranged from 81–95% and 86–93%, respectively for single use and unused applicators. The dye-based method was only 47–77% sensitive for participant-inserted applicators obtained from women using gel twice daily.

Conclusion

The dye test for HTI polypropylene applicators had a sensitivity of 47–95% depending on the evaluator and whether gel was present in the vagina. The sensitivity was decreased with multiple gel applications. The dye-based method cannot be recommended for HTI polypropylene applicators to monitor product adherence.

Introduction

More than 25 years following its discovery, human immunodeficiency virus (HIV) continues to infect millions of individuals each year and of these individuals, women comprise approximately half of all newly acquired infections worldwide ¹. For sexually active individuals, condom use is the only effective method available for the prevention of HIV and other sexually transmitted infections (STIs). However, social and cultural barriers and economic inequality can limit the negotiation of condom use by women ². While microbicides are being developed as vaginal rings, fast-dissolving films and gels, most of the products currently under evaluation in clinical trials are water-based gels. Microbicide gels can be delivered vaginally or rectally via a disposable applicator and offer an alternative method of protection against acquisition of HIV and STIs. Adherence to consistent microbicide use is critical in order to evaluate product effectiveness. Adherence is usually measured by self-report from participants enrolled in clinical trials to study personnel based on structured questionnaires ^{3, 4}, or by the use of computer-assisted interviews ⁵. The actual level of compliance with products can be overestimated with self-reporting ^{6, 7}.

Biomarkers for determining applicator usage have been developed to assess whether applicators have been exposed to vaginal fluid. A dye-based method, first proposed by Wallace et al. ^{8, 9} and recently performed by Govender et al. ¹⁰ and Hogarty et al. ¹¹ has been used to monitor adherence of microbicide product use in a recently completed feasibility study using 2.5% methyl cellulose placebo gel, a Phase 3 trial evaluating Carraguard ™ and a pilot study evaluating PRO 2000/5 (P), respectively. Despite some limitations encountered, these studies concluded that staining applicators offered a safe, inexpensive and easily replicated method for validating self-reported vaginal applicator use.

Our objectives for this study were two-fold: to perform the dye test using a common polypropylene applicator used widely in ongoing microbicide trials after a single documented insertion of gel and to evaluate the performance of this applicator type in a clinical trial in which gel is inserted intravaginally twice daily.

Materials and Methods

Vaginally inserted applicators representing positive controls (known to be vaginally inserted) were obtained from two groups. A total of 10, single-use, empty HTI polypropylene applicators (HTI Plastics, Lincoln, NE) were inserted by volunteers. A total of 60 HTI applicators inserted intravaginally by clinic staff were obtained from non-pregnant women aged 18–45 enrolled in a clinical trial evaluating the safety and efficacy of a single dose of UC 781 at Magee-Womens Hospital in Pittsburgh, PA. Written informed consent was obtained from each study participant prior to enrollment. The protocol was approved by the institutional review board of Magee-Womens Hospital of the University of Pittsburgh Medical Center.

A total of 169 participant–inserted applicators, corroborated by self-report, were obtained from non-pregnant women aged 18–24 enrolled in an ongoing trial evaluating the safety and efficacy of VivaGel (SPL7013). Women were enrolled at the Universities of South Florida and Puerto Rico in Tampa, FL and Puerto Rico, respectively. Participants were instructed to insert an applicator containing the gel and expel its contents twice daily, once in the morning and evening for 14 days and return used as well as unused applicators in their corresponding labeled bags to the study site on return visits. Written informed consent was obtained from each study participant prior to enrollment. The protocol was approved by the institutional review board of the Universities of South Florida and Puerto Rico.

Testing also included 70 applicators not exposed to vaginal fluid and were included in the study as negative controls.

Microlax applicators (Norden Pac, Kalmar, Sweden) containing no product were provided by the Population Council (New York, NY) and were used to document that the published method could be reproduced using the same applicator type.

Prior to staining, all HTI applicators were assigned randomly generated numbers using the program Research Randomizer ® and labeled accordingly using indelible ink on the grip of the applicator body. One half of each participant-inserted applicator body was swabbed with a cotton-tipped applicator moistened with sterile water and rolled onto a glass slide for Gram staining. In order to maintain masking of evaluators, Gram stains were labeled with another set of randomly generated numbers in order to prevent any unblinding of Gram stains with their corresponding applicators.

All applicators were stained with a 0.05% FD&C Blue Dye No. 1 solution (Prime Ingredients INC, Saddlebrook, NJ). Applicator staining was performed in batches. Briefly, applicators were placed in racks and each applicator body was sprayed 3 times with the FD&C Blue Dye No. 1 solution. All staining was performed in a biological safety cabinet with catch basins for excess dye. Following staining, each applicator was rinsed with tap water for five seconds and allowed to air dry.

Applicators and Gram stains were assessed independently by three evaluators to determine vaginal applicator use. Intravaginally inserted applicators should present with a “grainy turquoise pattern” following exposure to the dye to indicate use while unused applicators should present with a “smooth turquoise pattern”, indicating only gel present on the applicator body, or the applicator would not retain any stain. Gram stains were visualized using oil-immersion microscopy (1000X) for the presence of vaginal epithelial cells and/or common vaginal bacteria morphotypes to determine vaginal applicator insertion. An example of a Gram stain prepared from a used participant-inserted applicator is shown in Figure 1. An applicator was categorized as vaginally inserted if typical squamous vaginal epithelial cells with or without common bacterial morphotypes were present. Of the 169 participant-inserted applicators, 168 had vaginal epithelial cells with or without vaginal bacteria morphotypes present on their corresponding Gram stained smears.

Figure 1. Detection of Vaginal Material on Slide Prepared from HTI Applicator.

A Vaginal epithelial cell

B Lactobacillus morphotypes

The individual responsible for randomly labeling and preparing the applicators and smears did not serve as an evaluator.

Results

The sensitivity and specificity of the dye-based method ranged from 81–95% and 86–93%, respectively for single-use and unused applicators (Table 1). The dye-based method was only 47–77% sensitive for participant-inserted applicators obtained from women using gel twice daily for 14 days (Table 1).

Table 1.

Percentage of Applicators Correctly Identified Following Exposure to FD&C Blue Dye No. 1

			No. (% Correct)
Used Applicators	N	Dosing Schedule	Observer 1*	Observer 2	Observer 3
Volunteer Inserted	10		57/60=95%	66/70=94%	57/70=81%
Personnel Inserted	60
Participant-Inserted	169	Twice Daily/ 14 Days	130/169=77%	125/169=74%	80/169=47%
Unused Applicators	70		57/66=86%	64/70=91%	65/70=93%

^*Observer 1 did not evaluate the ten volunteer inserted applicators or four unused applicators.

Figure 2 and Figure 3 display the remarkable differences in retention of stain seen after staining single-use Microlax applicators and single-use polypropylene HTI applicators. The retention of stain is diminished further on participant-inserted applicators when gel is present in the vagina (Figure 4).

Figure 2. Single-Use Microlax Applicators.

1 Intravaginally Inserted (Grainy turquoise pattern present only)

2 Intravaginally Inserted with Gel (Grainy and smooth turquoise pattern present)

3 Gel Applied to Applicator without Vaginal Insertion (Smooth turquoise pattern present only)

4 Applicator Not Vaginally Inserted or Having Gel Applied (Note: Absence of dye on applicator)

Figure 3. Single-Use HTI Applicators.

1 Intravaginally Inserted

2 Intravaginally Inserted with Gel

3 Gel Applied to Applicator without Vaginal Insertion

4 Applicator Not Vaginally Inserted or Having Gel Applied

Figure 4.

Participant-Inserted Applicators (Twice Daily Gel Use for 14 Days)

Discussion

In published studies evaluating dye-based methods for determining vaginal applicator insertion, only Microlax and low density polyethylene HTI applicators (HTI Plastics) were used ^{9, 10, 11}. The applicators used in this study as well as several other studies evaluating BufferGel, PRO 2000/5 (P), UC 781 and Tenofovir are polypropylene. The feasibility study performed by Wallace et al. ⁹ with an observed accuracy of 95% and the Phase 3 trial evaluating Carraguard ™ ¹⁰ did not evaluate the effects of consecutive multiple applications of gel on retention of stain for the Microlax applicators. In our study, multiple applications of gel intravaginally resulted in a decreased binding of stain to the applicators, minimizing the sensitivity of the method (range 47–77%). In the pilot study performed by Hogarty et al. ¹¹ women were instructed to insert 0.5% PRO2000/5 (P) once daily for 14 days. The sensitivity and specificity (≥ 98% for both, respectively) was not found to decrease with multiple gel applications in their study. However, the differences observed in the sensitivities and specificities between our studies may be due to two factors. The additional gel present in the vagina of women using product twice daily as well as the differences in plastics, polypropylene vs. polyethylene, may interfere with the binding of vaginal fluid to the applicator surface and thus decrease the binding of stain.

Differences observed in sensitivities in this study were evaluator dependent and ranged from 47–95% depending on whether gel was present in the vagina. No explanation for the large degree of variability among evaluators can be made. All three evaluators were trained laboratory scientists with a minimum of five years of experience. All received a similar length of training regarding applicator staining and had continuous access to a protocol describing testing outcomes and applicators representing each outcome. The high level of variability in interpretation of test results may be due to the subjective nature of the colorimetric test, the low capacity of vaginal fluid to adhere to the smooth surface of the applicator and the subtle differences in the color resulting from vaginal fluid vs. gel exposure.

The present study differs from the other published studies with respect to the measurement of adherence by study participants. The published studies correlated positivity of the dye test with self-report only. In our study, adherence was not measured by the dye test or self-report but by the microscopic detection of vaginal squamous epithelial cells and/or common bacterial morphotypes in the sample eluted from the applicator surface. Of the 169 participant-inserted applicators, 99% had vaginal epithelial cells and/or common vaginal bacteria morphotypes observed on smears prepared from the applicators, indicating that the applicator surface was exposed to the vaginal wall but does not indicate if gel was actually discharged into the vagina.

The dye-based method cannot be recommended for evaluation of polypropylene HTI applicator insertion during microbicide trials. The low sensitivity observed under trial conditions requiring multiple applications of gel suggests that the performance of this dye-based method is applicator dependent and cannot be recommended for widespread use.

Adherence to product use in prevention trials is critical. Any interruption or deviation from protocol can decrease study power and can result in unreliable interpretation. Validation of self-reported behaviors requires objective methods in order to determine true compliance with intercourse-related interventions. Although the use of Audio Computer-Assisted Self Interviewing (ACASI) vs. Face to Face Interviewing (FTFI) has been shown to elicit more honest responses regarding questions requiring desirable outcomes ^{12, 13, 14}, published studies have reported that using tests such as Prostate-Specific Antigen testing, Rapid Stain Identification of Human Semen (RSID) and the dye-based method for determining applicator insertion, that study participants are still likely to over report desirable outcomes such as condom and microbicide use ^{15, 16, 17}.

Although these studies have reported that objective methods to detect product usage indicate that study participants over report desired behaviors, there are limitations. Human semen degrades rapidly within the vagina and Prostate-Specific Antigen testing and RSID must be performed within 48 hours of intercourse. The high degree of variability with the dye-based method for determining vaginal applicator insertion with polypropylene HTI applicators observed in this study, even when performed under optimized laboratory conditions, suggests that it is not an effective method of determining compliance among individuals enrolled in microbicide trials. However, continued research directed toward development of objective biomarkers for measuring adherence to product use is of great importance.