Biomarker Validation of Reports of Recent Sexual Activity: Results of a Randomized Controlled Study in Zimbabwe

Abstract

Challenges in the accurate measurement of sexual behavior in human immunodeficiency virus (HIV) prevention research are well documented and have prompted discussion about whether valid assessments are possible. Audio computer-assisted self-interviewing (ACASI) may increase the validity of self-reported behavioral data. In 2006–2007, Zimbabwean women participated in a randomized, cross-sectional study that compared self-reports of recent vaginal sex and condom use collected through ACASI or face-to-face interviewing (FTFI) with a validated objective biomarker of recent semen exposure (prostate-specific antigen (PSA) levels). Of 910 study participants, 196 (21.5%) tested positive for PSA, an indication of semen exposure during the previous 2 days. Of these 196 participants, 23 (11.7%) reported no sex in the previous 2 days, with no difference in reported sexual activity between interview modes (12.5% ACASI vs. 10.9% FTFI; Fisher's exact test: P = 0.72). In addition, 71 PSA-positive participants (36.2%) reported condom-protected vaginal sex only; their reports also indicated no difference between interview modes (33.7% ACASI vs. 39.1% FTFI; P = 0.26). Only 52% of PSA-positive participants reported unprotected sex during the previous 2 days. Self-report was a poor predictor of recent sexual activity and condom use in this study, regardless of interview mode, providing evidence that such data should be interpreted cautiously.

The challenges of obtaining accurate measures of self-reported sexual behavior in sexually transmitted infection and human immunodeficiency virus (HIV) prevention research are well documented. Study participants may provide inaccurate reports of their sexual behavior for a variety of reasons, including social desirability bias (e.g., as a result of repeated counseling messages related to condom and study product use), poor recall, concerns about being dropped from the study if they report low levels of sexual activity or product use, or poorly worded questions or response categories (1–4). The need for accurate behavioral data remains important for interpreting results from product effectiveness studies (to differentiate poor adherence from lack of product efficacy) and for assessing product safety (5). Furthermore, reliable behavioral outcomes may be needed to evaluate interventions conducted in settings or among population groups where disease incidence is low or where behavior change is the key outcome of interest (6, 7). Nonetheless, persistent difficulties in collecting accurate behavioral data on sexual activity and condom use have prompted some investigators to question whether the collection of valid behavioral data is even possible.

Interviewer-administered assessments are the standard data collection mode used to ascertain sexual behavior in sexually transmitted infection or HIV prevention research. However, researchers have adopted audio computer-assisted self-interviewing (ACASI) because it has shown promise in increasing the accuracy of self-reported behaviors through the use of a self-administered questionnaire that affords privacy (by not requiring an interviewer or a written record of responses). In addition, ACASI can be designed for low-literacy populations and can accommodate questionnaires with complex skip patterns (8, 9). Evaluations of ACASI to date have been designed as comparisons of responses between data collection modes (e.g., ACASI vs. face-to-face interviewing (FTFI)), with inferences regarding validity being based on the assumption that higher reports of sensitive behaviors are more accurate. Numerous randomized studies, mostly conducted in the United States, have shown higher levels of reported risky sex-related behaviors when ACASI is employed compared with FTFI. However, other studies have found inconsistent results (lower reports of risky behavior with ACASI or no differences between interview modes) depending on the behavior assessed (10–20). Because of the lack of a standard in self-report measures, the use of an objective biomarker of sexual activity is needed for direct evaluation of the effect of ACASI on the validity of behavioral assessments.

We conducted a study among Zimbabwean women to compare the validity of reports of recent vaginal sex (with and without condoms) as measured by ACASI with the validity of reports measured by FTFI. We used the detection of prostate-specific antigen (PSA), a biomarker with high positive predictive value (21), as the reference standard for recent semen exposure and hypothesized that ACASI would yield a lower proportion of discrepant results (e.g., PSA-positive but no recent sexual activity reported) than FTFI.

MATERIALS AND METHODS

Study design

This study consisted of a cross-sectional assessment of recent sexual activity and condom use. It was conducted from December 2006 through June 2007 among a subset of women who had recently completed participation in the Methods for Improving Reproductive Health in Africa (MIRA) Trial. MIRA was a large randomized controlled trial that examined the protective effects of diaphragms plus lubricant gel in decreasing susceptibility to HIV. The trial was conducted among sexually active HIV-negative women aged 18–49 years at 3 sites in South Africa and Zimbabwe (22). Women participated in our study a median of 8.8 months (range, 2.5–20.5) after their final MIRA study visit. Study participants provided a self-obtained vaginal swab that was tested for PSA and were randomized to complete a short behavioral interview administered through either ACASI or FTFI. The interview comprised questionnaire items similar to those completed during the MIRA trial.

Study participants and data collection

Former MIRA Trial participants from the 2 Zimbabwe MIRA study clinics in Chitungwiza and Epworth were eligible for participation. Women learned about the PSA study at their final MIRA study visit, through community outreach, or during post-MIRA clinic visits (e.g., visits made to obtain additional condoms). Women who expressed interest were encouraged to return to the MIRA clinic to consent to and enroll in the PSA study. Only nonpregnant former MIRA participants who had not had a vaginal delivery or third-trimester stillbirth in the previous 6 weeks were eligible for enrollment. All participants gave written informed consent prior to enrollment in the PSA study. Institutional review boards at the University of California, San Francisco, the Medical Research Council of Zimbabwe, and Family Health International approved the study.

Participants were randomized to either the ACASI group or the FTFI group by means of sequentially numbered, sealed opaque envelopes containing randomization assignments. Using their assigned mode of interviewing, participants answered questions about their sexual activity and condom use in the past 7 days. They also were asked about the following problems with condom use: breakage, slippage, and spillage. Participants then provided a self-collected vaginal swab by following instructions developed for a MIRA ancillary study (designed to examine whether diaphragm use conferred protection against human papillomavirus) in which most of the PSA study participants had taken part. Vaginal swabs were air-dried and shipped in batches to a research laboratory at the University of North Carolina, Chapel Hill, for PSA testing.

PSA detection

PSA is produced by the prostate gland and secreted into the urethra during ejaculation. The protein can be detected in vaginal fluid specimens collected after exposure to semen and has high positive predictive value (21, 23, 24). PSA begins to clear from the vaginal fluid promptly after exposure (e.g., PSA is only detectable in an estimated 29% of women 24 hours after known exposure) and is rarely present beyond 48 hours after semen exposure (21). Because of the rapid clearance of PSA, our analysis focused on discrepant reporting among women who tested PSA-positive only (i.e., no report of recent intercourse or condom-protected intercourse only and a PSA-positive test result). We used the Abbott Laboratories IMx assay (Abbott Laboratories, Abbott Park, Illinois) to detect PSA in the vaginal swab specimens. PSA levels greater than 1.0 ng/mL indicate semen exposure within the past 48 hours (21).

For recovery of vaginal secretions, each swab was placed into 3.0 mL of phosphate-buffered saline, incubated at room temperature for 15–30 minutes, and then agitated and pressed against the side of the tube to elute the sample. To avoid including swabs that had not been used to collect vaginal fluid, we scored vaginal swab eluates from self-obtained specimens for the presence of epithelial cells. A small volume of specimen (0.01 mL) was loaded into the chamber of a hemacytometer and examined at 200× magnification in a Nikon Labphot 2 light microscope (Nikon Instruments, Inc., Melville, New York). Epithelial cells were present in all self-obtained vaginal swab eluates. Vaginal specimens were then centrifuged at 250 × g for 10 minutes, and supernatants were removed from cell pellets and stored at −80°C until testing. Supernatants (0.20 mL) from vaginal swab eluates were tested using the IMx PSA assay according to the manufacturer's instructions. The enzyme immunoassay measures PSA concentrations from 0.04 ng/mL to 50 ng/mL. Samples with initial test results greater than 50 ng/mL were diluted 1:100 with phosphate-buffered saline and retested to obtain PSA concentrations.

Analysis

The analysis population consisted of participants who provided a vaginal swab specimen for PSA testing and completed the sexual behavior interview. Data on participant sociodemographic characteristics and sexual history were extracted from the baseline MIRA study visit. We used chi-squared tests to compare the proportions of women who reported having vaginal sex and condom-protected vaginal sex in the past 2 days via ACASI with the proportions who reported those activities via FTFI. To compare self-reported behaviors with PSA test results, we conducted a 1-sided Fisher's exact test, on the basis of our hypothesis that ACASI would yield increased reporting of sensitive behaviors (i.e., no recent vaginal sex and vaginal sex unprotected by a condom) and thus lower the proportion of results that were discrepant with the PSA test results as compared with FTFI. We established our sample size to ensure 80% power to detect a 33% reduction in discordance (reporting no vaginal sex in the previous 2 days but testing PSA-positive) by ACASI and assumed that the discordance in the FTFI group would be 20% (1-sided test; α = 0.05). Our primary endpoint was defined as reporting no sex for the past 2 days yet testing positive for PSA. Secondary endpoints included having a PSA-positive test result yet reporting: 1) only vaginal sex protected by a condom; 2) only vaginal sex protected by a condom that was used without breakage, slippage, or spillage of semen; and 3) no vaginal sex in the previous 7 days. To assess the robustness of our findings to different PSA levels, we evaluated a semiquantitative measure by using 4 categories of PSA concentration as evidence of PSA positivity (1.1–5.0 ng/mL, 5.1–25.0 ng/mL, 25.1–100.0 ng/mL, and >100.0 ng/mL). Our findings for all study endpoints were consistent with those obtained using the dichotomous cutpoint of >1.0 ng/mL (i.e., P values remained nonsignificant); therefore, we present only the results of the primary analysis.

RESULTS

Study participants

The PSA study included 918 former participants from the MIRA Trial (36.7% of the MIRA Zimbabwe sample). Approximately equal numbers of participants were assigned to the ACASI and FTFI groups. Eight women in the ACASI arm did not complete their interviews because of loss of electrical power for several consecutive days at 1 site. Consequently, the analysis population consisted of 910 participants (450 ACASI and 460 FTFI). At the time of entry into the MIRA Trial, most PSA study participants were married (96.6%) and had not completed high school (55.7%). Participants reported a mean of 1.3 lifetime sexual partners (range, 1–5) and a mean age of 18.5 years at first sexual intercourse (range, 10–28). The baseline characteristics of women in the ACASI and FTFI groups were similar (Table 1).

Table 1.

Sociodemographic Characteristics, Reproductive History, and Self-Reported Sexual Behavior^a Among 910 Sexually Active Women, by Randomization Group, Prostate-Specific Antigen Study, Zimbabwe, 2006–2007

	ACASI group (n = 450)		FTFI group (n = 460)
	No.	%	No.	%
Age, years
18–24	131	29.1	157	34.1
25–34	219	48.7	215	46.8
≥35	100	22.2	88	19.1
Education
Less than high school	253	56.2	254	55.2
High school or more	197	43.8	206	44.8
Earned income in past year
Yes	335	74.4	360	78.3
No	115	25.6	100	21.7
Married
Yes	438	97.3	441	95.9
No	12	2.7	19	4.1
Mean lifetime no. of sexual partners (range)	1.3 (1–5)		1.3 (1–4)
Mean age at first sexual intercourse, years (range)	18.5 (10–28)		18.5 (13–28)
Coital frequency, no. of times per week
≤3	238	52.9	255	55.4
>3	212	47.1	205	44.6
Tested positive for sexually transmitted infectionb
Yes	33	7.3	30	6.5
No	417	92.7	430	93.5
Tested positive for herpes simplex virus type 2c
Yes	238	53.1	239	52.0
No	210	46.9	221	48.0
High behaviorial riskd
Yes	107	23.8	117	25.4
No	343	76.2	343	74.6
Condom use in past 3 months
Never	122	27.1	131	28.5
Sometimes	199	44.2	206	44.8
Always	129	28.7	123	26.7
Primary type of contraceptive use at screeninge
Tubal ligation, vasectomy, intrauterine device, implants	11	2.4	17	3.7
Injectable hormones	63	14.0	67	14.6
Oral contraceptives	279	62.0	293	63.7
Male or female condoms	58	12.9	55	12.0
Other/none	39	8.7	28	6.1
MIRA Trial randomization group
Diaphragm, gel, and condom	216	48.0	226	49.1
Condom-only	234	52.0	234	50.9

Abbreviations: ACASI, audio computer-assisted self-interviewing; FTFI, face-to-face interviewing; MIRA, Methods for Improving Reproductive Health in Africa.

^aMeasured at baseline for the MIRA Trial.

^bAt least 1 positive test for Neisseria gonorrhoeae, Chlamydia trachomatis, or Trichomonas vaginalis at baseline in the MIRA Trial.

^cData on herpes simplex virus type 2 status were missing for 2 subjects because of missing laboratory test results.

^dAt least 1 of the following: any exchange of sex for money, food, drugs, or shelter; ≥2 sexual partners within the past 3 months; having vaginal sex under the influence of drugs or alcohol in the past 3 months; ever using a needle for injectable drug use; and ever having anal sex.

^eParticipants were classified on the basis of the most effective method reported.

Self-reported recent sexual activity

Thirty-four percent of the women (n = 310) reported having no vaginal sex in the past 2 days, and 36.4% (n = 331) reported using a condom for all vaginal sex acts during this period. The proportion of women who reported no vaginal sex did not vary between interview modes (34.0% ACASI vs. 34.4% FTFI; P = 0.48), but the proportion who reported condom-protected sex only was lower in the ACASI group than in the FTFI group (32.7% vs. 40.0%; P = 0.01).

PSA detection and discordance between self-reported behaviors and PSA results

We found 21.5% of participants (n = 196) to be PSA-positive, and the proportions of PSA-positive results did not differ by randomization group (P = 0.25). The median PSA concentration among participants who tested positive (>1 ng/mL) was 15.7 ng/mL (interquartile range, 5.3–82.6). The distribution of positive PSA values is depicted in Figure 1. Twenty-three (11.7%) participants who tested positive for PSA reported having no vaginal sex in the previous 2 days, and we observed no difference between interview modes (12.5% ACASI vs. 10.9% FTFI; P = 0.72; Table 2). Seventy-one (36.2%) participants who tested positive for PSA reported condom-protected sex only, and no difference by interview mode was observed (33.7% ACASI vs. 39.1% FTFI; P = 0.26). Five of these 71 participants experienced condom breakage (n = 2), slippage (n = 1), or semen spillage (n = 2) during this time period; no difference between interview modes was observed. Thus, reports of condom problems also did not explain discordant results in general (the overall level of discordance was reduced from 48% to 45% when these women were excluded) or between the ACASI and FTFI groups. Finally, among the PSA-positive participants, 7.7% (n = 15) reported having no vaginal sex over the prior 7 days. However, the samples were too small for us to make meaningful comparisons between interview modes.

Figure 1.

Distribution of prostate-specific antigen (PSA) concentrations among 910 sexually active women who completed an interview about sexual activity and condom use in the past 7 days, Prostate-Specific Antigen Study, Zimbabwe, 2006–2007. PSA concentrations greater than 1.0 ng/mL were considered as providing evidence of semen exposure within the past 2 days.

Table 2.

Biomarker Validation (by PSA Testing^a) of Reports of Sexual Activity and Condom Use Among Sexually Active Women^b, Prostate-Specific Antigen Study, Zimbabwe, 2006–2007

Reported Sexual Activity During the Past 2 Days	Randomization Group				Total (n = 196)		P Valuec
	ACASI (n = 104)		FTFI (n = 92)
	No.	%	No.	%	No.	%
No vaginal sex	13	12.5	10	10.9	23	11.7	0.72
Sex protected by a male or female condom only	35	33.7	36	39.1	71	36.2	0.26
Total	48	46.2	46	50.0	94	48.0	0.35

Abbreviations: ACASI, audio computer-assisted self-interviewing; FTFI, face-to-face interviewing; PSA, prostate-specific antigen.

^aPSA concentrations greater than 1.0 ng/mL were considered as providing evidence of semen exposure within the past 2 days.

^bAnalysis was restricted to the subset of women (n = 196) who tested positive for PSA (>1 ng/mL) in vaginal eluate.

^c1-sided Fisher's exact test.

DISCUSSION

In this study, we found substantial levels of discrepant reporting of recent vaginal sex and sex with a condom relative to the presence of a validated marker of recent semen exposure. Nearly half of participants with biologic evidence of recent semen exposure reported either that they had not had sex (12%) or that they had had condom-protected sex only (36%) during the previous 2 days. Direct comparison of self-reports between interview modes in the absence of the biomarker suggested that ACASI yielded lower reports of consistent condom use during recent sexual activity—ostensibly an indication that ACASI had a modest effect on improving the accuracy of self-reports. Nevertheless, as compared with FTFI, ACASI did not improve the level of concordance between self-reports and the PSA biomarker results.

The high level of discrepant reporting in this study, regardless of interview mode, highlights the question of whether sufficiently valid self-reported behavioral data can be obtained in HIV prevention research or in other studies that rely on reports of sexual activity. Our findings corroborate those from several studies, including 2 that found high levels of discrepancy between reports of condom use and PSA positivity among female sex workers in Kenya and Madagascar (25, 26); 1 that found a substantial discrepancy between reports of condom use and the presence of Y chromosome in the vagina (another biomarker of recent sexual behavior) among adolescents in the United States (27); and another that identified detectable spermatozoa in a substantial proportion of women reporting consistent condom use in a microbicide clinical trial in South Africa (28). The consistency of apparent overreporting of condom use or underreporting of sexual activity (even with ACASI data collection (27)), as corroborated by the presence of a biomarker of recent sexual activity, is worthy of further exploration. These results point to several possible explanations, including perceived stigma or social desirability bias associated with admitting nonadherence to HIV prevention strategies (e.g., condom use, partner reduction, or product use in a microbicide trial) that may prompt intentional misreporting (29); poor recall, even of recent sexual activity; complex sexual behavior that cannot be captured using standard quantitative instruments (30); or poorly worded questions that can be misinterpreted by study participants.

Given these seemingly intractable problems with behavioral assessments of sexual activity, particularly condom use, should we simply conclude that such data cannot be collected at all? In part this depends upon the research questions being investigated and the availability of adequate biomarkers for assessing the behaviors of interest. A biomarker like PSA detects only recent semen exposure and constitutes a measure distinct from those that assess condom-use behaviors more thoroughly. Because of user errors and mechanical failures, some exposure to semen does occur with condom use (31–33), although that did not seem to influence results here. Future in-depth qualitative interviews (which we were not able to conduct) with study participants for whom discrepancies between PSA test results and self-reported sexual behavior were identified might aid in the interpretation of these findings and guide subsequent decisions on questionnaire wording and on the influence of technological innovations like ACASI.

Several issues should be noted in interpreting our findings. First, although PSA has proven to be a valid biomarker of recent semen exposure (21), low levels of PSA could in fact reflect false-positive results due to specimen contamination (34). However, the proportions of discrepant responses overall, and between ACASI and FTFI, were comparable when we examined a semiquantitative measure of PSA that permitted comparison at 4 PSA concentrations, indicating that our results are robust to the PSA cutpoint chosen. Second, because our measure of consistent and correct condom use was incomplete (in that we did not evaluate the timing of use and whether condoms may have been applied following onset of genital contact or removed prior to ejaculation), we might have overestimated the level of discrepant results for a minority of participants who may have used condoms but did so incorrectly. Previous studies suggest that these problems may be common (35); however, delayed application of condoms is unlikely to explain our findings, because PSA levels in preejaculate are below the threshold we used for defining PSA positivity (36).

In contrast, it is far more likely that we underestimated the overall prevalence of discrepant results (and possible misreporting) in our study population because of the rapid decay curve for PSA (21), which generally is not detectable after 24 hours for the majority of samples. As a result, some participants who reported no sexual activity over the last 2 days may have had recent intercourse but with enough lead time for their PSA result to be negative. Even if this had occurred, it is unlikely that this underestimation would differ between the randomized groups, and thus it cannot explain the lack of difference between interview modes. Finally, because reporting of sexual history and other sensitive behaviors probably varies across populations and settings, our findings may have limited generalizability for measuring the utility of ACASI as compared with FTFI, especially given that women in the study had all previously participated in an HIV prevention trial.

In this study, ACASI did not improve the validity of reports of recent sexual activity and condom use over FTFI based on comparison of self-reports with a validated biomarker of semen exposure. Overall levels of discordance between self-reports of sexual behavior and the presence of a PSA biomarker were high. These findings underscore the limitations of relying exclusively on self-reported sexual behavioral data and highlight the need for cautious interpretation of behavioral data on condom use collected using current approaches. Objective biologic measures of sexual activity and product use (37) that can be employed in research that requires valid assessments of sexual activity are urgently needed.

Glossary

Abbreviations

ACASI

audio computer-assisted self-interviewing

FTFI

face-to-face interviewing

HIV

human immunodeficiency virus

MIRA

Methods for Improving Reproductive Health in Africa

PSA

prostate-specific antigen