Aggregate Versus Individual-Level Sexual Behavior Assessment: How Much Detail Is Needed to Accurately Estimate HIV/STI Risk?

Abstract

The sexual behaviors of HIV/sexually transmitted infection (STI) prevention intervention participants can be assessed on a partner-by-partner basis: in aggregate (i.e., total numbers of sex acts, collapsed across partners) or using a combination of these two methods (e.g., assessing five partners in detail and any remaining partners in aggregate). There is a natural trade-off between the level of sexual behavior detail and the precision of HIV/STI acquisition risk estimates. The results of this study indicate that relatively simple aggregate data collection techniques suffice to adequately estimate HIV risk. For highly infectious STIs, in contrast, accurate STI risk assessment requires more intensive partner-by-partner methods.

Introduction

The effectiveness of behavioral interventions to reduce participants’ risk of acquiring or transmitting the HIV or another sexually transmitted infection (STI) typically is evaluated by assessing changes in one or more specific behaviors associated with HIV/STI acquisition, such as engaging in unprotected sex or having multiple sex partners (Holtgrave and Pinkerton 2000). This is sensible in as much as HIV/STI risk is mainly determined by the number of sex acts, the number of sex partners, and the proportion of sex acts that are protected by condoms.

Importantly, the risk of acquiring HIV or another STI depends on all the above factors—not any one in isolation. For example, engaging in frequent “one night stands” (isolated sexual activities with new partners) but always using condoms may not be as risky as having repeated unprotected sex with a single partner (Pinkerton and Abramson 1993; Reiss and Leik 1989). We have suggested elsewhere that HIV/STI prevention program evaluations should include a more comprehensive, integrative measure of program efficacy, based on the estimated risk of HIV/STI acquisition (Holtgrave et al. 1997; Pinkerton et al. 1998). Isolated measures of sex acts, sex partners, and condom use provide only a piecemeal assessment of intervention impact and do not directly address the critical questions: Did the program reduce participants’ risk of HIV/STI acquisition, and if so, by how much?

It might seem that quite extensive data would need to be collected to accurately gauge HIV/STI risk. Data requirements are determined, in part, by the mathematical model used to estimate risk. There is a natural trade-off between the level of sexual behavior detail and the precision of HIV/STI acquisition risk estimates. But how large is this trade-off?

The question of data precision becomes even more important when the sexual behavior data are intended to be used to estimate HIV/STI acquisition risks (Pinkerton et al. 1998). Risk estimates have been used in a number of studies to characterize risk levels in particular groups or as an outcome indicator (Bell and Trevino 1999; Benotsch et al. 2006; Kalichman et al. 2001; Smith et al. 1993). Even more commonly, HIV/STI acquisition risk estimates provide the basis for evaluating the cost-effectiveness of HIV/STI prevention programs (Pinkerton and Holtgrave 1998; Weinstein et al. 1989).

The current “best practice” for sexual behavior assessment entails asking detailed questions, on a partner-by-partner basis, for up to m sexual partners, then collecting aggregate data on the remaining partners (National Institute of Mental Health Multisite HIV Prevention Trial Group 1998). For example, a study might ask about sexual activities (unprotected and condom-protected vaginal and anal intercourse) with up to five partners and then ask about these activities, in aggregate, across any remaining partners. Alternately, if the number of partners is not too large, all partners might be individually assessed in detail. At the other extreme, all partners might be assessed in aggregate. This is the least costly and burdensome assessment strategy but provides the least detailed sexual behavior information.

The current study examined the potential gains in HIV/STI risk estimate accuracy obtained when the number of individually assessed partners is increased from zero (a completely aggregate assessment) up to the total number of partners reported by each study participant. Data from a daily sexual behavior diary study were used to simulate various types of assessment strategies, and mathematical models of HIV/STI transmission were used to estimate HIV/STI risk. Estimation accuracy was evaluated by comparing the risk estimates obtained for a particular assessment strategy to the presumed “gold standard,” in which detailed sexual behavior information is collected for each and every sex partner. We were especially interested in whether different assessment strategies might be required to accurately estimate the risk of acquiring a highly infectious STI such as syphilis, chla-mydia, or gonorrhea, as opposed to estimating the risk of HIV acquisition. We demonstrate below that, for HIV risk estimation, very simple assessment techniques are sufficient to adequately characterize risk, whereas accurate STI acquisition risk estimation requires more elaborate methods.

Methods

Sexual behavior data were collected from 160 high-risk men (HRM), 163 high-risk women (HRW), and 145 high-risk men who have sex with men (MSM) using a daily diary methodology (McAuliffe, DiFranceisco, and Reed 2007). Participants were recruited at health centers serving a predominately low-income African American clientele, at an anonymous HIV testing clinic and at various gay-oriented community events. All participants were 18 years of age or older, had been sexually active in the 3 months prior to enrollment, and were not in an exclusive sexual relationship. Participants used a structured diary form to record their daily sexual activities over a 90-day period. This form enabled the participant to report comprehensive information about each sexual encounter, including the date of the activity and a partner code; types of sexual activity (unprotected or condom-protected anal, vaginal, or oral intercourse); and the number of times each sexual activity type took place during the encounter. Sample demographic and detailed sexual behavior information is available elsewhere (McAuliffe, DiFranceisco, and Reed 2007).

A simulated partner-by-partner data set was created from the diary data by collecting together, for each study participant, all sex acts with each individual partner. This data set corresponds to a data collection strategy that elicits detailed sexual behavior information about each and every partner in the past 90 days. To simulate other data collection strategies—for example, collecting detailed information about m sex partners and aggregate data for the remaining partners—partners were prioritized according to (a) the total number of sex acts with the partner; (b) the total number of unprotected sex acts; (c) the date of last intercourse with the partner; or (d) the “actual” risk of HIV or STI acquisition associated with the partner. The m highest priority partners were included in the partner-by-partner portion of the simulated data set and the remaining partners were relegated to the aggregate portion. (When m = 0, the simulated data set consisted of the aggregate portion only.)

Mathematical models were used to estimate HIV and STI acquisition risks for each study participant, subject to the constraints of the simulated data sets. The accuracy of the resultant risk estimates depended on the level of detail included in the simulated data sets. The mathematical models, which are described in detail in the Appendix, took into account the number of condom-protected and unprotected sex acts, the number of sex partners, the prevalence of HIV or STI infection among partners, the per-act transmission probabilities associated with specific sexual activities (e.g., receptive vaginal intercourse), and the effectiveness of condoms in preventing HIV/STI transmission (Pinkerton and Abramson 1993, 1998).

For a simulated assessment strategy in which up to m partners are assessed in detail and the remaining partners assessed in aggregate, each participant’s overall HIV or STI risk was calculated using the equation:

$$P(m)=1-(1-P_{\mathrm{A}})(1-P_1)\dots (1-P_{\mathrm{m}}),$$ (1)

where P_k (k = 1 … m) is the risk associated with individually assessed partner k and P_A is the combined risk of the partners assessed in aggregate (see Appendix for additional information). It can be shown that this risk estimate, P (m), never underestimates the “true” risk, P(M), obtained by assessing each partner individually and none in aggregate (Pinkerton and Abramson 1993). Consequently, the estimation error for a particular simulated assessment strategy equals (P (m) — P(M)) P(M). The mean HIV or STI estimation error across study participants was the main outcome indicator used in the analyses presented below. Larger mean errors indicate reduced accuracy in the HIV/STI risk estimates obtained for a particular assessment strategy.

The analyses assumed a 5% prevalence of HIV/STI infection among sex partners (alternate values of 1% and 10% also were considered) and used the following per-act transmission HIV probabilities: .02 (receptive anal intercourse), .001 (receptive vaginal intercourse), and .0005 (insertive vaginal or anal intercourse; Katz and Gerberding 1997). All three transmission probabilities were set to .5 for the STI analyses (Stone 1994). Condoms were assumed to be 90% effective in preventing HIV/STI transmission (Pinkerton and Abramson 1997).

Results

The first set of analyses assumed that, when individual partners were assessed, they were prioritized based on the total number of sex acts; thus, the detailed partner-by-partner assessment, if any, always included the partner with whom the study participant engaged in the most sex acts. Very likely this person would be the participant’s “main” partner if he or she had one.

The results of the HIV risk analysis are presented in Table 1. The simplest assessment strategy—which entails only an aggregate assessment over all partners, without collecting detailed information about any specific partner—produced a 0.55% (HRM) to 8.47% (MSM) overestimation in the risk of HIV acquisition compared to the partner-by-partner risk estimates. Supplementing the aggregate data with a detailed assessment of the partner with whom the participant reported the most sex acts decreased the estimation error to less than 0.5% for the HRW and HRM samples and to 1.61% for the MSM sample.

Table 1.

HIV Risk Analysis: Estimated Risk of Acquiring HIV Based on Number of Individually Assessed Partners

	Maximum # of Individually Assessed Partners
	0	1	2	3	Alla
High-risk men (N = 160)
Mean HIV risk × 1,000	0.9045	0.9013	0.9006	0.9003	0.8995
Estimation error (%)b	0.55	0.20	0.12	0.10	0
High-risk women (N = 163)
Mean HIV risk × 1,000	3.6191	3.4768	3.4636	3.4614	3.4603
Estimation error (%)	4.59	0.48	0.09	0.03	0
Men who have sex with men (N =145)
Mean HIV risk × 1,000	5.2187	4.8884	4.8524	4.8396	4.8113
Estimation error (%)	8.47	1.61	0.86	0.59	0
Combined sample (N = 468)
Mean HIV risk × 1,000	3.1866	3.0337	3.0176	3.0128	3.0034
Estimation error (%)	6.10	1.01	0.47	0.31	0

^aPresumed “gold standard” to which other sexual behavior assessment strategies were compared.

^bPercentage error compared to “gold standard,” in which all partners are individually assessed. The estimation error for a simulated assessment strategy in which only m sex partners are assessed in detail equals (P (m) - P(M))/P(M), where P(m) is the risk estimate obtained for the simulated strategy and P(M) is the risk estimate obtained when all sexual partners are individually assessed. Larger mean errors indicate reduced accuracy in the HIV/STI risk estimates obtained for a particular assessment strategy.

Across the three samples, the completely aggregate data collection strategy produced a 6.10% HIV risk estimation error. Adding a single detailed “main” partner assessment decreased the error to 1.01%, an 83% reduction. Collecting detailed information on additional partners produced only a very slight improvement in accuracy (see Table 1). Similar findings were obtained regardless of whether the prevalence of infection among partners was 1%, 5%, or 10% (data not shown).

For highly infectious STIs such as syphilis, chlamydia, or gonorrhea, aggregate data collection resulted in a very high 42.7% estimation error across the three samples, as shown in Table 2. Assessing a single partner in detail decreased the error by 59%, to 17.3%. To reduce the error below 5% would require assessing 3 or more partners in detail and to reduce it below 1% would necessitate collecting detailed information on at least 6–10 partners, depending on the presumed prevalence of infection among these partners.

Table 2.

Mean HIV and Sexually Transmitted Infection (STI) Risk Estimation Errors for Combined Sample (N = 468)

	Maximum # of Individually Assessed Partners (m)
Prioritize Partners by	0	1	2	3	4	6	8
HIV risk estimation error (%)
Most recent partner	6.10	1.30	1.14	0.98	0.95	0.84	0.49
Most sex acts	6.10	1.01	0.47	0.31	0.14	0.08	0.08
Most unprotected acts	6.10	0.77	0.45	0.31	0.13	0.04	0.03
Greatest actual risk	6.10	0.66	0.37	0.24	0.12	0.03	0.02
STI risk estimation error (%)
Most recent partner	42.68	23.54	15.79	9.71	7.12	4.72	3.03
Most sex acts	42.68	17.32	7.89	4.31	3.04	1.61	0.96
Most unprotected acts	42.68	16.73	7.19	3.69	2.25	1.05	0.49
Greatest actual risk	42.68	16.54	6.90	3.44	2.11	0.83	0.39

Because unprotected sex acts are 10 times riskier than protected sex acts (assuming 90% condom effectiveness), prioritizing partners based on the number of unprotected sex acts rather than the total number of sex acts improved the accuracy of the HIV/STI risk estimates, but only slightly, as indicated in Table 2. In theory, the maximum benefit from conducting detailed individual partner assessments would be realized if partners were prioritized by actual risk rather than by the number of sex acts. As indicated in Table 2, such a strategy—even if feasible—would not substantially increase accuracy. Prioritizing partners by recency of last intercourse produced the largest errors of any of the four prioritization schemes considered in the analyses.

Linearity and Nonlinearity

The results presented above are a direct consequence of the very different transmission dynamics of HIV, which is not readily transmitted during a single sex act, and those of highly infectious STIs (Pinkerton et al. 2002, forthcoming). As indicated in Figure 1, transmission of highly infectious STIs is a decidedly nonlinear function of the number of sex acts with each partner. The risk of acquiring a highly infectious STI grows very quickly during the first few sex acts with a new partner. For a highly infectious STI with a per-act transmission probability of .5, for example, the risk of STI acquisition equals 98.4% of its theoretical maximum after only six sex acts; additional sex acts do not appreciably increase risk further. However, when sex acts are distributed among multiple partners, each new partner essentially “resets” the risk accumulation process, so that the first few sex acts with each new partner are extremely risky, as illustrated in Figure 2. Consequently, the number of sex partners and, moreover, the distribution of sex acts among these partners are both important determinants of the risk of acquiring a highly infectious STI (Pinkerton et al., forthcoming).

Figure 1.

Risk (probability) of acquiring HIV or a highly infectious sexually transmitted infection (STI) from a potentially infected partner, as a percentage of the risk associated with a maximum of 25 unprotected sex acts. For highly infectious STIs, risk grows rapidly in the first several sex acts and then more slowly as saturation is approached. For HIV transmission during receptive vaginal or anal intercourse, the per-partnership risk is essentially a linear function of the number of unprotected sex acts with that partner.

Figure 2.

Risk of acquiring a highly infectious sexually transmitted infection (STI) as a function of the number of unprotected sex acts and the number of partners. Each new partner essentially “resets” the risk accumulation process but at a higher baseline level. Consequently, the number of sex partners is a critical determinant of the risk of acquiring a highly infectious STI.

In contrast, because the probability of acquiring HIV during a single sex act is quite small, the overall probability of HIV acquisition across multiple acts with multiple partners is essentially a linear function of the total number of sex acts (Pinkerton et al., forthcoming). Each sex act is associated with approximately the same risk of HIV acquisition as all other acts, regardless of the number of sex partners (see Figure 3). Consequently, aggregate strategies that focus only on the total number of sex acts, without regard to the distribution of acts among partners, produce highly accurate HIV risk estimates.

Figure 3.

Risk of acquiring HIV through receptive anal intercourse as a function of the number of unprotected sex acts and the number of partners. The risk associated with each sex act is approximately the same, regardless of the number of sex partners.

Discussion

By convention, most statistical significance testing algorithms use either a 1% or a 5% criterion for establishing the validity of test results—that is, the results are considered valid up to a 1–5% margin of uncertainty. The results of the analysis presented above indicate that, for HIV risk estimation, collecting detailed information on a single partner and aggregate data for remaining partners is sufficient to achieve a similarly low (1–5%) margin of error. In contrast, for highly infectious STIs such as syphilis, chlamydia, and gonorrhea, 3 or more partners would need to be assessed in detail to reduce the risk estimation error to a minimally acceptable 5% level, and achieving a 1% error would require collecting detailed information on 6–10 partners.

Prioritizing partners by assessing in detail those partners with whom the participant had the most unprotected sex marginally improved accuracy relative to prioritizing by the total number of sex acts with the partner or—and to a much greater extent—the most recent partner. If partners could be prioritized by actual risk, this would further improve risk estimate accuracy, but not appreciably. The latter result indicates that prioritization by the total number of sex acts or by the number of unprotected sex acts are nearly optimal methods for determining which partners—if any—should be assessed in detail. Thus, when assessing individual partners in detail, study participants should be asked first about the partner they had sex with most frequently, then other partners in order of decreasing frequency of (risky) sexual activity.

In short, the results of the current analysis suggest that HIV risk can be reasonably well characterized using aggregate sexual behavior assessment techniques or by supplementing an aggregate assessment with detailed information on a single "main" partner. Accurately characterizing STI risk requires more extensive partner-by-partner assessment efforts. In the combined sample, a completely aggregate assessment produced an unacceptably high 42.7% STI error rate. Aggregate data collection collapses data across partners, thereby sacrificing information about the distribution of sex acts among partners. This information is especially critical for accurate risk estimation for highly infectious STIs but much less important for estimating HIV risk (Pinkerton et al. 2002).

There are several limitations to this study. First, the risk estimates obtained from the complete partner-by-partner assessment were adopted as a “gold standard” (i.e., were assumed veridical). The actual risk of HIV or STI acquisition depends on multiple factors such as individual susceptibility, partner infectivity, partner selection patterns, and so on (Aral 1993; Berry, Raymond, and McFarland 2007). However, the models used in the current analysis are typical of those used in risk estimation studies; the accuracy of such models is optimized when all partners are individually assessed in detail. Second, and again consistent with accepted modeling techniques, the likelihood that any particular partner was, in fact, HIV- or STI-infected was assumed to be the same for all partners. The likelihood of infection could depend on the type of partner (e.g., “main” vs. “casual”), his or her previous or concurrent sexual or drug injection history, or other factors. Third, although daily diaries reduce recall bias relative to retrospective data collection techniques, they do not necessarily produce veridical reports of participants’ sexual activities (McAuliffe, DiFranceisco, and Reed 2007). In the current study, the empirical sexual behavior data were used only to illustrate the differential sensitivity of HIV and STI risk estimates to the level of data specificity. As such, slight inaccuracies in the daily diary reports would not be expected to affect the main findings of the study. Fourth, the results pertain to a particular convenience sample of women and men who agreed to participate in a 3-month-long daily sexual behavior diary study (McAuliffe, DiFranceisco, and Reed 2007). Generalization to other samples is supported by theoretical considerations (see “Linearity and Nonlinearity” above) but cannot be assured based on the findings of a single study. Fifth, the aggregate and partner-by-partner data were obtained by “collapsing” the diary data across partners. The estimation errors listed in the tables do not take into account recall/reporting errors. Partner-by-partner assessment strategies have been found to be more reliable than aggregate assessments of frequencies of sexual risk behavior (McAuliffe, DiFranceisco, and Reed 2007).

The results of this study indicate that relatively simple aggregate data collection and modeling techniques generally suffice to accurately characterize HIV risk but that accurate STI risk assessment requires more intensive partner-by-partner methods. They also suggest that when conducting partner-by-partner assessments, partners should be prioritized based on the number of sex acts or unprotected sex acts with each partner, not the recency with which the study participant had sex with the partner.

Every assessment involves trade-offs—for example, between maximizing reporting accuracy and minimizing response burden. There are relatively few studies of the accuracy of self-reported sexual risk behaviors (in part because there is no unobtrusive means to verify these very private behaviors) and even fewer studies of the response burden associated with different sexual behavior data collection techniques. Additional research is needed in both these areas.

Biographies

Steven D. Pinkerton, PhD, is a professor of psychiatry and behavioral medicine at the Medical College of Wisconsin’s Center for AIDS Intervention Research (CAIR). Dr. Pinkerton is a leading expert in the cost-effectiveness of HIV prevention interventions and is the director of CAIR’s cost-effectiveness studies core. He has published more than 125 peer-reviewed articles on cost-effectiveness analysis, mathematical modeling of HIV/STI transmission, sexual behavior assessment, transmission risk behaviors, and human sexuality.

Carol L. Galletly, JD, PhD, is an assistant professor of psychiatry and behavioral medicine at the Medical College of Wisconsin’s Center for AIDS Intervention Research (CAIR). Dr. Galletly is an attorney and social scientist whose research applies empirical methods to guide effective law and policy in highly controverted areas including human sexuality and HIV/AIDS. She is a leading expert on the impact of criminal HIV disclosure laws on persons living with HIV.

Timothy L. McAuliffe, PhD, is a professor of psychiatry and behavioral medicine and population health—epidemiology at the Medical College of Wisconsin. His interests are in study design, accuracy of self reports, and statistical analyses.

Wayne DiFranceisco, MA, is a senior research scientist in psychiatry and behavioral medicine at the Medical College of Wisconsin’s Center for AIDS Intervention Research (CAIR). He is a member of CAIR’s quantitative and behavioral methods core. He has published numerous peer-reviewed articles in the areas of HIV prevention intervention evaluation and methodological issues in HIV/AIDS and sexual behavior research.

H. Fisher Raymond, MPH, is a director of behavioral surveillance at the San Francisco department of public health. Mr. Raymond develops and implements population-based sampling methods among hidden and hard to reach populations. He also has been involved in the development and implementation of partner-by-partner measures of sexual mixing and HIV risk.

Harrell W. Chesson, PhD, is a health economist in the Division of STD Prevention at the Centers for Disease Control and Prevention. His research interests include modeling the effectiveness and cost-effectiveness of STD and HIV prevention programs and policies, alcohol and substance abuse and risky sexual behavior, and decision making under uncertainty.

Appendix

A simulated partner-by-partner data set was created from the diary data by collecting together, for each study participant, all sex acts with each individual partner. The HIV or STI acquisition risk associated with partner / then was calculated using the equation:

$$P_J=\mathrm{\pi }[1-{(1-{\mathrm{\alpha }}_1)}^{n1}{(1-(1-e){\mathrm{\alpha }}_1)}^{k1}{(1-{\mathrm{\alpha }}_2)}^{n2}{(1-(1-e){\mathrm{\alpha }}_2)}^{k2}],$$ (2)

where π is the presumed prevalence of HIV or STI infection among sex partners; α₁ and α₂ are the per-act transmission probabilities for receptive vaginal and anal intercourse (for HRW), insertive vaginal and anal intercourse (for HRM), or insertive and receptive anal intercourse (for MSM); e denotes condom effectiveness; and the ns and ks give the numbers of unprotected and condom-protected sex acts, respectively, with the partner (Pinkerton and Abramson 1993, 1996). Each study participant’s overall risk, across all of his or her sex partners, was estimated as

$$P(M)=1-(1-P_1)(1-P_2)\dots (1-P_M),$$ (3)

where M is the total number of sex partners reported by the participant. (For simplicity, the female partners of MSM and the male partners of HRM, if any, were excluded from the risk calculations.) This estimate was used as the “gold standard” to which other risk estimates—derived from less comprehensive data elicitation strategies—were compared.

Risk estimates for other simulated data collection strategies—for example, collecting detailed information about m sex partners and aggregate data for the remaining partners—were derived by including the m highest priority partners in the partner-by-partner portion of the simulated data set (Equation 3 with M set equal to m) and relegating the remaining R = M — m partners to the aggregate portion of the data set. The risk of HIV or STI acquisition for the latter (aggregate) group was estimated via the equation:

$$P_A=1-[(1-\mathrm{\pi })+\mathrm{\pi }{[1-{(1-{\mathrm{\alpha }}_1)}^{n1/R}{(1-(1-e){\mathrm{\alpha }}_1)}^{k1/R}{(1-{\mathrm{\alpha }}_2)}^{n2/R}{(1-(1-e){\mathrm{\alpha }}_2)}^{k2/R}]}^R,$$ (4)

where the ns and ks now represent the total numbers of unprotected and protected sex acts with the R partners included in the aggregate portion of the simulated data set. Of note, this model assumes that sex acts are evenly distributed among the R sex partners; this type of model often is used when limited information is available about the exact distribution of sex acts among sex partners (Pinkerton and Abramson 1998).

As described in the main text, for each simulated assessment strategy, the corresponding overall HIV or STI risk—including data from both the partner-by-partner and aggregate components of the simulated assessment—was estimated using Equation 1: P(m) = 1 — (1 — P_A)(1 — P₁)…(1 — P_M), where m is the number of individually assessed partners, P_k (k = 1 … m) is the risk associated with individually assessed partner k, and P_A is the combined risk for the partners assessed in aggregate (see Equation 4).