Abstract
IMPORTANCE
With the emphasis on structural-level interventions that target social determinants of human immunodeficiency virus (HIV) transmission to curb the HIV epidemic, there is a need to develop evaluation models that can detect changes in individual factors associated with HIV-related structural changes.
OBJECTIVE
To describe whether structural changes developed and achieved by community coalitions are associated with an effect on individual factors associated with the risk of contracting HIV.
DESIGN, SETTING, AND PARTICIPANTS
In this serial cross-sectional survey design, data were collected from 8 cities during 4 rounds of annual surveys from March 13, 2007, through July 29, 2010. Study recruitment took place at venues where the population of focus was known to congregate, such as clubs, bars, community centers, and low-income housing. The convenience sample of at-risk youth (persons aged 12–24 years) included 5337 individuals approached about the survey and 3142 (58.9%) who were screened for eligibility. Of the 2607 eligible participants, 2559 (98.2%) ultimately agreed to participate.
INTERVENTIONS
Achievement of locally identified structural changes that targeted public and private entities (eg, federal agencies, homeless shelters, and school systems) with the goal of fostering changes in policy and practice to ultimately facilitate positive behavioral changes aimed at preventing HIV.
MAIN OUTCOMES AND MEASURES
Number of sexual partners, partner characteristics, condom use, and history of sexually transmitted infections and HIV testing.
RESULTS
Exposure to structural changes was not statistically significantly associated with any of the outcome measures, although some results were in the direction of a positive structural change effect (eg, a 10-unit increase in a structural change score had an odds ratio of 0.88 [95%CI, 0.76–1.03; P = .11] for having an older sexual partner and an odds ratio of 0.91 [95% CI, 0.60–1.39; P= .39] for using a condom half the time or less with a casual partner).
CONCLUSIONS AND RELEVANCE
This study evaluated a broad representation of at-risk individuals and assessed the effect of numerous structural changes related to various HIV risk factors. No structural changes as measured in this study were associated with a statistically significant reduction in risk behaviors. These null findings underscore the need for a long-term approach in evaluating structural interventions and the development of more nuanced methods of quantifying and comparing structural-change initiatives and determining the appropriate strategies for evaluating effect.
Traditional human immunodeficiency virus (HIV) prevention efforts have targeted individual-level factors, including knowledge, attitudes, beliefs, and behaviors, through the use of risk-reduction counseling programs and interventions. During the past decade, public health researchers and practitioners have begun to focus on social determinants of health and structural interventions that are designed to alter the context in which individuals engage in risk behaviors.1–4 Structural interventions target public and private entities (eg, federal agencies, private homeless shelters, and public school systems), with the goal of fostering changes in policy and practice that will facilitate positive behavioral changes in individuals.5 Despite the increasing popularity of structural change interventions to health promotion and HIV prevention, there is a lack of rigorous studies that assesses the effect of structural changes on individual-level behavioral changes.6
The objective of this study is to determine whether structural changes achieved through a project called Connect to Protect (C2P): Partnerships for Youth Prevention Interventions are associated with an effect on individual risk factors associated with HIV acquisition and transmission. Connect to Protect is a 10-year, community-mobilization, HIV-intervention project that was developed and implemented by the Adolescent Medicine Trials Network for HIV/AIDS Interventions (ATN).The ATN is a collaborative research network established in 2001 by the National Institutes of Health to implement clinical, biomedical, and behavioral research on youth (persons aged 12–24 years) who have or are at risk for HIV infection.
The C2P project has established coalitions in urban communities with high concentrations of youth who are at risk for HIV infection. The coalitions were tasked with developing and achieving structural change objectives (SCOs) (ie, new or modified policies, programs, and practices) that address local risk factors associated with HIV.7 Rather than provide coalitions with a menu of interventions from which to select, coalitions were given strategic planning materials and a logic model to assist them in developing their own localized structural changes.8 We chose this approach to maximize the proposed structural changes’ relevance and acceptance by the community. The ultimate goal of these coalitions is a decrease in HIV infection rates among youth in targeted communities of risk. Examples of structural changes developed by C2P coalitions include extending neighborhood clinic hours to accommodate more youth, modifying physical structures so that high-risk behaviors are discouraged (eg, install lighting in public areas where risky behaviors are known to occur), and addressing regulatory barriers, such as a requirement for parental consent to access reproductive or mental health services. Prior articles7–9 by the C2P investigators have characterized the nature of the coalitions, strategies used in building and maintaining them, and factors that contributed to successful implementation.
Methods
Study Design
The study design is a serial cross-sectional survey.10 We collected data through 4 anonymous cross-sectional surveys of youth given once per year at public venues in each site from March 13, 2007, to July 29, 2010. We chose not to enroll a cohort to maintain anonymity of participants, thereby maximizing participation and reporting of sensitive information and eliminating attrition bias. Each participating site’s institutional review board reviewed and approved this study. Oral consent was obtained from all study participants.
Study Population and Recruitment Procedures
Eight research sites across the United States and Puerto Rico participated in the surveys: Tampa, Los Angeles, Washington, DC, Chicago, San Juan, New York City, San Francisco, and Baltimore. Surveys were given via audio computer-assisted self-interview technology, and no personal identifiers were collected. The surveys measured multiple constructs, including perceived community resources and support, risk behaviors, HIV stigma, substance use, and depression (see eAppendix in the Supplement for a description of risk-related outcome variables).
Each site’s coalition identified a population of focus for HIV prevention interventions. Five sites (Los Angeles, Washington, DC, New York City, San Francisco, and Baltimore) prioritized males who have sex with males; 2 sites (Tampa and Chicago) prioritized females of color who have sex with males; and the San Juan site prioritized individuals who abuse substances (including, but not exclusively, those who use intravenous drugs), regardless of sex or sexual behaviors. All sites focused on low-income urban neighborhoods with high rates of sexually transmitted infections (STIs) among youth.
Recruitment for the surveys took place in person at venues identified through mapping efforts and qualitative research by the coalitions, where the population of focus was known to congregate (eg, clubs, bars, community centers, or low-income housing). Chutuape et al11 describe in detail the venue identification process, which was repeated for all 4 survey rounds. Depending on the results of this process, venues could be used in 1 or more of the survey rounds. At each venue, study staff visited at varying times to screen and recruit participants until the desired sample size per round was reached. Venue type varied across sites, while screening recruitment procedures were standardized across sites and survey rounds. Eligible respondents reported being between 12 and 24 years of age (inclusive), had a demographic and sexual orientation and experience profile that reflected the site’s population of focus, and reported having engaged in any type of consensual sexual activity during the past 12 months.
Outcome Variables
Risk-related factors, which often serve as independent variables, are the dependent (ie, outcome) variables for the current analyses. Outcome variable selection was guided by the C2P logic model, which describes a variety of factors that mediate and influence HIV risk.7 These risk-related factors were grouped into 5 categories or risk areas: (1) partner characteristics, including questions regarding sex with intravenous drug users, young males who have sex with males, HIV-positive partners, and older partners; (2) number of sexual partners reported during the past 3 months; (3) condom use, including frequency and use of condoms at most recent sexual encounter with both main and casual partners; (4) STIs, including whether respondents had an STI during the past year and whether they sought care from a licensed health care professional; and (5) HIV testing in the past year. The full list of risk-related variables used in analysis is shown inTable 1 and Table 2 and described in detail in the eAppendix in the Supplement. All risk factors were analyzed as dichotomous (yes or no) variables.
Table 1.
Participants Who Reported Risk Behaviors by SCO Score Tertile From the Connect to Protect Project, 2007–2010
| Total | No. (%)a | P Value | |||
|---|---|---|---|---|---|
| Lowest SCO Tertile (n = 732) |
Middle SCO Tertile (n = 910) |
Highest SCO Tertile (n = 750) |
Total | ||
| Risk area 1: partner characteristics | |||||
| Had sex with IDU in past 3 mo (n = 2255) | 36 (5.2) | 32 (3.7) | 40 (5.6) | 108 (4.8) | .69 |
| Had sex with MSM in past 3 mo (n = 2284) | 342 (48.2) | 241 (27.6) | 434 (60.7) | 1017 (44.3) | <.001 |
| Had sex with an HIV+ partner in the past 3 mo (n = 2268) | 50 (7.1) | 33 (3.8) | 67 (9.5) | 150 (6.6) | .07 |
| Oldest sexual partner is >5 y older than respondent (n = 2141) | 157 (23.2) | 174 (21.5) | 161 (24.6) | 492 (23.0) | .56 |
| Oldest sexual partner is ≥25 y old (n = 2141) | 208 (30.7) | 264 (32.6) | 221 (33.7) | 693 (32.4) | .24 |
| Risk area 2: No. of sexual partners | |||||
| Respondent had >5 sexual partners in the past 3 mo (n = 2374) | 77 (10.5) | 53 (5.9) | 99 (13.1) | 229 (9.6) | .08 |
| Respondent had ≤1 sexual partner in the past 3 mo (n = 2374) | 336 (46.0) | 485 (53.7) | 316 (41.9) | 1137 (47.6) | .10 |
| (n = 870) | (n = 884) | (n = 638) | |||
| Risk area 3: condom use | |||||
| Used condom at past sexual encounter with a main partner (n = 1809)b | 328 (50.4) | 318 (45.2) | 232 (51.1) | 878 (48.5) | .98 |
| Used condom at past sexual encounter with a casual partner (n = 1272)c | 280 (56.7) | 250 (58.3) | 218 (62.5) | 748 (58.8) | .10 |
| Used condom every time with main partner (n = 1808)b | 261 (40.0) | 232 (33.0) | 175 (38.5) | 668 (36.9) | .42 |
| Used condom every time with casual partner (n = 1271)c | 230 (46.7) | 203 (47.1) | 180 (51.7) | 613 (48.2) | .18 |
| Used condom half the time or less with main partner (n = 1808)b | 285 (43.7) | 360 (51.3) | 187 (41.2) | 832 (46.0) | .68 |
| Used condom half the time or less with casual partner (n = 1271)c | 192 (39.0) | 157 (36.4) | 115 (33.0) | 464 (36.5) | .08 |
| Risk area 4: STI | |||||
| Diagnosed with STI in past year (n = 2393) | 90 (10.3) | 70 (7.9) | 93 (14.5) | 253 (10.6) | .02 |
| Sought care for STI in past year (n = 251) | 138 (75.0) | 106 (60.6) | 136 (73.9) | 380 (70.0) | .82 |
| Risk area 5: testing behaviors | |||||
| Tested for HIV in past year (n = 2334) | 537 (62.5) | 411 (47.8) | 462 (73.4) | 1410 (60.1) | <.001 |
Abbreviations: HIV, human immunodeficiency virus; IDU, intravenous drug use; MSM, males who have sex with males; SCO, structural change objective; STI, sexually transmitted infection.
Tertiles for risk areas 1 and 2 are based on the distribution of SCO scores for risk areas 1 and 2; tertiles for risk areas 3, 4, and 5 are based on the distribution of SCO scores for risk areas 3, 4, and 5.
Denominator only includes respondents who reported having a main sexual partner.
Denominator only includes respondents who reported having a casual sexual partner.
Table 2.
Multivariable Modeling for SCO Score From the Connect to Protect Project, 2007–2010a
| Risk Areas | OR (95% CI)b | P Value |
|---|---|---|
| Risk area 1: partner characteristics | ||
| Had sex with IDU in past 3 mo | 1.22 (0.88–1.67) | .23 |
| Had sex with MSM in past 3 mo | 0.95 (0.80–1.14) | .59 |
| Had sex with an HIV+ partner in past 3 mo | 1.05 (0.83–1.33) | .69 |
| Oldest sexual partner is >5 y older than respondent | 0.91 (0.78–1.07) | .25 |
| Oldest sexual partner is ≥25 y | 0.88 (0.76–1.03) | .11 |
| Risk area 2: No. of partners | ||
| Respondent had >5 sexual partners in the past 3 mo | 1.01 (0.83–1.23) | .91 |
| Respondent had ≤1 sexual partner in the past 3 mo | 0.97 (0.85–1.12) | .70 |
| Risk area 3: condom use c | ||
| Used condom at past sexual encounter with a main partner | 0.94 (0.79–1.13) | .52 |
| Used condom at past sexual encounter with a casual partner | 1.05 (0.85–1.30) | .63 |
| Used condom every time with main partner | 0.86 (0.71–1.03) | .12 |
| Used condom half the time or less with main partner | 1.15 (0.95–1.39) | .16 |
| Used condom every time with casual partner | 1.03 (0.84–1.26) | .78 |
| Used condom half the time or less with casual partner | 0.91 (0.60–1.39) | .39 |
| Risk area 4: STI | ||
| Diagnosed with STI in past year | 1.09 (0.86–1.38) | .48 |
| Sought care appropriately for STI in past year | 0.97 (0.72–1.31) | .84 |
| Risk area 5: testing behaviors | ||
| Tested for HIV in the past year | 1.09 (0.92–1.31) | .37 |
Abbreviations: HIV, human immunodeficiency virus; IDU, intravenous drug user; MSM, males who have sex with males; OR, odds ratio; SCO, structural change objective; STI, sexually transmitted infection.
Each risk area variable is the dependent variable in a multivariable model that included the SCO score and significant covariates (eg, survey round, age, sex/sexual orientation, race or ethnicity, ever used intravenous drugs, and ever had an STI). Only the OR associated with the SCO score is shown. The full models are provided in the Supplement.
The OR for the SCO score is expressed per 10-unit change (eg, the odds of participants reporting a risk factor from a site and time point with an SCO score of 17 vs participants from a site and time point with an SCO score of 7).
The models for questions concerning condom use with a main partner include only the respondents who reported having a main partner. The models for questions concerning condom use with a casual partner include only those respondents who reported having a casual partner.
SCO Score
The C2P coalitions developed and implemented their own site-specific SCOs designed to address a wide range of behavioral risk factors. Details of the SCO development process have been described elsewhere.8 Each SCO was associated with 1 or more of the 5 risk areas described above and was assigned a point value. Drawing on extensive data provided by the sites, SCOs were assigned 1 point for lower-effect “traditional” SCOs and 2 points for the higher-effect SCOs. Lower-effect SCOs were thought to have less influence on risk behaviors and included sharing information, such as distribution in school health offices of pamphlets that list HIV testing sites. Higher-effect SCOs had potential for greater intensity because they targeted structures and systems (eg, community-based organizations and government agencies) and addressed barriers to care and services or altered the risk environment to promote defaulting to healthier choices. Examples of higher-effect SCOs included modifying service delivery by changing school policy to allow HIV testing in school health offices and changing consent procedures to allow minors to provide consent for confidential HIV testing (see eTable 1 in the Supplement for examples of SCOs).7
We calculated a cumulative SCO score for each of the 4 annual survey rounds at each site based on the number and point value of SCOs completed before or during the round of the survey. For example, if 3 high-effect (2-point) SCOs and 2 low-effect (1-point) SCOs related to partner selection were completed at site 1 before the end of data collection during year 2, the total SCO score assigned to participants who completed surveys at site 1 during year 2 would equal 8. Scores were cumulative; the SCO score for year 3 at a given site would include both the SCOs counted toward the year 2 score and any new SCOs completed during year 3. Thus, we computed an SCO score for each risk area that corresponded to a particular site and survey round. To account for a potential lag time for the effect of a new SCO to be felt, we also conducted an analysis that assigned the SCO score from the previous survey round to each respondent. All respondents who completed surveys during year 1 were assigned an SCO score of zero, while respondents who completed surveys in year 2 were assigned the SCO score calculated for year 1, and so on.
Each survey respondent was assigned the SCO score for each risk area corresponding to their site and survey round. All respondents recruited at that site and round were considered to have the same SCO exposure. This SCO score served as the respondent’s exposure measure (ie, independent variable) for evaluating whether the structural change exposure was associated with an HIV-related risk behavior (ie, dependent variable).
Other Covariates
While the focus of the analysis was on the relationship between the SCO score and outcome variables (described in the next section), other covariates were examined to control for potential confounding factors. In addition, these potential confounders were examined for their independent associations with the outcome variables. The key demographic variables of interest included age, race or ethnicity, current sex, and sexual orientation. To evaluate and control for potential temporal trends, study year (ie, years 1–4) was included in multivariable models. We also included history of STI (except in the cases in which an STI-related variable was the outcome being modeled) and history of intravenous drug use. A derived variable (described in detail in eTable 2 in the Supplement) was created based on sex at birth, current sex, and sexual orientation that classified participants as females, young males who have sex with males, and males who have sex with females.
Statistical Analysis
Univariate and bivariate frequencies were produced for the demographic and risk factor variables. No adjustments were made for multiple comparisons. Multivariable analyses were conducted using generalized linear mixed models (SAS PROC GLIMMIX procedure; SAS Institute, Inc) to examine the association between reported HIV risk-related behaviors (dependent variable) and respondents’ SCO scores (independent variable) while controlling for potential confounders. The study site was designated as a class variable to account for the inherent correlation created by selecting participants within sites. All covariates were included in initial models with SCO scores and were removed 1 by 1 (in order of descending significance) until all remaining covariates were significant at an overall level of P < .10. The SCO scores were included in all models regardless of statistical significance. All analyses were conducted using SAS, version 9.2 (SAS Institute, Inc).
Results
Study Population
During the 4 survey rounds, 5337 youth were approached about participating in the survey and 3142 (58.9%) completed the screening process. Of those screened, 2607 (83.0%) were eligible to participate in the survey and 2559 (98.2%) ultimately consented to participate. There were 167 individuals excluded from the analyses owing to the following factors: being outside the eligible age range (n = 42), not reporting voluntary sex in the past year (n = 110), and reporting no sexual partners in the previous year (n = 15). The remaining 2392 participants were included in the analyses (Figure). Approximately 8.6 of respondents in rounds 2 through 4 indicated they had participated in a previous round of data collection. A profile of selected demographic characteristics of the study population is shown in Table 3.
Figure 1. Derivation of Analysis Sample.
The Figure depicts the number of participants who met the criteria for these analysis. ACASI indicates audio computer-assisted self-interview.
Table 3.
Demographic Characteristics of Study Population From the Connect to Protect Project, 2007–2010
| Characteristic | Respondents (n = 2392)a |
|---|---|
| Research site | |
| Tampa, FL | 152 (6.4) |
| Los Angeles, CA | 234 (9.8) |
| Washington, DC | 219 (9.2) |
| Chicago, IL | 250 (10.5) |
| San Juan, PR | 688 (28.8) |
| New York, NY | 276 (11.5) |
| San Francisco, CA | 346 (14.5) |
| Baltimore, MD | 227 (9.5) |
| Total | 2392 (100) |
| Current sex | |
| Male | 1574 (65.9) |
| Female | 741 (31.0) |
| Transgender MTF | 57 (2.4) |
| Transgender FTM | 15 (0.6) |
| Rather not answer | 2 (0.1) |
| Missing | 3 |
| Sexual orientation | |
| Heterosexual | 823 (34.4) |
| Gay/lesbian | 1053 (44.0) |
| Bisexual | 435 (18.2) |
| Not sure/questioning | 67 (2.8) |
| Rather not answer | 14 (0.6) |
| Sex/sexual orientation | |
| Females | 759 (31.7) |
| Males who have sex with males | 1401 (58.6) |
| Males who have sex with females | 232 (9.7) |
| Age, mean (median) | 20.1 (20.0) |
| Race | |
| Asian/Pacific Islander | 78 (3.3) |
| Black/African American | 974 (40.8) |
| Native American/Alaskan native | 70 (2.9) |
| White | 286 (12.0) |
| Mixed race | 701 (29.3) |
| Other | 254 (10.6) |
| Rather not answer | 26 (1.1) |
| Missing | 3 |
| Ethnicity | |
| Hispanic/Latino | 1244 (52.1) |
| Non-Hispanic/Latino | 1140 (47.7) |
| Rather not answer | 5 (0.2) |
| Missing | 3 |
| Race/ethnicity | |
| Hispanic | 1244 (52.0) |
| Black, non-Hispanic | 855 (35.7) |
| White, non-Hispanic | 91 (3.8) |
| Other | 202 (8.4) |
| Highest grade completed | |
| ≤Eighth grade | 143 (6.0) |
| >Eighth grade but did not complete high school | 412 (17.3) |
| High school graduate | 771 (32.3) |
| GED | 156 (6.5) |
| Some college/technical education | 605 (25.3) |
| Technical school graduate | 69 (2.9) |
| College graduate | 153 (6.4) |
| Graduate school, no degree | 38 (1.6) |
| Graduate school, degree | 21 (0.9) |
| Rather not answer | 21 (0.9) |
| Missing | 3 |
| Ever homeless | |
| Yes | 578 (24.2) |
| No | 1811 (75.8) |
| Rather not answer | 1 (0.04) |
| Missing | 2 |
| Ever had an STI | |
| Yes | 542 (22.7) |
| No | 1843 (77.1) |
| Rather not answer | 6 (0.3) |
| Missing | 1 |
| Ever used intravenous drugs | |
| Yes | 48 (2.2) |
| No | 2140 (97.2) |
| Rather not answer | 13 (0.6) |
| Missing | 191 |
Abbreviations: FTM, female to male; GED, General Education Development; MTF, male to female; STI, sexually transmitted infection.
Values are presented as number (percentage) unless otherwise specified.
SCO Scores
Based on the study team’s review and coding of SCOs, all SCOs were coded as applicable to risk areas 1 (partner characteristics) and 2 (number of partners). A subset of these SCOs were also determined to be relevant to risk areas 3 (condom use), 4 (STI infection), and 5 (HIV testing). Because all SCOs were applicable to risk areas 1 and 2, the set of site-year-specific SCO scores for risk areas 1 and 2 were the same. Likewise, the set of site-year-specific SCO scores for risk areas 3, 4, and 5 were the same because SCOs that were applicable to risk area 3 were also found to be applicable to risk areas 4 and 5. The distribution of SCO scores by site and study year is shown in Table 4. Scores ranged from zero at year 1 (indicating that a site had not yet achieved an SCO by the end of their year-1 round of surveys) to 33 at year 4.
Table 4.
Distribution of Structural Change Objective Scores From the Connect to Protect Project, 2007–2010
| Risk Area by Study Site | Score | ||||
|---|---|---|---|---|---|
| Year 1 | Year 2 | Year 3 | Year 4 | Total | |
| Risk areas 1 and 2a | |||||
| Tampa, FL | 6 | 10 | 17 | 20 | 53 |
| Los Angeles, CA | 6 | 12 | 18 | 33 | 69 |
| Washington, DC | 4 | 10 | 18 | 25 | 57 |
| Chicago, IL | 4 | 4 | 7 | 13 | 28 |
| San Juan, PR | 0 | 6 | 8 | 10 | 24 |
| New York, NY | 3 | 3 | 12 | 17 | 35 |
| San Francisco, CA | 2 | 3 | 6 | 19 | 30 |
| Baltimore, MD | 0 | 9 | 13 | 15 | 37 |
| Mean (all sites) | 3.1 | 7.1 | 12.4 | 19.0 | 42.6 |
| Risk areas 3, 4, and 5a | |||||
| Tampa, FL | 6 | 10 | 17 | 18 | 41 |
| Los Angeles, CA | 4 | 10 | 16 | 27 | 57 |
| Washington, DC | 4 | 10 | 16 | 19 | 49 |
| Chicago, IL | 4 | 4 | 5 | 11 | 24 |
| San Juan, PR | 0 | 6 | 8 | 10 | 24 |
| New York, NY | 3 | 3 | 12 | 17 | 35 |
| San Francisco, CA | 2 | 3 | 4 | 15 | 24 |
| Baltimore, MD | 0 | 5 | 9 | 11 | 25 |
| Mean (all sites) | 2.9 | 6.4 | 10.9 | 25.6 | 34.9 |
Risk areas were defined as 1, high-risk partnerships; 2, number of sexual partners; 3, condom use; 4, sexually transmitted disease coinfection; and 5, human immunodeficiency virus testing.
Outcome Variables
Table 1 shows the bivariate associations between SCO score and each risk factor. The SCO scores were divided into tertiles, and the percentage of respondents with each of the risk factors is shown in each of the SCO tertiles. In general, there were no strong relationships observed between SCO tertiles and risk factors. However, there was a trend of increasing condom use with casual partners associated with SCO tertiles. The percentage of participants who reported using condoms during their most recent sexual encounter with a casual partner was 56.7%, 58.3%, and 62.5% for the low, medium, and high SCO tertiles, respectively. Likewise, the percentages who reported using condoms during every sexual encounter with casual partners increased (46.7%, 47.1%, and 51.7%), and the percentages who reported using condoms half the time or less with casual partners decreased (39.0%, 36.4%, and 33.0%) with each increase in SCO tertile. These trends, however, were not statistically significant. Although our data are not shown, results based on using the 1-year lagged value for the SCO score produced similar results.
To adjust for potential confounders, generalized linear mixed models for predicting each risk factor (ie, outcome variable) are shown in Table 2. The complete models for each variable are given in eTable 3 in the Supplement. The SCO scores were included in all models; other covariates were entered into the model if they were at least marginally significant (ie, P < .10) to control for potential confounders. As previously discussed, our primary interest was in the relationship between SCO scores (our measurement of an individual’s exposure to structural change interventions in his or her community) and that individual’s reported risk-related behaviors. The SCO scores were not statistically significantly associated with any of the risk-related factors. For example, a 10-unit increase in an SCO score had an odds ratio of 0.88 (95% CI, 0.76–1.03; P = .11) for having an older sexual partner. While this result is in the direction of a positive SCO effect (ie, higher SCO score implies a lower likelihood of engaging in the risk behavior), it was not statistically significant. Likewise, the direction of the associations between the SCO score and the 3 variables related to condom use with casual partners indicated more frequent condom use with an increase in SCO score but not to a statistically significant degree.
Discussion
The objective of this study was to determine whether structural changes were effective in changing individual-level risk behaviors, leading to reduced risk of HIV acquisition and transmission. While there were some suggested associations between SCO scores and risk factors in the bivariate analysis (eg, condom use with casual partners), the outcome variables selected for this analysis were not statistically significantly associated (ie, P < .05) with structural changes in the multivariable analyses. These null findings highlight the challenges of evaluating interventions focused on making structural changes that span the spectrum of community sectors and target varying social-ecologic levels.
Compared with the types of randomized clinical trials that are the highest criterion in clinical research and evidence-based decision making, evaluation of structural change interventions are limited by several factors, including accounting for external changes in policies and practices, identification and assignment of appropriate comparison groups, and the inability to administer precise or uniform “doses.”12,13 Our measurement of both structural and individual variables may not have been precise enough to detect change. Our survey instrument and/or study design (ie, cross-sectional vs cohort) may have also failed to detect individuals’ change in risk behavior over time. An additional obstacle to demonstrating the effectiveness of structural change is the observation time frame14; in this case, data were collected during a 4-year intervention period. Because many SCOs targeted more distal factors along the causal pathway (such as addressing issues of housing or safe spaces for at-risk youth) rather than focusing directly on the risk-related behaviors themselves (eg, condom use, number of partners), we expect that additional follow-up time will be needed to determine whether there is an effect on youths’ behavioral change. There are also well-documented problems with relying on self-report of sexual behavior, which is this study’s primary outcome measure.14 It is also possible that the structural changes chosen and implemented by the communities were not the appropriate changes to affect the pertinent individual risk behaviors.
This study design had a number of strengths. Rather than providing communities with predefined structural interventions, C2P coalitions used a strategic planning process to identify changes that target the issues most relevant to HIV risk in their communities.7,8 A serial cross-sectional design allowed us to examine changes over time.10 Multiple sites with similar community context and common baseline data allowed for cross-site comparison and development of a statistical comparison group. Assessment of SCOs and development of an SCO score accounted for variations in the amount and intensity of structural change in the different communities. Rather than a simple comparison of before vs after intervention, SCOs were developed and implemented incrementally during the 4-year study period. Our measure of the amount of exposure to the intervention attempted to account for this varying level of exposure over time.
Conclusions
The evaluation field has had only modest success in producing strong evidence linking structural changes to improvements in population-level outcomes across a variety of health issues.13 Most structural-level interventions for HIV have targeted behaviors that are proximal to HIV risk (eg, clean needles and condom policies) and that reach a narrow population (eg, substance users and commercial sex workers).2 There are few examples of effect evaluations done on a project of this size and scope: a multisite, multiyear, community-level intervention encompassing numerous completed SCOs derived from the communities themselves and related to various areas associated with HIV risk. However, this study highlights the challenges inherent in quantifying the effectiveness of such a project.
Supplementary Material
Acknowledgments
Funding/Support: This work was supported by grants U01 HD 040533 and U01 HD 040474 from The Adolescent Medicine Trials Network for HIV/AIDS Interventions from the National Institutes of Health through the National Institute of Child Health and Human Development (Dr Kapogiannis), with supplemental funding from the National Institutes on Drug Abuse and Mental Health.
Role of the Funder/Sponsor: The National Institute of Child Health and Human Development medical officer provided advice on the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, and approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank and acknowledge the contribution of the investigators and staff at the following ATN sites that participated in the C2P project: Ana Puga, MD, Jessica Roy, MSW, and Jamie Blood, MSW, Children’s Diagnostic and Treatment Center, Ft Lauderdale, Florida; Marvin Belzer, MD, Miguel Martinez, MSW/MPH, Veronica Montenegro, and Julia Dudek, MPH, Childrens Hospital of Los Angeles, Los Angeles, California; Lisa Henry-Reid, MD, Jaime Martinez, MD, Ciuinal Lewis, MS, and Antionette McFadden, BA, John H. Stroger, Jr Hospital of Cook County and the CORE Center, Chicago, Illinois; Lawrence D’Angelo, MD, William Barnes, PhD, and Stephanie Stines, MPH, Children’s Hospital National Medical Center, Washington, DC; Donna Futterman, MD, Michelle Lyle, MPH, and Bianca Lopez, MPH, Montefiore Medical Center, New York, New York; Linda Levin-Carmine, MD, Meg Jones, MPH, and Michael Camacho, BA, Mount Sinai Medical Center, New York, New York; Sue Ellen Abdalian, MD, and Sybil Schroeder, PhD, Tulane University Health Sciences Center, New Orleans, Louisiana; Ligia Peralta, MD, Bethany Griffin-Deeds, PhD, and Kalima Young, BA, University of Maryland, Baltimore; Lawrence Friedman, MD, and Kenia Sanchez, MSW, University of Miami School of Medicine, Miami, Florida; Bret Rudy, MD, Marne Castillo, PhD, Alison Lin, MPH, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; Irma Febo, MD, and Carmen Rivera, RN, MPH, University of Puerto Rico, San Juan; Barbara Moscicki, MD, Johanna Breyer, MSW, and Kevin Sniecinski, MPH, University of California at San Francisco; and Patricia Emmanuel, MD, Amanda Schall, MA, and Rachel Stewart-Campbell, BA, University of South Florida, Tampa. Craig Wilson, MD, and Cindy Partlow, MEd, ATN Coordinating Center, The University of Alabama at Birmingham, provided network, scientific, and logistical support. James Korelitz, PhD, Barbara Driver, MS, Richard Mitchell, BS, Marie Alexander, BS, and Dina Monte, BS, ATN Data and Operations Center, Westat, Inc, Rockville, Maryland, provided network operations and analytic support. The National Coordinating Center at Johns Hopkins University, Baltimore, provided technical assistance, training, and protocol support. The authors also acknowledge the ATN Community Advisory Board and the participants who contributed to this study. Study participants received $25.
Footnotes
Author Contributions: Dr Ellen had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Ellen, Willard, Korelitz, Kapogiannis, Harper, Friedman.
Acquisition, analysis, or interpretation of data: Ellen, Greenberg, Willard, Korelitz, Monte, Boyer, Harper, Henry-Reid, Gonin.
Drafting of the manuscript: Ellen, Greenberg, Willard, Korelitz.
Critical revision of the manuscript for important intellectual content: Ellen, Greenberg, Korelitz, Kapogiannis, Monte, Boyer, Harper, Henry-Reid, Friedman, Gonin.
Statistical analysis: Greenberg, Korelitz, Gonin.
Obtained funding: Ellen, Harper.
Administrative, technical, or material support: Ellen, Willard, Korelitz, Kapogiannis, Monte, Boyer, Harper, Friedman.
Study supervision: Ellen, Willard, Korelitz, Kapogiannis, Friedman.
Conflict of Interest Disclosures: None reported.
Contributor Information
Jonathan M. Ellen, Department of Pediatrics, School of Medicine, All Children’s Hospital, Johns Hopkins Medicine, St Petersburg, Florida.
Lauren Greenberg, Health Studies Sector, Westat, Rockville, Maryland.
Nancy Willard, Department of Pediatrics, Johns Hopkins University School of Medicine, Baltimore, Maryland.
James Korelitz, Health Studies Sector, Westat, Rockville, Maryland.
Bill G. Kapogiannis, National Institute of Child Health and Human Development, National Institutes of Health, Bethesda, Maryland.
Dina Monte, Health Studies Sector, Westat, Rockville, Maryland.
Cherrie B. Boyer, Division of Adolescent Medicine, Department of Pediatrics, University of California, San Francisco.
Gary W. Harper, Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor.
Lisa M. Henry-Reid, Department of Pediatrics, John H. Stroger Jr Hospital of Cook County, Chicago, Illinois.
Lawrence B. Friedman, Division of Adolescent Medicine, Department of Pediatrics, University of Miami Miller School of Medicine, Miami, Florida.
René Gonin, Health Studies Sector, Westat, Rockville, Maryland.
REFERENCES
- 1.Gupta GR, Parkhurst JO, Ogden JA, Aggleton P, Mahal A. Structural approaches to HIV prevention. Lancet. 2008;372(9640):764–775. doi: 10.1016/S0140-6736(08)60887-9. [DOI] [PubMed] [Google Scholar]
- 2.Golden RE, Collins CB, Cunningham SD, Newman EN, Card JJ. Best Evidence Structural Interventions for HIV Prevention. New York, NY: Springer Science and Business Media; 2013. [Google Scholar]
- 3.Auerbach J. Transforming social structures and environments to help in HIV prevention. Health Aff (Millwood) 2009;28(6):1655–1665. doi: 10.1377/hlthaff.28.6.1655. [DOI] [PubMed] [Google Scholar]
- 4.Nash D, Wu Y, Elul B, Hoos D, El Sadr W International Center for AIDS Care and Treatment Programs. Program-level and contextual-level determinants of low-median CD4+ cell count in cohorts of persons initiating ART in eight sub-Saharan African countries. AIDS. 2011;25(12):1523–1533. doi: 10.1097/QAD.0b013e32834811b2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Blankenship KM, Bray SJ, Merson MH. Structural interventions in public health. AIDS. 2000;14(suppl 1):S11–S21. doi: 10.1097/00002030-200006001-00003. [DOI] [PubMed] [Google Scholar]
- 6.Latkin C, Weeks MR, Glasman L, Galletly C, Albarracin D. A dynamic social systems model for considering structural factors in HIV prevention and detection. AIDS Behav. 2010;14(suppl 2):222–238. doi: 10.1007/s10461-010-9804-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ziff MA, Harper GW, Chutuape KS, et al. Adolescent Medicine Trials Network for HIV/AIDS Intervention. Laying the foundation for Connect to Protect: a multi-site community mobilization intervention to reduce HIV/AIDS incidence and prevalence among urban youth. J Urban Health. 2006;83(3):506–522. doi: 10.1007/s11524-006-9036-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Willard N, Chutuape K, Stines S, Ellen JM Adolescent Medicine Trials Network for HIV/AIDS Interventions. Bridging the gap between individual-level risk for HIV and structural determinants: using root cause analysis in strategic planning. J Prev Interv Community. 2012;40(2):103–117. doi: 10.1080/10852352.2012.660122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Deeds BG, Peralta L, Willard N, Ellen J, Straub DM, Castor J. The role of community resource assessments in the development of 15 adolescent health community-researcher partnerships. Prog Community Health Partnersh. 2008;2(Adolescent Medicine Trials Network for HIV/AIDS Interventions)(1):31–39. doi: 10.1353/cpr.2008.0010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Biglan A, Ary D, Wagenaar AC. The value of interrupted time-series experiments for community intervention research. Prev Sci. 2000;1(1):31–49. doi: 10.1023/a:1010024016308. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Chutuape KS, Ziff M, Auerswald C, Castillo M, McFadden A, Ellen J Adolescent Medicine Trials Network for HIV/AIDS Intervention. Examining differences in types and location of recruitment venues for young males and females from urban neighborhoods: findings from a multi-site HIV prevention study. J Urban Health. 2009;86(1):31–42. doi: 10.1007/s11524-008-9329-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Bonell C, Hargreaves J, Strange V, Pronyk P, Porter J. Should structural interventions be evaluated using RCTs? the case of HIV prevention. Soc Sci Med. 2006;63(5):1135–1142. doi: 10.1016/j.socscimed.2006.03.026. [DOI] [PubMed] [Google Scholar]
- 13.Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annu Rev Public Health. 2000;21:369–402. doi: 10.1146/annurev.publhealth.21.1.369. [DOI] [PubMed] [Google Scholar]
- 14.Hayes R, Kapiga S, Padian N, McCormack S, Wasserheit J. HIV prevention research: taking stock and the way forward. AIDS. 2010;24(suppl 4):S81–S92. doi: 10.1097/01.aids.0000390710.04255.2b. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.