Abstract
Social desirability bias and underreporting of HIV risk behaviors are significant challenges to the accurate evaluation of HIV prevention programs for orphans and vulnerable children (OVC) in sub-Saharan Africa. Valid and reliable HIV risk behavior instruments are critical to address these challenges. We assessed the psychometric properties of two risk behavior measures, the World Aids Foundation Survey (WAF) and the Peer HIV Risk Behavior Screener (PHRBS), administered to 210 OVC in Zambia using Audio Computer Assisted Self-Interviewing. All WAF subscales exhibited good internal reliability (α>.80); only the Sexual Behavior Practices subscale strongly distinguished (P<.01) adolescents who engaged in HIV risk behaviors (‘cases’) from those who did not (‘non-cases’). An 8-item version of the PHRBS, refined using exploratory factor analysis, demonstrated good internal reliability (α=87), differentiated ‘cases’ from ‘non-cases’ (P<.01), and correlated strongly with the Sexual Behavior Practices subcale (r=.34, P<.01). Results suggest that report of peers’ sexual behaviors can serve as a proxy for OVCs’ own behavior in contexts where social desirability bias affects reporting.
Keywords: HIV/AIDS, orphans and vulnerable children, risky sexual behavior, peer behavior, assessment validation, sub-Saharan Africa
Introduction
Orphans and vulnerable children (OVC) affected by HIV are at an increased risk of engaging in risky sexual behaviors (Operario, Underhill, Chuong, & Cluver, 2011). Reducing risky sexual behaviors is therefore an important component of interventions aimed at preventing the infection and transmission of HIV among youth in sub-Saharan Africa (Scott-Sheldon, Walstrom, Harrison, Kalichman, & Carey, 2013; Thurman, Kidman, Carton, & Chiroro, 2016). Accurate evaluation of the effectiveness of such interventions requires valid and reliable measures of HIV risk behavior (DiClemente, 2016).
In a recent editorial, DiClemente (2016) identified several of the serious measurement challenges that a growing body of literature has suggested affects the assessment of adolescent risky sexual behaviors with self-report methodologies: recall, retrospective, and social desirability biases, misunderstanding of sexual behavior questions, and low prevalence of sexual behavior. Social desirability may be a particular challenge in countries like Zambia, in which populations have high levels of religiosity and discussion of sexual behavior is considered taboo (Cowan et al., 2002; Glynn et al., 2001; Plummer et al., 2004).
One strategy to improve HIV risk behavior data quality is through administering surveys via Audio Computer Assisted Self-Interviewing (ACASI), which permits adolescents to respond to sensitive questions on a laptop computer with headphones instead of to an in-person interviewer. Increasingly, studies conducted in low- and middle-income countries (LMIC) have indicated that the use of ACASI is feasible and associated with lower rates for non-response and higher rates for reporting of sexual behavior compared to self-administered questionnaires and face-to-face interviews (Adebajo et al., 2014; Beauclair et al., 2013; Kane et al., 2016; Langhaug, Sherr, & Cowan, 2010).
A second, less utilized strategy for improving HIV risk behavior data quality involves the use of measurement scales (in place of or in addition to single item indicators) that cover a broad range of HIV risk behavior determinants, such as risk reduction intentions, readiness to change behaviors, and self-efficacy. Although the use of single item sexual behavior practice questions (e.g., use of condom at last sexual encounter) can result in substantial measurement error (Wang, Fisher, Siegal, Falck, & Carlson, 1995) and have low response rates due to social desirability bias or low prevalence of sexual activity (DiClemente, 2016), these indicators still predominate in the HIV risk behavior literature (Johnston et al., 2010; Qiao, Li, & Stanton, 2014; Tenkorang, Rajulton, & Maticka-Tyndale, 2009). Scales have the dual advantage of providing more statistical power than binary single item indicators in impact evaluations and by including a wider variety of items that more fully cover the many HIV risk behavior constructs. Among adolescents, in addition to the domains listed above, these constructs may also include peer HIV risk behavior. There is strong evidence to suggest that risky sexual behavior among youth is linked to friendships with peers who engage in such behaviors (e.g., early sexual debut) (Harrison et al., 2012; Magnani et al., 2002). Peer behavior may therefore be an appropriate gauge of HIV risk behavior in populations where self-reported sexual behavior is low due to social desirability or low prevalence.
In the present study, we aimed to assess the psychometric properties of two adolescent HIV risk behavior instruments using these two strategies, i.e.: 1) ACASI and 2) sub-scales with multiple HIV risk behavior constructs, among an OVC population in Zambia.
Methods
The current study was part of a larger NICHD-funded investigation to implement, adapt, and validate a comprehensive measure of HIV risk behaviors, mental health, well-being, and functioning among OVC in Zambia. These measures were subsequently used in a randomized controlled trial of an intervention to reduce HIV risk behaviors among OVC in Zambia (https://ClinicalTrials.gov identifier: NCT02054780).
Participants and Procedure
For this validation study, we collaborated with home-based care workers (HBCWs) operating in three communities in Lusaka, Zambia (Kane et al., 2016). The HBCWs had extensive connections throughout the communities, and their role was to visit families regularly to link them to services (e.g., nutrition education). We trained 68 HBCWs and asked them to inform the families under their care of the study using a brief recruitment script. HBCWs were asked to identify OVC (13–17 years old) who they believed would meet one or more of four possible inclusion criteria: 1) adolescents who were at significant HIV risk because of their behaviors; 2) adolescents who were at low or no HIV risk because of their behaviors; 3) adolescents who had significant psychosocial problems; and 4) adolescents who had no significant psychosocial problems. In the present analysis, we included only those adolescents who were identified as being at significant HIV risk because of their behaviors and those identified as not being at risk for HIV because of their behaviors (groups 1 and 2).
Eligibility for these two categories was ascertained by asking two screening questions to the adolescent, his/her caregiver, and the HBCW who recruited the family. The first question asked if the adolescent was at high risk of HIV because of his/her behaviors with a binary yes/no response option. The second question asked if the adolescent was at little or no risk of HIV because of his/her behaviors with a binary yes/no response option. An adolescent was classified as a high risk ‘case’ if a) the adolescent indicated “yes” that he/she had HIV risk behaviors or b) the caregiver and HBCW both reported “yes” that the adolescent had HIV risk behaviors. An adolescent was classified as a low risk ‘non-case’ if all three respondents indicated “yes” to the second question that the adolescent was at little or no risk.
Measures
Eligible adolescents completed the full study assessment on a laptop-based ACASI (Tufts University School of Medicine, 2014). The full interview was translated from English to the two most commonly spoken local languages in Zambia, Nyanja and Bemba, back-translated, and reviewed for comprehension and cultural relevance in pilot groups. The assessment included several measures of mental and behavioral health (Kane, Murray, Bass, Johnson, & Bolton, 2016). This paper reports on the psychometric properties of two of these instruments, the World AIDS Foundation Survey and the Peer HIV Risk Behavior Screener.
World AIDS Foundation Survey (WAF)
The WAF includes a large range of items and subscales on sexual and behavioral health, substance use, and HIV risk (NIMH Multisite HIV Prevention Trial Group, 1998; Sikkema et al. 2004, 2005). In this study, we evaluated several of the pre-established WAF scales and single item indicators that are used to assess HIV risk and HIV risk behavior, described in Table 1.
Table 1.
Description of scales and single item indicators included in the WAF
| Scale/Item | Number of items |
Sample included |
Description |
|---|---|---|---|
| Self-Efficacy | 7 | All | Assesses how confident adolescents are in acting certain ways in hypothetical sexual situations (e.g., “how confident are you that you can talk with a sexual partner about condom use).” Response options are on a Likert- type scale ranging from 0 (very sure I can’t) to 3 (very sure I can). An average score is calculated across the seven items with higher scores indicative of greater self-efficacy. |
| Sexual Relations and Violence |
4 | All | Assesses how much adolescents agree with statements on beliefs about violence in relationships (e.g., “beating up a woman is a good sign that a man loves her”). Responses are on a Likert-type scale ranging from 0 (strongly disagree) to 3 (strongly agree). An average sexual relations and violence score is calculated across the four items with higher scores indicating stronger agreement that violence is acceptable in relationships. |
| Risk Reduction Intentions |
8 | Participants who ever had sex |
Assesses how empowered adolescents feel about the ability to perform risk reduction behaviors (e.g., “refusing to have sex with my current partner if I don’t want to”). Response options include: “I feel I am NOT strong enough to do this” (0) and “I feel I AM strong enough to do this (1).” A total risk reduction intentions score is calculated by summing responses with higher scores an indication of feeling more empowered. |
| Readiness to Change |
5 | Participants who ever had sex |
Assesses how strongly adolescents agree with statements such as “I am starting to think about the HIV risk from my behavior.” Response options are on a Likert-type scale from 0 (strongly disagree) to 3 (strongly agree). An average score is calculated with higher scores indicative of greater readiness to change. |
| Sexual Behavior Practices |
12 | Participants who ever had sex |
12-item section of the WAF that we summed to create a total score. Items ask participants to identify the number of sexual encounters and partners (lifetime and past 2 months), number of times they had coerced or forced sex, frequency of condom use and other prophylaxis, the number of sexual encounters following drug use, and the number of times the adolescent had sex in exchange for drugs, money, or a place to stay. A higher score represents greater risky sexual behaviors. Adolescents who never had sex are assigned a score of 0. |
| Based on your own behavior, how high is your own risk of getting HIV? |
1 | All | Response options range from 0 (no risk) to 4 (a great deal of risk). For this study, response categories were collapsed into a binary variable such that an original response of 2, 3, or 4 was categorized as 1 and a response of 0 or 1 was categorized as 0. |
| Do you suspect or think you may have HIV? |
1 | All | Yes (1)/No (0) |
| How concerned are you about getting HIV? |
1 | All | Response options range from 0 (not at all) to 3 (very much). For this study, response categories were collapsed into a binary variable such that an original response of 1, 2, or 3 was coded as 1 and a response of 0 was coded as 0. |
| I am fearful of talking about condom use |
1 | Participants who ever had sex |
Response options range from 0 (not at all) to 2 (very much). For this study, response categories were collapsed into a binary variable such that an original response of 1 or 2 was categorized as 1 and a response of 0 was coded as 0. |
| Have you ever had sex? |
1 | All | Yes (1)/No (0) |
| Have you ever had an STI? |
1 | Participants who ever had sex |
Yes (1)/No (0) |
| Have you ever been tested for HIV? |
1 | All | Yes (1)/No (0) |
| Has your partner ever been tested for HIV? |
1 | Participants who ever had sex |
Yes (1)/No (0) |
| Have you ever been taught about safe sex? |
1 | All | Yes (1)/No (0) |
Peer HIV Risk Behavior Screener
The Peer HIV Risk Behavior Screener (hereafter referred to as the PHRBS) is an 11-item measure that uses an indirect method of assessing HIV risk by using items that refer to behaviors of an adolescent’s peers. We reviewed the literature and chose items from the WAF and the Positive Outcomes for Orphans (POFO) measure that reflected significant risk factors for high risk behaviors among OVC populations (Sikkema et al., 2004, 2005; Whetten et al., 2009). We then modified the items to use social referencing and refer to peer behavior instead of the adolescent’s own behavior. Adolescents were asked if “some of the friends they spend time with…” had certain characteristics or behaviors such as: “have had sexually transmitted diseases,” “have sex without a condom,” and “are orphans or do not live with an adult who looks after them well.” Response options were a binary yes (coded as 1)/no (coded as 0). Items were summed to create a total scale score.
Data analysis
Analyses were conducted with Stata, version 13 (StataCorp, 2013). We first conducted an exploratory factor analysis (EFA) to examine the factor structure and dimensionality of the PHRBS. For the resulting refined scale of the PHRBS and the pre-established WAF scales, we calculated Cronbach’s alpha (Cicchetti, 1994; Nunnally & Bernstein, 1994).
Discriminant validity of all scales was then examined using a method described by Bolton (2001) for LMIC settings where a gold standard criterion is not available (Bolton, 2001). We analyzed differences in mean scale scores between adolescent HIV risk behavior “cases” -- adolescents categorized as having high risk of HIV risk behaviors based on screening responses of adolescent, caregiver, and HBCW -- and “non-cases”-- adolescents categorized as having low risk of HIV risk behaviors. We believed this method was applicable for HIV risk behavior assessment given research that has shown observation of adolescents by a reliable third party (in this case, the caregiver and/or HBCW) is a useful method for identifying adolescents with HIV risk behaviors (Plummer et al., 2004). We further explored the utility of scales that displayed significant differences between “cases” and “non-cases” by using receiver operating characteristic (ROC) curve analysis, including the area under the curve, optimal cut-off scores, and corresponding sensitivity and specificity values. Discriminant validity testing of single item indicators was conducted by comparing responses to each item between the HIV risk behavior “cases” and “non-cases” using chi-squared tests.
Results
The study sample included 179 adolescents, with approximately equal numbers classified during screening as being at high risk for HIV due to their behaviors (‘cases’, n=89) and not being at high HIV risk (‘non-cases’, n=90). Demographic characteristics are summarized in Table 2.
Table 2.
Characteristics of study sample
| HIV Risk Behaviors ‘Cases’ (N=89) |
No HIV Risk Behaviors ‘Non-cases’ (N=90) |
Total sample (n=179) |
Independent samples t-Test/χ2 |
|
|---|---|---|---|---|
| mean (SD) | ||||
| Age | 15.24 (1.49) | 14.51 (1.41) | 14.87 (1.49) | -3.34** |
| N (%) | ||||
| Female | 54 (60.7) | 47 (52.2) | 101 (56.4) | 1.3 |
| Currently in school | 59 (66.3) | 76 (84.4) | 135 (75.4) | 7.95** |
| Days absent from school (past 6 months) 0 days 1–4 days 5–10 days 11 or more days Not attended school in past 6 months |
20 (33.9) 22 (37.3) 7 (11.9) 6 (10.2) 4 (6.8) |
47 (61.8) 17 (22.4) 8 (10.5) 2 (2.6) 2 (2.6) |
67 (49.6) 39 (28.9) 15 (11.1) 8 (5.9) 6 (4.4) |
12.3* |
| Primary caretaker Mother/Father Someone else No one |
45 (50.6) 35 (39.3) 9 (10.1) |
56 (62.2) 31 (34.4) 3 (3.3) |
101 (56.4) 66 (36.9) 12 (6.7) |
4.4 |
| Orphan status Both mother and father alive Father alive only Mother alive only Neither mother nor father alive |
21 (23.6) 7 (7.9) 32 (36.0) 29 (32.6) |
15 (16.7) 5 (5.6) 42 (46.7) 28 (31.1) |
36 (20.1) 12 (6.7) 74 (41.3) 57 (31.8) |
2.70 |
| HIV status HIV positive HIV negative Don’t know |
14 (15.7) 74 (83.2) 1 (1.1) |
3 (3.3) 86 (95.6) 1 (1.1) |
17 (9.5) 160 (89.4) 2 (1.1) |
8.0* |
| Biological parent HIV status One or both HIV positive Neither HIV positive Don’t know |
18 (20.2) 71 (79.8) 0 (0.0) |
11 (12.2) 78 (86.7) 1 (1.1) |
29 (16.2) 149 (83.2) 1 (0.6) |
3.0 |
| Disability Yes |
26 (29.2) |
16 (17.8) |
42 (23.5) |
3.26 |
P<.05
P<.01
Exploratory Factor Analysis
The principal components analysis (PCA) on the PHRBS suggested a two-factor solution, with eigenvalues of 6.5 and 1.3, respectively. The two factors combined to explain approximately 70.8% of the total variance. Examination of the scree plot and a parallel analysis with 1,000 replications also suggested two factors. An initial exploratory factor analysis (EFA) specifying two factors suggested that one item be dropped due to high uniqueness: “some of the friends I know have HIV or AIDS” (0.52). A two-factor EFA with the remaining ten items indicated that no item had uniqueness above 0.42. Factor loadings for those ten items are displayed in Table 3. Eight items loaded clearly onto one factor related to peer risk behaviors and two items loaded clearly onto a second factor, which more broadly described peer OVC well-being. Only Factor 1 (the peer risk behavior items) was explored further in analysis.
Table 3.
Factor loadings and communalities with promax rotation for the Peer HIV Risk Behavior Screener (PHRBS)
| Factor 1: Peer risky sexual behavior |
Factor 2: Peer well-being |
Communality | |
|---|---|---|---|
| Some of the friends I spend time with… | |||
| Are orphans or do not live with an adult who looks after them well | -.12 | .94 | .80 |
| Are stigmatized | -.08 | .79 | .58 |
| Have been or are pregnant | .67 | .23 | .65 |
| Have had sexually transmitted diseases | .59 | .40 | .73 |
| Have sex | .99 | -.14 | .86 |
| Have sex without a condom | .89 | -.09 | .72 |
| Had sex by the time they were 11 or 12 years old | .81 | .09 | .73 |
| Have frequent, brief sexual relationships with the opposite gender | .62 | .34 | .71 |
| Have had multiple sexual partners | .84 | .07 | .77 |
| Drink alcohol or use drugs | .97 | -.17 | .82 |
Reliability and validity of scales
Table 4 presents descriptive statistics, Cronbach’s alpha coefficients, and discriminant validity results for the shortened 8-item PHRBS as well as for all WAF scales. Internal consistency of all scales was very good with Cronbach’s alpha values of 0.80 or above for each except for Sexual Relations and Violence (α=.58).
Table 4.
Descriptive statistics, internal reliability, and discriminant validity testing of PHRBS and WAF scales
| No. of items |
Range | Cronbach’s α |
Mean (SD) |
||||
|---|---|---|---|---|---|---|---|
| Overall | HIV Risk behaviors ‘Cases’ |
No HIV Risk behaviors ‘Non-cases’ |
t/za | ||||
| PHRBS (n=179) |
8 |
0–8 |
0.87 |
2.59 (2.68) |
3.69 (2.87) |
1.51 (2.01) |
-5.30** |
|
WAF Scales | |||||||
| Sexual behavior practices (n=179) |
12 | 0–22 | 0.81 | 1.08 (3.28) | 1.66 (4.08) | 0.5 (2.09) | -2.88** |
| Readiness to change (n=35) |
5 | 0–2.6 | 0.83 | 1.34 (0.90) | 1.42 (0.94) | 1.12 (0.78) | -0.88 |
| Self-efficacy (n=179) | 7 | 0–3 | 0.88 | 1.0 (0.84) | 1.10 (0.85) | 0.91 (0.82) | -1.53 |
| Risk reduction intentions (n=35) |
8 | 0–8 | 0.87 | 5.03 (2.88) | 4.96 (2.85) | 5.2 (3.12) | 0.17 |
| Sexual relations and violence (n=179) |
4 | 0–3 | 0.58 | 1.07 (0.64) | 1.08 (0.58) | 1.07 (0.70) | -0.12 |
P<.05
P<.01
Mann-Whitney U test (z statistic) for pairwise comparisons is analogous to a t-test for non-normally distributed data
Mean scores for the PHRBS and the WAF Sexual Behavior Practices scale were both significantly higher among HIV risk behavior cases compared to non-cases (P<.01 for both). The two scales were also highly correlated with each other (Table 5). There were no significant differences in mean scores between the two groups for the other WAF scales.
Table 5.
Correlations between scales
| (1) | (2) | (3) | (4) | (5) | (6) | |
|---|---|---|---|---|---|---|
| (1) PHRBS | 1 | |||||
| (2) Sexual Behavior Practices | 0.34** | 1 | ||||
| (3) Self Efficacy | 0.21** | 0.22** | 1 | |||
| (4) Sexual Relations and Violence | 0.06 | 0.12 | 0.20** | 1 | ||
| (5) Risk Reduction Intentions | 0.42* | 0.11 | 0.54** | 0.17 | 1 | |
| (6) Readiness to Change | 0.23 | 0.24 | 0.54** | 0.24 | 0.56** | 1 |
P<.05
P<.01
We conducted sensitivity and specificity analyses for the PHRBS and Sexual Behavior Practices (Table 6). The area under the curve (AUC) was 0.72 for the PHRBS and 0.59 for Sexual Behavior Practices scale. The lower bound 95% confidence interval for both AUC values was greater than 0.50 indicating that the scales had the ability to identify adolescents with HIV risk behaviors with a probability greater than random chance.
Table 6.
Test characteristics using receiver operating curves for the PHRBS and Sexual Behavior Practices scales
| Scale | AUC (95% CI) |
ROC Sens (%) | ROC Spec (%) | % Correctly classified | Optimal cut-off |
|---|---|---|---|---|---|
| PHRBS | 0.72 (0.65, 0.80) |
68.5 | 63.3 | 65.9 | 2 |
| Sexual Behavior Practices | 0.59 (0.53, 0.64) |
28.1 | 88.9 | 58.9 | 1 |
Validity of individual WAF items
Discriminant validity testing with single items from the WAF is displayed in Table 7. HIV risk behavior ‘cases’ were significantly more likely than ‘non-cases’ to report: ever having sex (28.1% vs. 11.1%, P<.01), a belief that their risk of HIV was higher based on their own behavior (34.2% vs. 19.5%, P<.05), a suspicion or thought that they may have HIV (20% vs. 2.9%, P<.01), a concern about contracting HIV (61.5% vs. 46%, P<.05), and ever having a sexually transmitted infection (9% vs. 0%, P<.01).
Table 7.
Discriminant validity testing of WAF single item indicators
| N (%) |
||||
|---|---|---|---|---|
| Overall | HIV Risk behaviors ‘Cases’ |
No HIV Risk behaviors ‘Non-cases’ |
χ2 | |
| Based on own behavior, have at least some risk of getting HIV (n=169) |
45 (26.6) | 28 (34.2) | 17 (19.5) | 4.61* |
| Suspect or think you may have HIV (n=124) | 13 (10.5) | 11 (20.0) | 2 (2.9) | 9.54** |
| Concern about getting HIV (“a little” or more) (n=169) | 91 (53.5) | 51 (61.5) | 40 (46.0) | 4.09* |
| Ever had sex (n=178) | 35 (19.6) | 25 (28.1) | 10 (11.1) | 8.2** |
| Ever taught about safe sex (n=179) | 80 (44.9) | 46 (51.7) | 34 (38.2) | 3.27 |
| Ever had an STI (n=179) | 8 (4.5) | 8 (9.0) | 0 (0.0) | 8.47** |
| Fear of talking about condom use (n=35) | 15 (42.9) | 10 (40.0) | 5 (50.0) | 0.29 |
| Ever HIV tested (n=179) | 77 (43.0) | 41 (46.1) | 36 (40.0) | 0.67 |
| Partner ever HIV tested (n=34) | 14 (41.2) | 10 (41.7) | 4 (40.0) | 0.01 |
P<.05
P<.01
Discussion
This study evaluated the validity and reliability of ACASI-administered HIV risk behavior assessments among a population of OVC in Zambia. To our knowledge, the use of a criterion for validity testing comprised of triangulating adolescent, caregiver, and community health worker reports is unique in sub-Saharan Africa, and draws on previous research suggesting that observation by a third party is a valid method for identifying adolescent HIV risk behaviors (Plummer et al., 2004). Our results suggest that two scales, the PHRBS and WAF Sexual Behavior Practices, as well as several of the individual WAF items can be reliable and validly used to assess HIV risk behaviors with this OVC population.
The PHRBS displayed strong internal reliability and the ability to discriminate between HIV risk behavior cases and non-cases. A growing body of literature has demonstrated that peer behavior, both actual and perceived, has a substantial impact on an adolescent’s likelihood of engaging in HIV risk behaviors, such as risky sex and substance use (Brook, Morojele, Zhang, & Brook, 2006; Brook, Morojele, Pahl, & Brook, 2006; Cherie & Berhane, 2012; Potard, Courtois, & Rusch, 2008). Our findings similarly suggest that measurement of peer behavior may be an appropriate proxy for evaluating an adolescent’s own HIV risk behavior. Indeed, there was a strong and significant correlation between the PHRBS and WAF Sexual Behavior Practices, which includes direct and sensitive questions on sexual behavior. In addition to correlating highly with the PHRBS, Sexual Behavior Practices also performed very well in discriminating cases and non-cases. ROC curve analyses suggested that both scales performed adequately in identifying those with and without HIV risk behaviors. Therefore, both scales are recommended for use in measuring HIV risk behaviors among OVC populations. The PHRBS, in particular, may be useful among populations in which reporting of sexual activity is low, due either to social desirability bias or a low prevalence of sexual activity.
The relatively poorer performance of two other WAF scales, Risk Reduction Intentions and Readiness to Change, underscores the importance of including risk behavior instruments that can be administered to adolescents regardless of their self-reported sexual history. Only adolescents who reported ever having had sex (20% of our sample) responded to items in these scales. Given that 50% of the sample was classified as having high HIV risk behaviors by our triangulation criterion method, these scales alone would not be sufficient in measuring HIV risk behavior in this OVC population. In populations where reported prevalence of sexual activity is low, oversampling of adolescents who are sexually active may be necessary in order to have the necessary statistical power for these instruments.
Our results do not support the use of the WAF Self Efficacy and WAF Sexual Relations and Violence scales in their current form for the specific purpose of assessing HIV risk behaviors among OVC. Neither scale displayed significant discriminant validity. These scales included the full sample of adolescents, suggesting that the lack of significant findings was not attributable to statistical power.
Finally, we evaluated individual WAF HIV risk behavior items. Our results suggest two domains of items that effectively discriminated cases and non-cases. The first domain could be described as self-appraised HIV risk. We found that three items: 1) ‘based on your own behavior, what is your risk of getting HIV?’ 2) ‘do you suspect or think you may have HIV?’ and 3) ‘are you concerned about getting HIV?’ all displayed significant discriminant validity between HIV risk behavior cases and non-cases. This suggests that asking OVC directly about their own risk for contracting HIV is a useful gauge of HIV risk behavior. A second domain of items that performed well was sexual behavior, specifically items asking if the adolescent had 1) ‘ever had sex’; and 2) ‘ever had an STI’. Both were reported at significantly higher rates among HIV risk behavior cases than non-cases. Our results indicate that the two best performing individual items in discriminating cases and non-cases were ‘suspect or think you may have HIV’ and ‘ever had an STI.’ As described in previous studies of indicators for risky sexual behavior (Slaymaker, 2004), these single item indicators, even the best performing ones, should not be used alone but rather in tandem with other single item indicators.
Limitations
There were limitations to this study. First, the statistical power for analyses with the WAF Risk Reduction Intentions and Readiness to Change scales was limited because they were only administered to the 20% (n=35) of our sample that reported ever having had sex. Therefore, the results from the reliability and validity testing for these scales should be considered exploratory. We reiterate the call for researchers to be mindful of the possible need for oversampling adolescents who are sexually active when using these measures. Second, we did not have the ability to test the impact of interview modality—all participants were administered the study instruments via ACASI and there was no comparison method (i.e., face-to-face interviewing, biomarker collection). Finally, although we provide ‘optimal’ cut-off scores for the PHRBS and Sexual Behavior Practice scales for this population (scores of ≥2 and ≥1, respectively), these are based on the closest possible balance of sensitivity and specificity. Our analyses do not provide recommendation for cut-offs that should be used in specific HIV risk behavior prevention/intervention programming.
Conclusions
Developing valid and reliable instruments of HIV risk behaviors among OVC in sub-Saharan Africa is critical for accurately evaluating the effectiveness of HIV prevention programs. This study provides evidence supporting the use of the PHRBS, an indirect method of assessing an adolescent’s HIV risk behaviors that asks about the risky sexual behavior of peers. We recommend the use of this scale in combination with more traditional measures of HIV risk behavior (i.e., WAF Sexual Behavior Practices) and single item indicators that ask directly about HIV risk. Given the acceptability and feasibility of using ACASI-administered questionnaires in this setting, and previous studies in sub-Saharan Africa suggesting that ACASI provides more accurate reporting of sensitive behaviors (Langhaug et al., 2010), we recommend that HIV risk behavior measures be administered via ACASI methods when possible.
Acknowledgments
This project was supported by the National Institute of Child Health and Human Development (LKM/PB: R01HD070720), the National Institute of Mental Health (LKM: K23MH077532; SMM: T32MH103210), and the National Institute on Drug Abuse (JCK: T32DA007292).
References
- Adebajo S, Obianwu O, Eluwa G, Vu L, Oginni A, Tun W, … Karlyn A (2014). Comparison of Audio Computer Assisted Self-Interview and Face-To-Face Interview Methods in Eliciting HIV-Related Risks among Men Who Have Sex with Men and Men Who Inject Drugs in Nigeria. PLoS ONE, 9(1), e81981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bass JK, Ryder RW, Lammers M-C, Mukaba TN, & Bolton PA (2008). Post-partum depression in Kinshasa, Democratic Republic of Congo: validation of a concept using a mixed-methods cross-cultural approach. Tropical Medicine & International Health, 13(12), 1534–1542. [DOI] [PubMed] [Google Scholar]
- Beauclair R, Meng F, Deprez N, Temmerman M, Welte A, Hens N, & Delva W (2013). Evaluating audio computer assisted self-interviews in urban south African communities: evidence for good suitability and reduced social desirability bias of a cross-sectional survey on sexual behaviour. BMC Medical Research Methodology, 13(1), 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Betancourt TS, Bass J, Borisova I, Neugebauer R, Speelman L, Onyango G, & Bolton P (2009). Assessing local instrument reliability and validity: A field-based example from northern Uganda. Social Psychiatry and Psychiatric Epidemiology, 44(8), 685–692. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bolton P (2001). Cross-cultural validity and reliability testing of a standard psychiatric assessment instrument without a gold standard. The Journal of Nervous and Mental Disease, 189(4), 238–242. [DOI] [PubMed] [Google Scholar]
- Brook DW, Morojele NK, Zhang C, & Brook JS (2006). South African adolescents: pathways to risky sexual behavior. AIDS Education and Prevention, 18(3), 259–72. [DOI] [PubMed] [Google Scholar]
- Brook JS, Morojele NK, Pahl K, & Brook DW (2006). Predictors of drug use among South African adolescents. The Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine, 38(1), 26–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cherie A, & Berhane Y (2012). Peer Pressure Is the Prime Driver of Risky Sexual Behaviors among School Adolescents in Addis Ababa, Ethiopia. World Journal of AIDS, 2(3), 159–164. [Google Scholar]
- Cicchetti DV (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. [Google Scholar]
- Cowan FM, Langhaug LF, Mashungupa GP, Nyamurera T, Hargrove J, Jaffar S, … Regai Dzive Shiri Project. (2002). School based HIV prevention in Zimbabwe: feasibility and acceptability of evaluation trials using biological outcomes. AIDS, 16(12), 1673–8. [DOI] [PubMed] [Google Scholar]
- DiClemente RJ (2016). Validity of Self-reported Sexual Behavior Among Adolescents: Where Do We Go from Here? AIDS and Behavior, 20 Suppl 1, S215–7. [DOI] [PubMed] [Google Scholar]
- Glynn JR, Caraël M, Auvert B, Kahindo M, Chege J, Musonda R, … Study Group on the Heterogeneity of HIV Epidemics in African Cities. (2001). Why do young women have a much higher prevalence of HIV than young men? A study in Kisumu, Kenya and Ndola, Zambia. AIDS, 15 Suppl 4, S51–60. [DOI] [PubMed] [Google Scholar]
- Harrison A, Smit J, Hoffman S, Nzama T, Leu CS, Mantell J, … Exner T (2012). Gender, peer and partner influences on adolescent HIV risk in rural South Africa. Sexual Health, 9(2), 178–186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnston L, O’Bra H, Chopra M, Mathews C, Townsend L, Sabin K, … Kendall C (2010). The associations of voluntary counseling and testing acceptance and the perceived likelihood of being HIV-infected among men with multiple sex partners in a South African township. AIDS and Behavior, 14(4), 922–31. [DOI] [PubMed] [Google Scholar]
- Kane JC, Murray LK, Bass JK, Johnson RM, & Bolton P (2016). Validation of a substance and alcohol use assessment instrument among orphans and vulnerable children in Zambia using Audio Computer Assisted Self-Interviewing (ACASI). Drug and Alcohol Dependence, 166, 85–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kane JC, Murray LK, Sughrue S, DeMulder J, Skavenski van Wyk S, Queenan J, … van de Wijgert J (2016). Process and implementation of Audio Computer Assisted Self-Interviewing (ACASI) assessments in low resource settings: a case example from Zambia. Global Mental Health, 3, e24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Langhaug LF, Sherr L, & Cowan FM (2010). How to improve the validity of sexual behaviour reporting: systematic review of questionnaire delivery modes in developing countries. Tropical Medicine & International Health, 15(3), 362–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Magnani RJ, Karim AM, Weiss LA, Bond KC, Lemba M, & Morgan GT (2002). Reproductive health risk and protective factors among youth in Lusaka, Zambia. Journal of Adolescent Health, 30(1), 76–86. [DOI] [PubMed] [Google Scholar]
- Murray LK, Bass J, Chomba E, Imasiku M, Thea D, Semrau K, … Bolton P (2011). Validation of the UCLA Child Post traumatic stress disorder-reaction index in Zambia. International Journal of Mental Health Systems, 5(1), 24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- NIMH Multisite HIV Prevention Trial Group. The NIMH Multisite HIV Prevention Trial: reducing HIV sexual risk behavior. Science, 280(5371):1889–94. [DOI] [PubMed] [Google Scholar]
- Nunnally JC, & Bernstein IH. (1994). Psychometric Theory (3rd ed.). New York: McGraw-Hill, Inc. [Google Scholar]
- Operario D, Underhill K, Chuong C, & Cluver L (2011). HIV infection and sexual risk behaviour among youth who have experienced orphanhood: systematic review and meta-analysis. Journal of the International AIDS Society, 14, 25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Plummer ML, Ross DA, Wight D, Changalucha J, Mshana G, Wamoyi J, … Hayes RJ (2004). "A bit more truthful": the validity of adolescent sexual behaviour data collected in rural northern Tanzania using five methods. Sexually Transmitted Infections, 80 Suppl 2(suppl 2), ii49–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Potard C, Courtois R, & Rusch E (2008). The influence of peers on risky sexual behaviour during adolescence. The European Journal of Contraception & Reproductive Health Care, 13(3), 264–70. [DOI] [PubMed] [Google Scholar]
- Qiao S, Li X, & Stanton B (2014). Social support and HIV-related risk behaviors: A systematic review of the global literature. AIDS and Behavior, 18(2), 419–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scott-Sheldon LAJ, Walstrom P, Harrison A, Kalichman SC, & Carey MP (2013). Sexual risk reduction interventions for HIV prevention among South African youth: a meta-analytic review. Current HIV Research, 11(7), 549–58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sikkema KJ, Anderson ES, Kelly JA, Winett RA, Gore-Felton C, Roffman RA, … Brondino MJ (2005). Outcomes of a randomized, controlled community-level HIV prevention intervention for adolescents in low-income housing developments. AIDS, 19(14), 1509–1516. [DOI] [PubMed] [Google Scholar]
- Sikkema KJ, Brondino MJ, Anderson ES, Gore-Felton C, Kelly JA, Winett RA, … Roffman RA (2004). HIV risk behavior among ethnically diverse adolescents living in low-income housing developments. The Journal of Adolescent Health, 35(2), 141–150. [DOI] [PubMed] [Google Scholar]
- Slaymaker E (2004). A critique of international indicators of sexual risk behaviour. Sexually Transmitted Infections, 80(suppl 2), ii13–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- StataCorp. (2013). Stata Statistical Software: Release 13. College Station, TX. [Google Scholar]
- Tenkorang EY, Rajulton F, & Maticka-Tyndale E (2009). Perceived risks of HIV/AIDS and first sexual intercourse among youth in Cape Town, South Africa. AIDS and Behavior, 13(2), 234–45. [DOI] [PubMed] [Google Scholar]
- Thurman TR, Kidman R, Carton TW, & Chiroro P (2016). Psychological and behavioral interventions to reduce HIV risk: evidence from a randomized control trial among orphaned and vulnerable adolescents in South Africa. AIDS Care, 28(sup1), 8–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tufts University School of Medicine. (2014). Audio Computer Assisted Self-Interviewing Software Boston, MA. [Google Scholar]
- Wang J, Fisher JH, Siegal HA, Falck RS, & Carlson RG (1995). Influence of measurement errors on HIV risk behavior analysis: A case study examining condom use among drug users. Structural Equation Modeling, 2(2), 319–334. [Google Scholar]
- Whetten K, Ostermann J, Whetten RA, Pence BW, O’Donnell K, Messer LC, … Team, T. P. O. for O. (POFO) R. (2009). A Comparison of the Wellbeing of Orphans and Abandoned Children Ages 6–12 in Institutional and Community-Based Care Settings in 5 Less Wealthy Nations. PLoS ONE, 4(12), e8169. [DOI] [PMC free article] [PubMed] [Google Scholar]