Evidence for the Reliability and Validity of the Internalized AIDS-Related Stigma Scale in Rural Uganda

Abstract

HIV infection remains highly stigmatized throughout sub-Saharan Africa despite the increasing availability of treatment. HIV-related stigma is commonly described to be highly prevalent in East Africa, but none of these studies have employed validated scales for measurement. We used data from 456 people living with HIV/AIDS in rural Uganda to validate the six-item Internalized AIDS-Related Stigma Scale. The scale demonstrated acceptable internal consistency (Cronbach’s alpha=0.73) and time stability. Exploratory factor analysis indicated the presence of a single factor. Construct validity was supported by observations that the scale was correlated with related constructs such as depression and mental health-related quality of life. The scale was able to discriminate between groups of persons who were different in terms of treatment status and their experience of HIV-related self-blame. Taken together, these findings suggest that the Internalized AIDS-Related Stigma Scale may be a useful tool for socio-behavioral HIV research.

The increasing availability of antiretroviral therapy in resource-limited settings has the potential to attenuate people’s fears about HIV infection [1–5], but the illness remains highly stigmatized throughout sub-Saharan Africa [6–9]. HIV-related stigma likely compromises both treatment and prevention efforts [10–15], and the measurement of HIV-related stigma has been the focus of an increasing number of research studies. However, most scales (and intervention studies) have focused on the stigmatizing beliefs of HIV-uninfected persons [16–18]. Fewer studies have focused on the measurement of stigma among people living with HIV/AIDS (PLWHA) [16].

HIV-related stigma is commonly described to be highly prevalent among PLWHA in Uganda [19–23] and other settings in East Africa [24–26], but few studies have employed validated scales for measurement. With the exception of the HIV/AIDS Stigma Instrument-PLWA -- which has been validated among PLWHA in Tanzania [27] -- no scales for the measurement of stigma among PLWHA have been developed or validated in East Africa. We used data from PLWHA in rural Uganda to validate a scale previously developed to measure internalized HIV-related stigma among PLWHA in Southern Africa [28]. Specifically, we assessed the scale’s reliability, factor structure, and construct validity.

METHODS

Study population, design, and data collection

Mbarara District is located in a rural area of Uganda southwest of Kampala, reachable by a five-hour automobile drive. Mbarara town (population 82,000) is the primary commercial hub, but the majority of district residents live in outlying rural areas. The Uganda AIDS Rural Treatment Outcomes (UARTO) study is an ongoing cohort of treatment-naïve PLWHA initiating HIV antiretroviral therapy who are recruited from the Mbarara Immune Suppression Syndrome Clinic. Written consent is obtained from all study participants. In the event that there are cultural literacy reasons why a signature is not appropriate, participants are allowed to mark consent forms with a thumbprint. Participants are seen every three months for blood draws and interviewed by a research assistant in the local language (Runyankole). Midway through the study, while recruitment of new treatment-naïve participants into UARTO was still ongoing, assessments for internalized HIV-related stigma were added to the survey instrument. Data for the validation analyses presented in this article were drawn from the first two visits attended by study participants after the stigma scale was introduced. Therefore approximately one-half of the initial stigma assessments (as described in detail below) can be considered true baseline assessments in that the survey instruments were administered to treatment-naïve PLWHA newly initiating treatment; the rest of the survey instruments were administered to PLWHA who were already treated, i.e., they had enrolled in UARTO and initiated ART at least three months prior to the first stigma assessment. Ethical approval for all study procedures was obtained from the Committee on Human Research, University of California at San Francisco; the Partners Human Research Committee, Massachusetts General Hospital; and the Institutional Ethical Review Committee, Mbarara University of Science and Technology.

Measures

The focus of this analysis was the Internalized AIDS-Related Stigma Scale, a six-item scale designed to measure the construct of internalized stigma [28]. Each item offers a binary (yes/no) response, and the total scale score is computed as the sum of the items. Internalized stigma represents the outcome of a process through which PLWHA accept their discredited status (imposed by the majority group) as valid and develop self-defacing internal representations of themselves [29]. Our use of a scale for one specific aspect of stigma, rather than a scale designed to more comprehensively measure different aspects of stigma [27, 30], was driven largely by practical concerns about the overall length of the survey instrument.

The UARTO psychosocial assessments also sought to measure several potentially related constructs, including self-blame for HIV status, depression symptom severity, enacted stigma, health-related quality of life, and HIV-related symptom burden. We measured self-blame by asking participants whether they agreed (yes/no) with the statement, “It is my own fault that I am HIV-positive.” To measure depression symptom severity, we used the 15-item Hopkins Symptom Checklist for Depression [31]. Following prior studies of depression in Uganda, we modified the depression scale for the local context by adding a 16th item, “feeling like I don’t care about my health” [32]. Each item is scored on a four-point Likert-type scale ranging from 1 to 4, and the total scale score is computed as the average of the 16 items. To assess enacted stigma, which refers to overt acts of HIV-related discrimination [29], we asked participants whether or not they had ever experienced (yes/no) any one of 13 different events, including spousal abandonment, discrimination in health care settings, loss of housing, threats of violence, or physical assault. Health-related quality of life was measured with the Medical Outcomes Study-HIV Health Survey (MOS-HIV) mental health summary (MHS) and physical health summary (PHS) scores. The MOS-HIV consists of 35 items grouped into 11 domains, and higher MHS and PHS scores reflect a better health-related quality of life [33–35]. The individual domains are scored as summated rating scales from 0–100, and the overall MHS and PHS scores are transformed to t-scores with a mean of 50 and a standard deviation of 10. For HIV-related symptom burden, we inquired about whether or not participants had experienced, in the previous month, any of 29 potentially HIV-related symptoms (e.g., “tremors or shakes,” “problems with weight loss or wasting,” “change in the way your body looks such as fat deposits or weight gain,” “skin problems, such as rash, dryness, or itching,” “hair loss or changes in the way your hair looks,” “enlarged bumps in your neck, armpits, or groin,” etc.). Among participants who had experienced a particular symptom, the extent to which they found each symptom bothersome was scored on a four-point Likert-type scale ranging from 0 to 4. These variables were used to create an equally weighted average of the z-scores [36], with the sign of the aggregate measure oriented so that greater values of the symptom index are associated with a greater symptom burden.

Statistical analysis

All statistical analyses were conducted using the Stata/MP software package (version 12.0, StataCorp LP, College Station, Tex.). The validation analyses were designed to parallel those implemented in the original development and validation study by Kalichman et al. [28]. We performed factor analysis on the scale items, using principal-factors extraction and orthogonal varimax rotation. Factors were retained based on three criteria. First, we examined the factor eigenvalues for those factors with eigenvalues greater than 1.0. Second, we graphed the eigenvalues in decreasing order to identify the scree, i.e., the portion of the graph where the slope of decreasing eigenvalues approaches zero [37]. Third, we examined the loadings of the individual items on the different factors. An item was assigned to a factor if its factor loading was greater than or equal to 0.40 [38]. We calculated Cronbach’s alpha to assess internal consistency of the identified factor, using 2,000 bootstrap replications to compute the standard error. We examined item-test correlations, and then re-calculated the Cronbach’s alpha after sequentially deleting each of the items in turn. To assess time stability, the internalized stigma scale was administered again to each participant three months after initial administration; we then used data from both assessments to calculate the intraclass correlation coefficient (ICC) [39].

We did not have access to a gold standard criterion to assess criterion-related validity. Instead we relied upon several different assessments of construct validity. First, we calculated the Pearson product-moment correlation coefficients between internalized stigma and other constructs hypothesized, on the basis of prior theoretical or empirical work, to be either related or unrelated to internalized stigma. Namely, multiple studies have identified internalized stigma to be a strong correlate of mental well being among PLWHA [40–45]. In a previously published meta-analysis, the correlation between stigma and mental well being was stronger than the correlation between stigma and physical health status or between stigma and HIV-related symptoms [46]. We therefore hypothesized that depression symptom severity and mental health-related quality of life would be strongly correlated with internalized stigma, and that HIV-related symptom burden and physical health-related quality of life would be less strongly correlated. Similar to the above, we performed the non-parametric equality-of-medians test to assess for differences in depression symptom severity, health-related quality of life, and HIV-related symptom burden between persons who reported any internalized stigma (i.e., responded “yes” to any one of the six items) vs. those who reported no internalized stigma (i.e., responded “no” to all of the six items). Second, to provide evidence of construct validity using the “known groups” method [47], we performed non-parametric equality-of-medians tests to test the equality of median levels of internalized stigma between persons who had experienced any enacted stigma vs. those who had not experienced any enacted stigma, between persons who blamed themselves for their HIV seropositivity vs. those who did not blame themselves, and between persons who were on ART vs. those who were newly initiating ART. We justified inclusion of the latter in the known groups analysis based on prior studies linking treatment availability and access to declines in stigma among PLWHA [1, 2, 4, 5, 48–50].

RESULTS

The sample consisted of 456 PLWHA, 259 (56.8%) of whom were newly initiating ART; the rest had been on treatment for a median of 14 months (interquartile range [IQR], 11–19 months) at the time of the initial stigma assessment (Table 1). The median age was 35 years (IQR, 29–40 years), 317 participants (69.5%) were women, and approximately one-half were married or cohabiting with a partner (200 [43.9%]). A minority of participants had achieved a secondary education (103 [22.6%]), and a minority of participants were unemployed (132 [29.0%]). The median CD4+ T-lymphocyte cell count was 203 cells/mm³ (IQR, 129–289).

Table 1.

Characteristics of the sample (N=456)

	Median (interquartile range) or No. (%)
Age, years	35 (29–40)
Women	317 (69.5%)
Marital status
Never married	32 (7.0%)
Separated, divorced, or widowed	212 (46.5%)
Married or cohabiting	200 (43.9%)
Number of household members	3 (1–5)
Educational attainment
None	78 (17.1%)
Primary schooling	263 (57.7%)
Secondary schooling or more	103 (22.6%)
Currently unemployed	132 (29.0%)
CD4+ T-lymphocyte cell count, per mL	203 (129–289)
Currently on antiretroviral therapy	259 (56.8%)
Duration of treatment, months	14 (11–19)
Internalized AIDS-Related Stigma Scale, total score	1 (0–2)
Endorses feelings of self-blame	106 (23.3%)
Hopkins Symptom Checklist for Depression, total score	1.3 (1.1–1.6)
Experienced any enacted HIV-related stigma	93 (20.4%)
Physical Health Summary, total score	52.3 (44.9–58.1)
Mental Health Summary, total score	51.9 (45.4–56.9)
HIV-related symptom burden	0.1 (−0.2 to 0.5)

The entire range of potential values of the internalized stigma scale was represented among the participants, and the distribution of the scores was right-skewed: 197 (43.2%) participants responded negatively to all six items, whereas nine (2.0%) participants responded affirmatively to all six items. Exploratory factor analysis revealed one factor with an eigenvalue of 2.03. All six items loaded positively on this factor, with factor loadings ranging from 0.49 to 0.72 (Table 2). None of the squared multiple correlations were so small as to warrant exclusion of any items. There was scant evidence for a second factor given that only one item (“it is difficult to tell people”) had a factor loading >0.40 on the second factor.

Table 2.

Reliability of the Internalized AIDS-Related Stigma Scale

	No. (%)	Factor loading	Item-test correlation	Cronbach’s alpha if deleted
It is difficult to tell people about my HIV infection	176 (38.6)	0.49	0.67	0.72
Being HIV positive makes me feel dirty	59 (12.9)	0.62	0.66	0.69
I feel guilty that I am HIV positive	93 (20.4)	0.57	0.64	0.71
I am ashamed that I am HIV positive	69 (15.1)	0.72	0.74	0.66
I sometimes feel worthless because I am HIV positive	39 (8.6)	0.53	0.55	0.72
I hide my HIV status from others	157 (34.4)	0.54	0.70	0.69

The internalized stigma scale was internally consistent, as suggested by its Cronbach’s alpha of 0.73 (95% confidence interval [CI], 0.69–0.78). Item-test correlations were approximately equal for all items. Deletion of any single item did not appreciably increase the average inter-item covariances or the Cronbach’s alpha; however, several items could potentially be deleted with limited impact on internal consistency. The ICC, as a measure of time stability, was 0.40.

Evidence in support of the scale’s construct validity was provided in analyses showing statistically significant correlations of moderately-sized magnitude between internalized stigma and depression symptom severity (r=0.33; 95% CI, 0.24–0.42) and between internalized stigma and mental health-related quality of life (r=−0.38; 95% CI, −0.46 to −0.30). Weaker, but still statistically significant, correlations with HIV-related symptom burden (r=0.28; 95% CI, 0.18–0.36) and physical health-related quality of life (r=−0.24; 95% CI, −0.33 to −0.14) were also observed. The correlation between internalized and enacted stigma was not statistically significant (r=0.08; 95% CI, −0.02 to 0.17). Expressed differently, persons who reported no internalized stigma had lower median levels of depression symptom severity (1.22 vs. 1.31; P=0.002) and HIV-related symptom burden (0.002 vs. 0.16; P=0.005), and higher median levels of mental (54.2 vs. 49.7; P<0.001) and physical (54.2 vs. 50.3; P=0.001) health-related quality of life.

Further evidence for construct validity was supplied with analyses conducted using the known-groups method (Table 3). The median level of internalized stigma was greater among participants who blamed themselves for their HIV seropositivity and among participants who were newly initiating ART. The median difference between participants who had vs. had not experienced HIV-related discrimination were not statistically significant.

Table 3.

Median differences in internalized stigma scores between known groups

	Has not experienced	Has experienced	χ2 test statistic (P-value)
Experienced HIV-related discrimination	1	1	0.05 (0.82)
Currently on antiretroviral therapy	1	0	10.4 (0.001)
Feels self-blame for HIV seropositivity	1	2	39.4 (<0.001)

DISCUSSION

Our analyses of data from PLWHA in rural Uganda provide support for the internal structure and construct validity of the Internalized AIDS-Related Stigma Scale. We modeled our reliability and validity analyses after those performed in the original study by Kalichman et al. [28] in order to maximize comparability between the two. Our findings suggest the Internalized AIDS-Related Stigma Scale may be used to measure the construct of internalized stigma among PLWHA in East African settings.

In general, our analyses confirmed that the Internalized AIDS-Related Stigma Scale is unidimensional, that the scale items have a high internal consistency, and that the scale can be appropriately interpreted as measuring the construct of internalized stigma. Our estimates of reliability and time stability were comparable to those provided by Kalichman et al. [28]. We provided additional evidence for construct validity by demonstrating that mental health-related quality of life was greater among non-stigmatized persons and that internalized stigma was greater among persons newly initiating ART and among persons who experienced HIV-related self-blame.

Our findings are consistent with previous work. Similar to the meta-analytic estimates presented in one previously published review [46], the magnitudes of the correlations we estimated between internalized stigma and two measures of mental well being were larger than the estimated correlations between internalized stigma and physical health status and between internalized stigma and HIV-related symptoms. We found that previous experience of HIV-related discrimination was associated with greater internalized stigma, also consistent with estimates from one study of PLWHA [51]. Overall, the patterns of association estimated in our analysis suggest that the scale may be used to examine internalized stigma and its effects on HIV treatment and health behaviors, or to assess the effects of interventions designed to mitigate the adverse effects of stigma on the quality of life of PLWHA. To date, however, most stigma reduction strategies (and stigma scale development studies) have focused on HIV-uninfected persons, have relied on improving HIV-related knowledge or building tolerance through sensitivity training, and have had limited success [17, 18]. New strategies for stigma reduction are clearly needed.

We noted that the median level of internalized stigma was greater among participants newly initiating treatment compared to treated participants. The treated participants had been on treatment for a median of 14 months, suggesting that internalized stigma may attenuate with HIV treatment. This hypothesis is consistent with previously documented programmatic experiences [1, 2, 48, 52, 53] and qualitative research [4, 5, 49, 50, 54] linking treatment availability and treatment access with reductions in stigma among PLWHA, as well as one longitudinal analysis of data from PLWHA in southern India [55]. Further longitudinal study may help confirm these findings and potentially clarify the mechanisms underlying the observed attenuation in stigma.

Our study design was limited in several respects. First, our sample was limited to PLWHA accessing ART. HIV-related stigma has been shown to be associated with depression [56] and negatively associated with both HIV testing [26, 57] and treatment access [58–63]. The participants in our sample were therefore likely to have internalized less of the stigma of HIV compared to PLWHA not in treatment. Consistent with this, the mean levels of internalized stigma in the development and validation samples used by Kalichman et al. [28] were greater than the mean level among the PLWHA in our study sample. Second, we did not have access to a gold standard measure of internalized stigma to establish criterion-related validity. To our knowledge no gold standard criterion has yet been developed for this purpose. Third, the stigma scale was administered to study participants in the context of an ongoing cohort study. Due to concerns about the overall length of the survey instrument, only a limited set of survey questions were administered and we did not initiate our investigation with a wider pool of potential items. HIV-related stigma is multifaceted [29, 64], consisting of other related constructs in addition to internalized stigma. Administering additional items would likely have suggested additional factors for retention.

In summary, we have estimated the reliability and characterized the construct validity and factor structure of the Internalized AIDS-Related Stigma Scale in a sample of PLWHA in rural Uganda. We conclude that this scale may be a useful tool for socio-behavioral HIV research in Eastern Africa.