Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Nov 1.
Published in final edited form as: Stigma Health. 2019 Mar 14;4(4):487–494. doi: 10.1037/sah0000170

Experience Sampling Method Versus Questionnaire Measurement of HIV Stigma: Psychosocial Predictors of Response Discrepancies and Associations With HIV Outcomes

Pariya L Fazeli a, Bulent Turan b,†,*
PMCID: PMC6876120  NIHMSID: NIHMS1016959  PMID: 31768417

Abstract

Accurate measurement of HIV-related stigma is key in understanding and reducing stigma for people living with HIV (PLWH). Experience sampling method (ESM) measures “state”-level phenomena and may improve understanding of daily stigma experiences of PLWH. In 109 men living with HIV, we examined: 1) associations between questionnaire (Q) and ESM internalized and enacted stigma measures; 2) psychosocial predictors (e.g., coping style, perceived HIV community stigma, helplessness) of discrepancies between Q and ESM internalized and enacted stigma; 3) whether Q or ESM measures better predict HIV outcomes. Hierarchical Linear Modeling showed moderate associations between ESM and Q measures of both internalized and enacted stigma. A majority of the psychosocial measures were associated with larger differences between both Q- and ESM-internalized stigma and enacted stigma, respectively. ESM measures were stronger predictors of visit adherence than Q measures. ESM may be advantageous in understanding moment-to-moment changes in stigma and associated processes in PLWH, particularly those with maladaptive psychological traits.

Keywords: ESM, EMA, HIV/AIDS, discrimination, stigma

Introduction

Stigma is deleterious to health, and HIV-related stigma is particularly damaging for people living with HIV (PLWH), leading to lower antiretroviral medication adherence, lower visit adherence, poorer cascade outcomes (including viral suppression), and higher rates of co-morbid mental illnesses such as depression and worse quality of life (Katz et al., 2013; Rice et al., 2017; Turan et al., 2017). Furthermore, HIV heavily affects already marginalized communities such as men who have sex with men (MSM), intravenous drug users, those with lower socioeconomic status, older adults, and minority groups, especially African Americans in the rural south (Emlet, 2006; Herek, 2002; Turan, Smith, et al., 2016). As such, intersectional stigma can lead to “double jeopardy” wherein these groups are discriminated against not just because of their HIV status, but also for overlapping personal characteristics or identities (that may be HIV risk factors in themselves) (Reference48J. M. Turan et al., In Press).

Given that HIV-related stigma persists in society, this psychosocial construct is an important factor to be considered in a range of research contexts. Valid and reliable measurement of HIV-related stigma is crucial in order to derive accurate findings. The four main dimensions of HIV-related stigma include enacted (or experienced) stigma (i.e., discrimination), perceived HIV stigma in the community, anticipated stigma, and internalized stigma (Berger, Ferrans, & Lashley, 2001; Earnshaw & Chaudoir, 2009; Earnshaw, Smith, Chaudoir, Amico, & Copenhaver, 2013; Turan et al., 2017). Given that perceived community stigma and anticipated stigma may be more stable from day to day, the current study focused on assessment of enacted and internalized stigma in daily life. Specifically, enacted HIV stigma refers to the degree to which PLWH experience prejudice or acts of discrimination by others. Internalized HIV stigma refers to accepting and endorsing negative beliefs about PLWH and applying them to the self (Berger et al., 2001; Earnshaw & Chaudoir, 2009; Earnshaw et al., 2013; Turan et al., 2017). Studies suggest that these two dimensions are associated, with enacted stigma predicting internalization of stigma, particularly for those possessing certain maladaptive individual traits and psychosocial risk factors (Fazeli et al., 2017). Further, there is evidence that the effect of enacted stigma on treatment outcomes may be mediated by internalized stigma (Kay et al., 2018).

While historically the most common approach to measuring stigma dimensions has been self-report questionnaire, such measures can only capture mental averages of one’s global tendencies and experiences over time. Thus, questionnaires are limited because they rely on respondents’ ability and willingness to report how they generally perceive, feel, and think across situations. Research grounded in cognitive psychology models (Tourangeau, 2000) and cognitive interviewing studies (French, Cooke, McLean, Williams, & Sutton, 2007) suggests that questionnaires can be subject to problems, particularly with retrieval and judgement processes, which may be influenced by both cognitive (e.g., retention interval, reference period) as well as psychological processes (e.g., emotion, personality) (Jobe, 2003). Specific retrospective recall biases of questionnaire report include disproportionally weighting experiences that are congruent with the current state (i.e., mood-congruent memory effect) (Ebner-Priemer et al., 2006), those experiences that are most recent (i.e., recency effect), or intense (i.e., salience/novelty effect) (Redelmeier & Kahneman, 1996), as well as less accuracy for recalling negative experiences (i.e., affective valence effect). Such biases can ultimately skew the reported estimation of the construct being measured via questionnaire.

Experience sampling method (ESM, also known as Ecological Momentary Assessment), a method widely used in psychological and medical research, is an effective way to query participants several times throughout the day about their emotions and experiences as they occur. In other words, ESM captures “state” levels of cognitions, emotions, and experiences in the moment, whereas questionnaires reflect general (trait) levels of these experiences and beliefs about emotions. ESM is able to avoid many of the inaccuracies and biases of recall and global assessments associated with questionnaires (Nezlek, 2007; Trull & Ebner-Priemer, 2014). As such, compared with traditional questionnaires, ESM provides a real-time, ecologically valid approach to measuring phenomena in an individual’s naturalistic environment, providing rich within-person contextual data and may facilitate a greater understanding of the daily experiences of living with a highly stigmatized condition for PLWH.

Given that most research uses self-report questionnaires, it is important to establish their validity in comparison with the ESM approach, and to determine which approach may be advantageous for certain situations and constructs. For example, one study found that when using comparable ESM and paper-and-pencil measures to detect change following a stress reduction intervention, only the ESM measures were sensitive to detecting improvements for mindfulness and depression (Moore, Depp, Wetherell, & Lenze, 2016). The authors hypothesized that the repeated measurements used in ESM reduces variability by minimizing the possibility of state effects that can influence measures only gathered at one point in time. Discrepancies between questionnaires and ESM have been shown consistently across many clinical conditions, such as pain, depression, and personality disorders, with a tendency towards symptoms to be described as more intense and frequent with recall based questionnaires compared to ESM (Houtveen & Oei, 2007; Solhan, Trull, Jahng, & Wood, 2009). While one study with PLWH found a moderate concordance between ESM and questionnaire HIV treatment self-efficacy (Turan, Fazeli, Raper, Mugavero, & Johnson, 2016), the discrepancies between these two formats suggests that there may be variables influencing differences in reporting between these modes of assessment.

Indeed, studies suggest that personal attributes and individual differences may affect questionnaire recall accuracy. One study found greater concordance between ESM and retrospective questionnaires of pain in those with stable levels of pain than those with greater variability in pain (Stone, Schwartz, Broderick, & Shiffman, 2005), which consistent with other studies (Perrine & Schroder, 2005) suggests that questionnaires may be strongly influenced by variability or instability in an individual’s experiences. In another study, respondents high in neuroticism retrospectively overestimated their experience of negative emotions in questionnaires compared to ESM (Barrett, 1997). Similarly, in a study of math anxiety, girls reported higher levels of trait anxiety (assessed with questionnaires) than boys, while there were no gender differences for state (ESM) anxiety. These discrepant findings between trait vs ESM were partially accounted for by girls’ lower perceived competence (Goetz, Bieg, Lüdtke, Pekrun, & Hall, 2013). Thus, recollection (i.e., questionnaire report) of enacted (or of internalized) stigma may be subject to transformation of the actual acts of discrimination (or of momentary feelings), either exaggerating the experience or minimizing its effect on one’s self. Research using both questionnaires (Emlet et al., 2013; Lee, 2002; Riggs, Vosvick, & Stallings, 2007) and ESM (Fazeli et al., 2017; Turan, Crockett, Buyukcan-Tetik, et al., In Press; Turan,. Crockett, Kempf, et al., In Press) suggests that PLWH with poorer functioning on several theoretically relevant person factors including depression, coping skills, attachment styles, and social support are more likely to internalize stigma, and ultimately have poorer HIV outcomes (Fazeli et al., 2017; Turan, Crockett, Buyukcan-Tetik, et al., In Press; Turan, Crockett, Kempf, et al., In Press). However, there is little to no research in PLWH on factors that may influence reporting of recall-based vs momentary assessment of stigma. Given the established inherent biases of traditional questionnaires and evidence of a negative recall bias in clinical populations, particularly personality disorders and depression, PLWH with poorer psychosocial functioning and maladaptive traits may be more likely to overestimate generalized stigma. Identifying predictors of discrepancies between these two formats is important for future research.

Further, it is also important to examine which of these formats may be more useful in predicting real world clinical outcomes in this population, or whether there is specificity in predicting unique outcomes for either approach. ESM assesses experiences, levels of cognitions, and emotions real-time, whereas questionnaires assess general levels, often after-the-fact, reflecting a person’s biases in memory and weighting the importance of specific episodes as well as the specific episodes themselves, and this process of bias may be an important factor in estimating the effects of stigma. Thus, these two measurement approaches may predict different outcomes or yield different effect sizes when evaluating the same outcome (Goetz et al., 2013; Porter et al., 2000; Robinson & Clore, 2002). If simple questionnaire measurement is as predictive or more predictive of outcomes as ESM, future work may not need to employ ESM or other intensive measurement techniques, unless the aim is to understand moment-to-moment changes and associated within-person processes rather than to predict long-term health outcomes.

Thus, the aim of this study is to compare two methods of HIV stigma measurement: questionnaires and ESM. Three research questions will be addressed: 1. How strongly are questionnaire and ESM measures of internalized stigma and enacted stigma, respectively, (ESM-internalized stigma and Q-internalized stigma, and ESM-enacted stigma and Q-enacted stigma), associated? 2. Do individuals higher on theoretically relevant negative psychosocial constructs (i.e., higher on attachment-related avoidance, attachment-related anxiety, avoidance and self-blame coping with HIV, perceived HIV-related stigma in the community, helplessness, depressive symptoms, and lower on social support) have larger differences between Q-internalized stigma and ESM-internalized stigma, and Q-enacted stigma and ESM-enacted stigma, respectively? 3. Given the evidence that questionnaire internalized stigma predicts important HIV outcomes such as treatment adherence, are momentary states of internalized stigma and enacted stigma (ESM) or a generalized sense of internalized stigma and enacted stigma (questionnaires) better predictors of HIV medication adherence and visit adherence, as well as depressive symptoms?

Methods

Participants and Procedures

Participants were recruited from an HIV clinic at a research university in Birmingham, AL for a study on psychosocial aspects of living with HIV. Men living with HIV who were not current substance users and were on antiretroviral therapy (ART) were included (N = 109). Specifically, the HIV clinic provided a list of patients meeting these criteria and subjects were then randomly selected from this list and invited to participate in the study at our research laboratory. Participants received smart phones provided by the study and subsequently responded to ESM questions three times daily for one week. Participants also completed self-administered (via computer) questionnaires on demographic characteristics and psychosocial and interpersonal constructs during in-person study visits. Thus, participants completed two study visits: one to orient them to the ESM protocol and complete demographic questionnaires and another visit approximately one week later (after ESM) to return the study phones and complete the additional questionnaires. This protocol was approved by the Institutional Review Board of the University and all procedures were carried out with the understanding and written consent of the participants.

Experience Sampling Method (ESM)

Participants received alerts to respond to the same ESM questions three times a day for seven consecutive days. ESM times were preset (i.e., programmed calendar reminders in study phones containing the ESM survey link) by research staff to be delivered between the hours of 10am and 8pm each day with at least two hours in between. Specifically, research staff chose times at random and changed the delivery time so the prompts were not sent at the same time each day, thus the ESM times were random to the participant. All ESM items were directly adapted from validated questionnaire measures. ESM questions included three items assessing enacted stigma by other people: (a) Since your last report, how much did you feel that someone treated you negatively because of your immune status? (b) Since last report, how much did you feel that someone kept their distance from you because of your immune status? (c) Since last report, how much did you feel that you were treated with less respect than other people because of your immune status? The term “immune status” was used instead of “HIV” for confidentiality purposes and this choice of wording was explained to the participants when they were given the smart phones and received instructions for using them to respond to the experience sampling questions. Response choices ranged from 1 (not at all) to 5 (very strongly) and the mean of the three items at each sampling occasion was used to assess enacted stigma.

ESM questions also included two items assessing internalized stigma (i.e., “Since your last report, how ashamed did you feel about your immune status?” and “Since your last report, how much did you feel that you are not as good as other people because of your immune status?”). Response choices ranged from 1 (not at all) to 5 (very strongly) and the mean of the two items at each sampling occasion was used to assess internalized stigma.

Questionnaire Measures

Internalized and Enacted Stigma.

A modified version (Bunn, Solomon, Miller, & Forehand, 2007) of the HIV Stigma Scale (Berger et al., 2001), which measures multiple dimensions of stigma associated with HIV was used to measure the constructs of internalized and enacted stigma. Specifically, the 11-item enacted stigma subscale was used to measure enacted stigma and the 7-item negative self-image subscale was used to measure internalized stigma. An example from the enacted stigma subscale is “I have lost friends by telling them that I have HIV/AIDS” and an example of the negative self-image subscale is “I feel guilty because I have HIV/AIDS.” Participants responded to each item on a 4-point Likert scale, with responses reflecting the degree of agreement ranging from 1 (strongly disagree) to 4 (strongly agree). No specified timeframe is provided for this measure. After reverse coding for certain items, the mean of scores for each subscale were used in analyses, such that higher scores reflect greater stigma. Cronbach’s alpha in the current study was .88 for internalized stigma and .94 for enacted stigma.

Perceived HIV stigma in the community.

The revised HIV Stigma Scale (Berger et al., 2001; Bunn et al., 2007) was also used to measure the construct of perceived HIV stigma in the community. Specifically, the subscale of concern with public attitudes was used, which includes 6 items (e.g., “People with HIV/AIDS are treated like outcasts”). Participants rated each item from 1 (strongly disagree) to 4 (strongly agree). The mean of the items was used to assess the participant’s perceived HIV stigma in the community. This subscale showed good internal consistency in the present study (Cronbach’s α = .87).

Attachment style.

A brief 18-item version of the most widely utilized attachment style measure, Experiences in Close relationships (ECR) (Brennan, Clark, & Shaver, 1998) was used. ECR measures two dimensions of insecure attachment: anxiety (e.g., “I worry about being abandoned”) and avoidance (“I am nervous when partners get too close to me”). Participants responded to each item using a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree) and mean scores were calculated for each 9-item scale. Both scales showed good internal consistency (Cronbach’s α = .88 and .90). There was a moderate correlation between attachment-related anxiety and attachment-related avoidance (r = .61).

Coping with HIV.

The 10-item avoidance and the 3-item blaming self subscales from the revised Ways of Coping List (Vitaliano, Russo, Carr, Maiuro, & Becker, 1985) were used to measure coping with HIV. An example item from the avoidance subscale is “I go on as if nothing had happened” and an example item from the blaming self subscale is “I blame myself”. Participants responded using a 5-point Likert scale ranging from 1 (I don’t do this at all) to 5 (I do this a lot) to rate degree of use for each strategy when dealing with experiences which remind them of their HIV status. Mean scores were used for each subscale. Cronbach’s α was .77 for avoidance coping and .82 for blame coping.

Cognitions of helplessness due to HIV.

The Illness Cognition Questionnaire for Chronic Diseases(Evers et al., 2001) was adapted to HIV as was done by Earnshaw et al (Earnshaw et al., 2013). This 18-item scale includes six items assessing helplessness (e.g., “My HIV controls my life”). Participants rated level of agreement with these six helplessness items using a 4-point scale, and a mean score was calculated. Cronbach’s α was .82 in the current data.

Social support.

We used the 16-item Interpersonal Support Evaluation List-Short Form (Payne et al., 2012). A sample item is “When I need suggestions on how to deal with a personal problem, I know someone I can turn to”. Participants rated each item to indicate how true it is for them using a 4-point rating scale ranging from 1 (definitely false) to 4 (definitely true). Mean scores were calculated for this measure. Cronbach’s α was .89 in the current data.

Depressive Symptoms.

Depressive symptom data were extracted from clinic records on the nine-item depression scale of the Patient Health Questionnaire (PHQ-9) (Kroenke, Spitzer, & Williams, 2001); α = .89 in the current study. The total score was used as a measure of depressive symptom severity.

HIV Outcomes

ART Adherence

ART adherence was assessed with a single question, developed by Lu et al (2008): “In the past 4 weeks, how was your ability to take all of your anti-HIV medications that were prescribed by your doctor?” Medication adherence data were available for 100 participants. Response choices ranged from very poor to excellent. Because 73% of participants indicated excellent adherence, responses were dichotomized (excellent versus all other response options). According to previous literature, this measure is as good as or better than other self-report adherence measures (Feldman et al., 2013). While self-report measures may over-estimate adherence, research has consistently demonstrated the predictive value of self-reported non-adherence (i.e., non-optimal adherence) (Thompson et al., 2012).

HIV Visit Adherence

Data on HIV clinic appointments for the prior 24-month period were extracted from clinic records and used to calculate HIV visit adherence. Visit adherence was operationalized as the proportion of attended visits divided by the number of total scheduled visits (total scheduled visits = attended visits plus no shows) that were not cancelled or rescheduled in advance (Mugavero, 2008). This measure has shown sensitivity in prior studies on HIV care engagement (Howe et al., 2014; Jones, Cook, Rodriguez, & Waldrop-Valverde, 2013).

Statistical Analyses

For Aim 1, we tested the association between questionnaire and ESM measures of internalized stigma (ESM-internalized stigma and Q-internalized stigma) using multi-level analyses with Hierarchical Liner Modeling (HLM) version 6.08 (Raudenbush, 2004), since ESM measures were nested within individuals. The model is described below. Level 2 variables were grand-centered (the mean score for all participants was subtracted from each participant’s score).

Level 1: ESM-internalized stigmati = β0i + eti

Level 2: β0i = γ00 + γ01 (age) + γ02 (race) + γ03 (Q- internalized stigma)+ u0i

At Level 1 (the within-person level), ESM-internalized stigmati is internalized stigma for the participant i at time t. β0i is the Level 1 intercept reflecting the average level of current (state) internalized stigma for each participant across all sampling occasions. The Level 2 equation (the between-person level) tests the predictors of β0i. γ03 is the coefficient reflecting the magnitude of the association between Q-internalized stigma and the average level of ESM-internalized stigma (controlling for the covariates). Then, using similar analytic strategies, we tested the association between questionnaire and ESM measures of enacted stigma (ESM-enacted stigma and Q-enacted stigma).

For the Aim 2 and Aim 3 analyses IBM SPSS (IBM Corp. 2017) was used. For Aim 2, we first brought the Q measures to the same scale as the ESM measures (dividing by 4 and multiplying by 5). We used dependent t-tests to examine whether Q and ESM stigma reports (using intercepts saved from our first HLM analysis above reflecting average ESM-internalized and ESM-enacted stigma for each participant) were different on average across participants. Then, we examined whether theoretically relevant personality traits predicted the magnitude of the discrepancy between ESM-enacted and internalized stigma and Q-enacted and internalized stigma. We computed difference scores between Q-internalized stigma and ESM-internalized stigma, as well as Q-enacted stigma and ESM-enacted stigma. We examined partial correlations of these difference scores with our psychosocial measures (attachment-related avoidance, attachment-related anxiety, avoidance and self-blame coping with HIV, perceived HIV-related stigma in the community, helplessness, depressive symptoms, and social support (controlling for age and race).

For Aim 3 we used separate models to examine whether ESM and Q enacted and internalized stigma measures, respectively, predicted the following dependent variables (controlling for age and race): HIV clinic visit adherence, depressive symptoms, and ART adherence. Multiple linear regression was used for HIV clinic visit adherence and depressive symptoms while logistic regression was used for ART adherence.

Results

Sample characteristics are presented in Table 1. Note that 74% of ESM prompts were responded to and thus included in analyses, given that HLM is robust to missing data at the within-person level by estimating the best fitting model using the available data for each participant (Hedeker, 2006).

Table 1.

Descriptive statistics on the study variables

Variable N %
Race
 White 50 45.9
 Black 59 54.1
Sexual Orientation
 Gay/Bisexual 91 84.3
 Heterosexual/Straight 17 15.7
ART Adherence
 Optimal 73 73.0
 Sub-optimal 27 27.0
Variable Mean (SD) Range
Visit Adherence 0.90 (0.17) 0.25 – 1.00
Age 41.36 (10.94) 24 – 68
ESM Questions
 Internalized HIV Stigma 1.35 (0.62) 1.00 – 3.84
 Enacted HIV Stigma 1.18 (0.49) 1.00 – 3.74
Questionnaire Measures
 Internalized HIV Stigma 1.96 (0.73) 1.00 – 4.00
 Enacted HIV Stigma 2.10 (0.71) 1.00 – 3.91
 Perceived HIV Community Stigma 2.64 (0.66) 1.00 – 4.00
 Attachment-related Avoidance 2.95 (1.44) 1.00 – 6.33
 Attachment-related Anxiety 3.48 (1.53) 1.00 – 7.00
 Avoidance Coping 2.12 (0.71) 1.00 – 4.10
 Blame Coping 2.25 (1.05) 1.00 – 5.00
 Helplessness 1.76 (0.64) 1.00 – 3.50
 Social Support 3.27 (0.55) 1.38 – 4.00
 Depressive Symptoms 3.1 (4.57) 0 – 27
Open in a new tab

Note. ART = Antiretroviral therapy; ESM = Experience sampling method. Unadjusted questionnaire stigma scores are reported (i.e., not on the same scale as ESM stigma items). n = 108 for sexual orientation and n = 100 for medication adherence.

AIM 1

In our HLM analysis for internalized stigma, the level 2 coefficient (γ03) reflecting the magnitude of the association between Q-internalized stigma and the average level of ESM-internalized stigma (controlling for the covariates) was significant (coefficient = .434, t = 4.504, p < .001), suggesting that ESM-internalized stigma and Q-internalized stigma are associated. The association between ESM-internalized stigma and Q-internalized stigma was also significant when the covariates were excluded. In order to obtain an effect size for this association, we saved the intercepts from the Level 1 equation (average ESM-internalized stigma across all sampling occasions) for each participant, which showed a Pearson correlation of r = .560, p < .001 with Q-internalized stigma. Another way of obtaining this effect size is to compare the variances of the residual errors at Level 2 when Q-internalized stigma is in the model versus when it is not. This analysis yielded an explained variance of .341, which corresponds to a correlation coefficient of .584.

Using similar analytic strategies, we tested the association between questionnaire and ESM measures of enacted stigma (ESM-enacted stigma and Q-enacted stigma). γ03, the coefficient reflecting the magnitude of the association between Q-enacted stigma and the average level of ESM-enacted stigma (controlling for the covariates) was significant (coefficient = .278, t = 3.222, p = .002). The association between ESM-enacted stigma and Q-enacted stigma was also significant when the covariates were excluded. The intercepts from the Level 1 equation (average ESM-enacted stigma) showed a Pearson correlation of r = .440, p < .001 with Q-enacted stigma.

AIM 2

The purpose of Aim 2 was to explore whether generalized perceptions of internalized stigma and enacted stigma differ for some individuals when compared to their reports of momentary feelings of internalized and enacted stigma. That is, some individuals may show similar levels of ESM-internalized stigma and Q-internalized stigma—and ESM-enacted stigma and Q-enacted stigma—whereas others may over or under report Q-internalized stigma versus ESM-internalized stigma—or Q-enacted stigma versus ESM-enacted stigma. Would theoretically relevant personality traits predict the magnitude of the discrepancy between ESM-internalized stigma (saved intercepts from our first HLM analysis above reflecting average ESM-internalized stigma for each participant) and Q-internalized stigma? A similar question can also be asked for enacted stigma. On average, Q-internalized stigma was higher than ESM-internalized stigma (t = 14.68, p < .001) and Q-enacted stigma was higher than ESM-enacted stigma (t = 18.66, p < .001). Partial correlations (controlling for age and race) between the computed difference scores and our psychosocial measures (attachment-related avoidance, attachment-related anxiety, avoidance and self-blame coping with HIV, perceived HIV-related stigma in the community, helplessness, depressive symptoms, and social support) showed that all of these measures except social support (marginally for depressive symptoms) predicted larger difference between Q-internalized stigma and ESM-internalized stigma, whereas for enacted stigma, all of these measures except for blame coping and attachment-related anxiety were significant predictors of a larger difference. Results are presented in Table 2.

Table 2.

Correlations between psychosocial variables and difference between ESM and Q internalized stigma and enacted stigma, respectively, controlling for age and race

Variable Internalized Stigma Enacted Stigma
Partial correlation P-value Partial
correlation		 P-value
Perceived Community Stigma 0.39 0.00 0.57 0.00
Attachment-Related Avoidance 0.33 0.00 0.26 0.01
Attachment-Related Anxiety 0.35 0.00 0.15 0.13
Avoidance Coping with HIV 0.35 0.00 0.29 0.00
Blame Coping 0.34 0.00 0.12 0.21
Helplessness 0.37 0.00 0.36 0.00
Social Support −0.12 0.23 −0.22 0.02
Depressive Symptoms 0.18 0.06 0.22 0.03
Open in a new tab

AIM 3

Next, multiple linear regression models (controlling for age and race) were used to examine HIV clinic visit adherence as the dependent variable. In separate models, ESM-internalized stigma and ESM-enacted stigma were significant predictors (β = −0.23, t = −2.54, p = .01 and β = −0.20, t = −2.24, p = .03, respectively), whereas Q-internalized stigma and Q-enacted stigma had smaller and non-significant effects (β = −0.15, t = −1.63, p = .11; β = −0.12, t = −1.21, p = .23, respectively).

In parallel analyses, we then tested logistic regression models predicting ART adherence. Controlling for the covariates, Q-internalized stigma was a significant predictor of non-optimal ART adherence (AOR = 0.52, p = .05, CI [0.27,1.01]). In similar analyses the effect of ESM-internalized stigma was of similar magnitude but not significant (AOR = 0.62, p = .22, CI [0.29,1.33]). Q-enacted stigma and ESM-enacted stigma were also not significant predictors of ART adherence in this small sample (AOR = 0.76, p = .48, CI [0.37,1.60], and AOR = 0.54, p = .20, CI [0.21,1.39], respectively).

Similar multiple linear regression models (controlling for age and race) were used to examine depressive symptoms as the dependent variable. ESM-internalized stigma and ESM-enacted stigma were significant predictors (β = 0.44, t = 5.01, p = .00 β = 0.37, t = 4.07, p = .00, respectively). Q-internalized stigma and Q-enacted stigma were also significant predictors with similar effect sizes (β = 0.45, t = 5.15, p = .00 β = 0.42, t = 4.51, p = .00, respectively).

Discussion

Stigma is a commonly experienced phenomenon in PLWH and is associated with a wide range of deleterious psychosocial and health outcomes. The current study aimed to compare ESM versus traditional questionnaire measurement of HIV-related stigma. Our first aim was to determine the degree of concordance between ESM and questionnaire measurement of both internalized and enacted stigma. We found significant moderate associations between ESM and questionnaire measures of both internalized and enacted stigma. These findings help to establish validity of these questionnaires by comparing them with a state level ESM approach. Yet, given that concordance was only moderate, there may be differences in the underlying constructs these formats measure, as well as differences in responses elicited from participants that may vary by person factors.

Given the evidence of a negative recall bias on retrospective questionnaires across many clinical populations as well the influence psychological processes and dispositions may have on such reports, our next aim was to examine psychosocial and personality predictors of discrepancies between the ESM and questionnaire stigma measures. In other words, do individuals with specific psychosocial and personal characteristics over or under report on one of these formats? These predictors included attachment and coping style, perceived HIV stigma in the community, helplessness, depressive symptoms, and social support. Poorer levels (i.e., higher scores) on all variables except social support predicted larger difference between Q-internalized stigma and ESM-internalized stigma (with Q reports indicating higher levels of stigma), whereas for enacted stigma, all measures except blame coping and attachment-related anxiety were significant predictors of the difference. It may be that social support is a more external, resource-based factor as compared to the other factors which may have less influence on over-reporting of general feelings of internalized stigma. Further, blame coping and attachment-related anxiety may not have emerged as predictors of the discrepancy between ESM and Q enacted stigma because both of these factors are related to a negative view of the self and may be associated with perceiving oneself as deserving of discrimination and accepting it at the time it occurs.

Overall, our findings are consistent with other studies showing that person factors may affect self-reports (Barrett, 1997; Goetz et al., 2013; Jobe, 2003; Perrine & Schroder, 2005; Stone et al., 2005); for example, respondents high in neuroticism retrospectively overestimate their experience of negative emotions and symptoms with questionnaires compared to ESM. Altogether, these findings suggest that PLWH with more maladaptive traits and psychosocial functioning may be more prone to distort their recollection and interpretation of experiences and feelings in a negative way and thus report higher generalized (i.e., trait) stigma compared to what they actually feel moment-to-moment in daily life, as captured by ESM. Thus, ESM may be a better approach for accurately assessing experiences and internalization of stigma particularly for more vulnerable individuals.

Our final aim was to determine whether ESM or questionnaire stigma measures would be stronger predictors of HIV outcomes, including visit adherence, medication (ART) adherence, and depressive symptoms. We found that both measurement approaches had relatively comparable associations with ART adherence and depressive symptoms, while for visit adherence ESM measures were stronger predictors. These findings suggest that overall, both approaches may be useful in predicting health outcomes in PLWH, and given the ease and low burden of questionnaires, this approach may be preferred in resource limited clinical and research settings. However, ESM may be a more useful approach in understanding moment-to-moment changes and associated processes. Specifically, in this study, the ESM measures were better predictors of the one outcome that reflects real-world data over a longer time period and is not self-reported (i.e., visit adherence), which suggests that ESM may be particularly sensitive to such objective measures. As such, interventions aimed at improving stigma and outcomes may benefit from ESM as it may allow for more granular examination of the intervention efficacy and the precise processes by which stigma affects outcomes. Indeed, research shows that ESM may be more sensitive to detecting change following interventions, and further, given its’ methodological and analytical advantages, the needed sample size may be lower in studies using ESM (Moore et al., 2016).

While ESM methodology is a major strength of the current study, there are some notable limitations. First, this study was conducted only in relatively young and middle aged men living with HIV, limiting the generalizability to both women and older counterparts, which are both growing and unique subgroups in the HIV/AIDS epidemic. Second, the questionnaires used in the current study were not repeated over time, which may have influenced associations with ESM measures, which was based upon an average of several time points. Third, our ESM questions included a subset of validated items derived from questionnaires, thus, the content was not exactly the same for the two measurement modalities. Finally, we did not examine anticipated stigma, thus future studies may examine ESM and questionnaire measurement of this additional stigma dimension.

Acknowledgements:

We would like to thank Maria Lechtreck, C. Blake Helms, Victoria C. Seghatol-Eslami, and Wesley R. Browning and all the research assistants in the Social Science Laboratory at the Department of Psychology at the University of Birmingham at Alabama for their help in data collection.

Funding: This research was supported by the University of Alabama at Birmingham (UAB) Center for AIDS Research CFAR, an NIH funded program (P30 AI027767) that was made possible by the following institutes: NIAID, NCI, NICHD, NHLBI, NIDA, NIA, NIDDK, NIGMS, and OAR. Dr. Fazeli is supported by R00 AG048762 from NIA (P. Fazeli, PI). Investigator support for this study was also provided by a Women’s Interagency HIV Study sub-study grant from the National Institute of Mental Health, R01MH104114. The contents of this publication are the sole responsibility of the authors and do not represent the official views of the NIH.

Footnotes

Conflicts of Interest: The authors have no conflicts of interest to disclose.

References

  1. Barrett LF (1997). The relationships among momentary emotion experiences, personality descriptions, and retrospective ratings of emotion. Personality and Social Psychology Bulletin, 23(10), 1100–1110. [Google Scholar]
  2. Berger BE, Ferrans CE, & Lashley FR (2001). Measuring stigma in people with HIV: psychometric assessment of the HIV stigma scale. Research in Nursing and Health, 24(6), 518–529. doi: 10.1002/nur.10011 [DOI] [PubMed] [Google Scholar]
  3. Brennan KA, Clark CL, & Shaver PR (1998). Self-report measurement of adult attachment: An integrative overview In Simpson JA & Rholes WS (Eds.), Attachment theory and close relationships (pp. 46–76). New York: Guilford Press. [Google Scholar]
  4. Bunn JY, Solomon SE, Miller C, & Forehand R (2007). Measurement of stigma in people with HIV: a reexamination of the HIV Stigma Scale. AIDS Education and Prevention, 19(3), 198–208. doi: 10.1521/aeap.2007.19.3.198 [DOI] [PubMed] [Google Scholar]
  5. Earnshaw VA, & Chaudoir SR (2009). From conceptualizing to measuring HIV stigma: a review of HIV stigma mechanism measures. AIDS and Behavior, 13(6), 1160–1177. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Earnshaw VA, Smith LR, Chaudoir SR, Amico KR, & Copenhaver MM (2013). HIV stigma mechanisms and well-being among PLWH: a test of the HIV stigma framework. AIDS and Behavior, 17(5), 1785–1795. doi: 10.1007/s10461-013-0437-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Ebner-Priemer UW, Kuo J, Welch SS, Thielgen T, Witte S, Bohus M, & Linehan MM (2006). A valence-dependent group-specific recall bias of retrospective self-reports: a study of borderline personality disorder in everyday life. J Nerv Ment Dis, 194(10), 774–779. doi: 10.1097/01.nmd.0000239900.46595.72 [DOI] [PubMed] [Google Scholar]
  8. Emlet CA (2006). A comparison of HIV stigma and disclosure patterns between older and younger adults living with HIV/AIDS. AIDS Patient Care and STDS, 20(5), 350–358. doi: 10.1089/apc.2006.20.350 [DOI] [PubMed] [Google Scholar]
  9. Emlet CA, Brennan DJ, Brennenstuhl S, Rueda S, Hart TA, Rourke SB, & Team OCS (2013). Protective and risk factors associated with stigma in a population of older adults living with HIV in Ontario, Canada. AIDS Care, 25(10), 1330–1339. doi: 10.1080/09540121.2013.774317 [DOI] [PubMed] [Google Scholar]
  10. Evers AW, Kraaimaat FW, van Lankveld W, Jongen PJ, Jacobs JW, & Bijlsma JW (2001). Beyond unfavorable thinking: the illness cognition questionnaire for chronic diseases. Journal of Consulting and Clinical Psychology, 69(6), 1026–1036. doi: 10.1037/0022-006X.69.6.1026 [DOI] [PubMed] [Google Scholar]
  11. Fazeli PL, Turan JM, Budhwani H, Smith W, Raper JL, Mugavero MJ, & Turan B (2017). Moment-to-moment within-person associations between acts of discrimination and internalized stigma in people living with HIV: An experience sampling study. Stigma and health, 2(3), 216–228. doi: 10.1037/sah0000051 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Feldman B, Fredericksen R, Crane P, Safren S, Mugavero M, Willig JH, . . . Kitahata M (2013). Evaluation of the single-item self-rating adherence scale for use in routine clinical care of people living with HIV. AIDS and Behavior, 17(1), 307–318. doi: 10.1007/s10461-012-0326-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. French DP, Cooke R, McLean N, Williams M, & Sutton S (2007). What do people think about when they answer theory of planned behaviour questionnaires? A ‘think aloud’ study. J Health Psychol, 12(4), 672–687. doi: 10.1177/1359105307078174 [DOI] [PubMed] [Google Scholar]
  14. Goetz T, Bieg M, Lüdtke O, Pekrun R, & Hall NC (2013). Do girls really experience more anxiety in mathematics? Psychological Science, 24(10), 2079–2087. [DOI] [PubMed] [Google Scholar]
  15. Hedeker D, Gibbons RD (2006). Longitudinal data analysis (Vol. 451): John Wiley & Sons. [Google Scholar]
  16. Herek GM (2002). Thinking about AIDS and stigma: A psychologist’s perspective. JL Med. & Ethics, 30, 594. [DOI] [PubMed] [Google Scholar]
  17. Houtveen JH, & Oei NY (2007). Recall bias in reporting medically unexplained symptoms comes from semantic memory. J Psychosom Res, 62(3), 277–282. doi: 10.1016/j.jpsychores.2006.11.006 [DOI] [PubMed] [Google Scholar]
  18. Howe CJ, Napravnik S, Cole SR, Kaufman JS, Adimora AA, Elston B, . . . Mugavero MJ (2014). African American race and HIV virological suppression: beyond disparities in clinic attendance. American Journal of Epidemiology, 179(12), 1484–1492. doi: 10.1093/aje/kwu069 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp. [Google Scholar]
  20. Jobe JB (2003). Cognitive psychology and self-reports: models and methods. Qual Life Res, 12(3), 219–227. [DOI] [PubMed] [Google Scholar]
  21. Jones D, Cook R, Rodriguez A, & Waldrop-Valverde D (2013). Personal HIV knowledge, appointment adherence and HIV outcomes. AIDS and Behavior, 17(1), 242–249. doi: 10.1007/s10461-012-0367-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Katz IT, Ryu AE, Onuegbu AG, Psaros C, Weiser SD, Bangsberg DR, & Tsai AC (2013). Impact of HIV-related stigma on treatment adherence: systematic review and meta-synthesis. J Int AIDS Soc, 16(3 Suppl 2), 18640. doi: 10.7448/IAS.16.3.18640 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kay ES, Rice WS, Crockett KB, Atkins GC, Batey DS, & Turan B (2018). Experienced HIV-Related Stigma in Health Care and Community Settings: Mediated Associations With Psychosocial and Health Outcomes. JAIDS Journal of Acquired Immune Deficiency Syndromes, 77(3), 257–263. doi: 10.1097/qai.0000000000001590 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lee RS, Kochman A, & Sikkema KJ (2002). Internalized Stigma Among People Living with HIV-AIDS. AIDS and Behavior, 6(4), 309–319. [Google Scholar]
  26. Lu M, Safren SA, Skolnik PR, Rogers WH, Coady W, Hardy H, & Wilson IB (2008). Optimal recall period and response task for self-reported HIV medication adherence. AIDS and Behavior, 12(1), 86–94. [DOI] [PubMed] [Google Scholar]
  27. Moore RC, Depp CA, Wetherell JL, & Lenze EJ (2016). Ecological momentary assessment versus standard assessment instruments for measuring mindfulness, depressed mood, and anxiety among older adults. J Psychiatr Res, 75, 116–123. doi: 10.1016/j.jpsychires.2016.01.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mugavero MJ (2008). Improving engagement in HIV care: what can we do? Topics in HIV Medicine, 16(5), 156–161. [PubMed] [Google Scholar]
  29. Nezlek JB (2007). A multilevel framework for understanding relationships among traits, states, situations and behaviours. European Journal of Personality, 21(6), 789–810. doi: 10.1002/per.640 [DOI] [Google Scholar]
  30. Payne TJ, Andrew M, Butler KR, Wyatt SB, Dubbert PM, & Mosley TH (2012). Psychometric evaluation of the interpersonal support evaluation list–short form in the ARIC study cohort. SAGE Open, 2(3), 2158244012461923. doi: 10.1177/2158244012461923 [DOI] [Google Scholar]
  31. Perrine MW, & Schroder KE (2005). How many drinks did you have on September September 11, 2001? J Stud Alcohol, 66(4), 536–544. [DOI] [PubMed] [Google Scholar]
  32. Porter LS, Marco CA, Schwartz JE, Neale JM, Shiffman S, & Stone AA (2000). Gender differences in coping: A comparison of trait and momentary assessments. Journal of Social and Clinical Psychology, 19(4), 480–498. [Google Scholar]
  33. Raudenbush SW (2004). HLM 6: Hierarchical linear and nonlinear modeling: Scientific Software International. [Google Scholar]
  34. Redelmeier DA, & Kahneman D (1996). Patients’ memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66(1), 3–8. [DOI] [PubMed] [Google Scholar]
  35. Rice WS, Crockett KB, Mugavero MJ, Raper JL, Atkins GC, & Turan B (2017). Association Between Internalized HIV-Related Stigma and HIV Care Visit Adherence. JAIDS Journal of Acquired Immune Deficiency Syndromes, 76(5), 482–487. doi: 10.1097/qai.0000000000001543 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Riggs SA, Vosvick M, & Stallings S (2007). Attachment style, stigma and psychological distress among HIV+ adults. J Health Psychol, 12(6), 922–936. doi: 10.1177/1359105307082457 [DOI] [PubMed] [Google Scholar]
  37. Robinson MD, & Clore GL (2002). Belief and feeling: evidence for an accessibility model of emotional self-report. Psychological Bulletin, 128(6), 934–960. [DOI] [PubMed] [Google Scholar]
  38. Solhan MB, Trull TJ, Jahng S, & Wood PK (2009). Clinical assessment of affective instability: comparing EMA indices, questionnaire reports, and retrospective recall. Psychol Assess, 21(3), 425–436. doi: 10.1037/a0016869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stone AA, Schwartz JE, Broderick JE, & Shiffman SS (2005). Variability of momentary pain predicts recall of weekly pain: a consequence of the peak (or salience) memory heuristic. Pers Soc Psychol Bull, 31(10), 1340–1346. doi: 10.1177/0146167205275615 [DOI] [PubMed] [Google Scholar]
  40. Thompson MA, Mugavero MJ, Amico KR, Cargill VA, Chang LW, Gross R, . . . Bartlett JG (2012). Guidelines for improving entry into and retention in care and antiretroviral adherence for persons with HIV: evidence-based recommendations from an International Association of Physicians in AIDS Care panel. Annals of Internal Medicine, 156(11), 817–833. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Tourangeau R, Rips LJ, & Rasinski K (2000). The psychology of survey response. Cambridge: Cambridge University Press. [Google Scholar]
  42. Trull TJ, & Ebner-Priemer U (2014). The Role of Ambulatory Assessment in Psychological Science. Current Directions in Psychological Science, 23(6), 466–470. doi: 10.1177/0963721414550706 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Turan B, Crockett KB, Buyukcan-Tetik A, Kempf M-C, Konkle-Parker D, Wilson TE, . . . Turan JM (In Press). Buffering Internalization of HIV Stigma: Implications for Treatment Adherence and Depression. JAIDS: Journal of Acquired Immune Deficiency Syndromes. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Turan B, Crockett KB, Kempf M-C, Konkle-Parker DJ, Wilson TE, Tien P, . . . Turan JM (In Press). Internal Working Models of Attachment Relationships and HIV Outcomes among Women Living with HIV. JAIDS Journal of Acquired Immune Deficiency Syndromes. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Turan B, Fazeli PL, Raper JL, Mugavero MJ, & Johnson MO (2016). Social Support and Moment-to-Moment Changes in Treatment Self-Efficacy in Men Living With HIV: Psychosocial Moderators and Clinical Outcomes. Health Psychology, 35(10), 1126–1134. doi: 10.1037/hea0000356 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Turan B, Hatcher AM, Weiser SD, Johnson MO, Rice WS, & Turan JM (2017). Framing Mechanisms Linking HIV-Related Stigma, Adherence to Treatment, and Health Outcomes. American Journal of Public Health, 107(6), 863–869. doi: 10.2105/AJPH.2017.303744 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Turan B, Smith W, Cohen MH, Wilson TE, Adimora AA, Merenstein D, . . . Turan JM (2016). Mechanisms for the Negative Effects of Internalized HIV-Related Stigma on Antiretroviral Therapy Adherence in Women: The Mediating Roles of Social Isolation and Depression. Journal of Acquired Immune Deficiency Syndromes, 72(2), 198–205. doi: 10.1097/QAI.0000000000000948 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Turan JM, Elafros M, Logie C, Banik S, Turan B, Crockett KB, . . . Murray S (In Press). Challenges and opportunities in examining and addressing intersectional stigma and health. BMC Medicine. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Vitaliano PP, Russo J, Carr JE, Maiuro RD, & Becker J (1985). The ways of coping checklist: Revision and psychometric properties. Multivariate Behavioral Research, 20(1), 3–26. doi: 10.1207/s15327906mbr2001_1 [DOI] [PubMed] [Google Scholar]

RESOURCES