Comparing the unmatched count technique and direct self-report for sensitive health-risk behaviors in HIV+ adults

Abstract

Researchers often rely on self-report measures to assess sensitive health-risk behaviors in HIV+ individuals, yet the accuracy of self-report has been questioned, particularly when inquiring about behaviors that may be embarrassing, risky, and/or taboo. We compared an anonymous reporting method—the Unmatched Count Technique (UCT)—to direct self-report in order to assess reporting differences for several health-risk behaviors related to medication adherence and sexual risk. Contrary to hypotheses, the UCT only produced a significantly higher estimated base rate for one sensitive behavior: reporting medication adherence to one's physician, which may have been contextually-primed by our study design. Our results suggest that anonymous reporting methods may not increase disclosure compared to direct self-report when assessing several health-risk behaviors in HIV+ research volunteers. However, our results also suggest that contextual factors should be considered and investigated further, as they may influence perception of sensitive behavior.

Assessing sensitive behavior is a particular challenge for researchers and clinicians. In HIV research, it is often necessary to ask HIV+ participants about sensitive health-risk behaviors, such as medication adherence, sexual risk behavior, and illicit drug use. Obtaining accurate information is critical given that many sensitive behaviors, particularly those linked to HIV-transmission risk, can have important public and individual health consequences. However, people may not always be willing to disclose information about behaviors that may be considered private, embarrassing, taboo, socially undesirable, risky, dangerous, or illegal when asked directly, leading some researchers to question whether or not self-report measures of sensitive behaviors are accurate. Research participants and patients may fear discomfort, judgment, or even punitive consequences, (Rosenthal, Persinger, & Fode, 1962), which could prevent them from being forthright about these behaviors. Consequently, several methodological approaches have been designed to minimize threats to frank disclosure and more accurately assess sensitive behaviors (Wolter & Preisendörfer, 2013; Yu, Tian, & Tang, 2008), including the unmatched count technique (UCT)—also referred to as the item count technique or the list experiment.

Introduced by Raghavarao & Federer (1979), the UCT allows individuals to respond to sensitive items in an embedded questionnaire format that preserves response anonymity. The UCT (described in detail in the Methods section) presents individuals with a list of statements, one of which targets the sensitive issue. Participants simply report how many statements in a given set are true for them, precluding definitive endorsement of potentially embarrassing or self-incriminating behaviors (Dalton, Wimbush, & Daily, 1994). It is reported to have higher validity than other anonymous response methods, including several types of randomized response techniques (Coutts & Jann, 2011), and may represent an alternative to the assessment methods commonly used in the current HIV literature (e.g., self-report).

However, the UCT is not without its limitations. By virtue of their design, UCT questionnaires are longer than comparable self-report measures, typically including 5 non-sensitive items for each sensitive item, and query is limited to a binary true/false or yes/no format. Because the UCT does not record item responses, no data is obtained on an individual's specific risk behaviors. It also requires a large sample size and its estimation method (i.e., embedding a sensitive item among several non-sensitive items) introduces significantly higher variance than self-report (Glynn, 2013). Given these trade-offs, it is important to evaluate the UCT carefully in each situation and/or population to determine whether the benefits outweigh the limitations.

Studies evaluating the UCT have reported mixed findings, suggesting that it may not be suited to all situations. In several studies examining college students, the UCT revealed higher estimated base rates for several sensitive behaviors compared to direct questioning, including excessive alcohol consumption and risky sexual behaviors (LaBrie & Earleywine, 2000; Walsh & Braithwaite, 2008), physical or sexual assault (Rayburn, Earleywine, & Davison, 2003) and hate-crime perpetration. Among US military, the UCT led to higher base rate estimation compared to anonymous self-report measures for problematic alcohol use (Sheppard, Forsyth, Earleywine, Hickling, & Lehrbach, 2013) as well as drinking and driving and carrying firearms (Sheppard & Earleywine, 2013). However, not all studies found this pattern. For example, the UCT did not produce statistically higher estimates of sexual assault among female undergraduate students (Krebs et al., 2011). Additionally, Droitcour and colleagues (1991) found higher estimates of IV drug use and unprotected sex when participants were asked directly compared to UCT-obtained estimates among a sample of randomly-selected residents of Dallas, Texas.

However, to the best of our knowledge, the UCT method has not been used to assess sensitive behaviors specifically among individuals with HIV/AIDS. This is an important gap in the literature. Researchers often rely on self-report measures to assess HIV risk behavior as well as other sensitive behaviors, yet the reliability and validity of self-report has frequently been questioned (Napper, Fisher, Reynolds, & Johnson, 2010). Therefore, this study sought to evaluate whether or not direct self-report (DSR) resulted in differential endorsement of sensitive behaviors compared to UCT-estimated base rates in an HIV+ sample. Additionally, for some behaviors (i.e., medication adherence), we compared UCT and DSR to an objective medication adherence measure obtained through an electronic medication-monitoring device, which served as a proxy for “true” behavior. We hypothesized that in our HIV+ sample: 1) the UCT would yield higher base rates of sensitive behaviors than DSR, and 2) the UCT would yield medication adherence estimates that more closely approximate an objective adherence measure than DSR.

Method

Participants

Participants included 229 HIV+ individuals enrolled in an ongoing study evaluating factors associated with medication adherence (R01 DA013799, PI: C. Hinkin) through the University of California, Los Angeles (UCLA). Participants were recruited from community-based organizations, infectious disease clinics, and medical centers in the greater Los Angeles area. All participants provided written, informed consent and received financial compensation for their time. The study was approved by the UCLA and West Los Angeles VA Institutional Review Boards (IRBs).

Materials

Unmatched count technique (UCT) Questionnaire

Participants completed a self-administered questionnaire using the unmatched count technique (UCT) designed by JLB. Participants were randomly assigned to receive either Form A (n = 126) or Form B (n = 103). Each questionnaire contained eight sets of items. For each set of items in both versions A and B, five identical, non-sensitive items were included. Additionally, for each set of items, one version of the form contained an extra item: the sensitive item of interest. (The version containing the sensitive item alternated by set.) For example, one set of items in Version A contained the following five, non-sensitive items:

1. I like watching the Jerry Springer television show.

2. I was the youngest child in my family.

3. Sometimes I have trouble sleeping and use a sleeping pill.

4. My T-cells have never been lower than 350.

5. My parents are divorced.

This set in Version B contained these same 5 items as well as one additional, sensitive item: “I have over-reported how good I am at taking my HIV meds to my doctor.”

Participants were instructed to read each statement in the set, and then write down how many of the statements were true for them. Participants simply provided a total, they did not endorse specific items. Since random assignment is used, the difference in mean scores between groups is thought to reflect the base rate for the sensitive item. To obtain the UCT, the mean score for the group completing the set without the sensitive item (i.e., 5 items total) is subtracted from the mean score for the group completing the set with the sensitive item (i.e., 6 items total). For this study, items of interest were related to medication adherence (i.e., missing medications, over-reporting adherence, reported taking medications when not) and high-risk sexual behavior in the past month (i.e., unprotected sex, unprotected sex with known HIV- partners).

Direct self-report (DSR)

All participants completed several self-report questionnaires. One questionnaire queried medication adherence, matching the items contained in the UCT (i.e., over-reporting HIV medication adherence to their doctor, telling their doctor that they are taking their HIV medications when they are not). Participants responded to these items in true/false format. Participants also completed a risk behavior questionnaire that queried instances of risk behavior, including the items assessed in the UCT (i.e., sex without a condom, sex without a condom after using alcohol/drugs, sex without a condom with HIV- partners). Finally, on a questionnaire assessing medication adherence, participants reported how many doses of their medication they missed in the last month. Participants were dichotomized into groups (i.e., reporting no vs. reporting one or more instances of the behavior), to mirror the UCT format.

Medication Event Monitoring System (MEMS) Cap

The MEMS cap (Aprex, Union City, CA) is an electronic pill bottle that records the dates and times when it is opened. Medication adherence rates obtained through the MEMS caps are often used as the “gold standard” for recent adherence. Upon study enrollment, patients were given a MEMS cap to use with one of their HIV medications for the next 30 days. Participants returned the MEMS cap 30 days after their study visit, and their adherence data was downloaded.

Demographic information

Participants completed a questionnaire assessing demographic information such as age, gender, ethnicity, and education and an interview about their medical history, including their date of HIV diagnosis.

Procedure

Statistical Analyses

Data were analyzed using the Statistical Package for the Social Sciences (SPSS) Version 22.0. One-sample z-tests were used to test hypothesis 1 (i.e., to compare the estimated base rates of sensitive behaviors obtained from the UCT versus DSR) and hypothesis 2 (i.e., to compare UCT adherence base rates, DSR rates, and objective adherence data collected from the MEMS Cap).

Results

Demographic and HIV-related characteristics for the study sample are presented in Table 1. Of note, we confirmed that individuals who received Form A of the UCT did not significantly differ from individuals who received Form B on any of these variables (all p's>.60).

Table 1.

Demographic and virological characteristics of the study sample (N = 229)

Demographic/Virological	M or %	SD
Age	42.02	6.95
Gender
Male	84
Female	16
Education	12.91	2.10
Ethnicity
African American	64
American Indian	<1
Asian/PI	3
Hispanic/Latino	13
Non-Hispanic white	17
Multiracial	2
Time since HIV diagnosis (years)	9.75	5.55
MEMS Adherence in following month (% who missed doses)	89%

First, we compared the mean number of items endorsed for each set between Forms A and Forms B to obtain the UCT estimate for engagement in recent risk behavior. Results showed that rates of sensitive behavior, as calculated from the UCT, ranged from 18 to 52% (See Table 2).

Table 2.

Deriving the UCT score from Forms A & B mean scores (N = 229)

	Group A (n=126)	Group B (n=103)	UCT
Variable	(M)	(M)	(%)
Missed HIV meds - in last month	3.49*	2.97	52
Missed HIV meds - due to alcohol/drugs	2.74	3.13*	39
Reported taking HIV meds to Dr. when they were not	2.64*	2.27	37
Over-reported HIV med adherence to Dr.	1.98	2.17*	19
Sex without a condom - in last month	2.76	2.94*	18
Sex without a condom - in last month, using drugs/alcohol	2.17*	2.11	6
Sex without a condom - with HIV-partners	2.27*	2.02	25

^*indicates form containing sensitive item.

Second, we compared the UCT-estimated base rates to the base rates obtained from DSR. Results, presented in Table 3, showed that DSR resulted in higher base rates for missing medications in the past month, missing medications in the last month due to alcohol or drugs, over-reporting adherence to one's doctor, and sexual risk behavior (i.e., unprotected sex in the past month and unprotected sex with HIV- partners). The UCT-estimated base rate was higher for only one item: telling the doctor that one is taking his/her HIV medication when he/she is not. Additionally, we confirmed that individuals who received Form A of the UCT did not significantly differ from individuals who received Form B on estimates obtained from DSR (all p's>.20).

Table 3.

Comparing UCT and direct self-report estimates of sensitive behaviors (N=229)

Variable	UCT (%)	DSR (%)	z	p
Missed HIV meds - in last month	52	73	6.36	<.01
Missed HIV meds - due to alcohol/drugs	39	46	2.17	.03
Reported taking HIV meds to Dr. when they were not	37	22	4.70	<.01
Over-reported HIV med adherence to Dr.	19	30	4.24	<.01
Sex without a condom - in last month	18	35	6.70	<.01
Sex without a condom – in last month, using drugs/alcohol	6	21	9.56	<.01
Sex without a condom - with HIV-partners	25	44	6.64	<.01

Finally, we compared both UCT and DSR estimates of medication adherence to objective data obtained from the MEMS cap. To mirror the UCT format, participants were dichotomized into those with perfect medication adherence vs. those who missed one or more doses of medication. Based on the MEMS data, 89% of the sample missed medication compared to just 11% who had perfect adherence. On one-sample z-tests, the MEMS rate significantly differed from both UCT (53%, z=17.90, p<.01) and DSR estimates (73%, z=7.74, p< 01).

Discussion

Direct self-report did not result in significantly lower endorsement of sensitive health-risk behaviors compared to the UCT technique among HIV+ participants, contrary to our hypotheses. Unexpectedly, direct self-report produced higher endorsement of sensitive behaviors. Additionally, the percentage of individuals who reported missing doses of their HIV medication in the last month when asked directly (73%) was closer to our objective estimate (i.e., MEMS data, 89%) than the estimate obtained from the UCT (52%), although both significantly differed. Importantly, this suggests that there was no significant decrement in responding to direct questioning due to the assumed sensitive nature of the questions. This may reflect specific characteristics of our population or possibly our study sample. For instance, our sample included HIV+ individuals who were willing to volunteer and participate in clinical research. This inherently requires participants to openly disclose their HIV status to study personnel, and as such, may deter individuals who find this uncomfortable. As such, it follows that the generalizability of our results may be limited to those who volunteer to participate in research or other disclosing situations. It is also possible that HIV+ individuals who seek to participate in HIV research may feel a personal responsibility to the larger HIV community, and may be more willing to respond openly to sensitive questions even when asked directly. Additionally, HIV+ individuals already have experience dealing with a highly stigmatized disease. Combined, these factors may select for individuals who are more willing to candidly discuss other sensitive topics. This would explain why the UCT technique has been effective in other populations with different characteristics, including populations in which frank discussion of illicit drug use or risky sexual behavior is more likely to be avoided.

Interestingly, however, direct self-report did produce significantly lower rates of endorsement compared to UCT estimates for one behavior: telling their physician that they are taking their HIV medications when they are not. We believe that this finding has important implications. It suggests that the UCT may still be warranted for targeting some behaviors among HIV+ individuals, and that identifying sensitive behaviors may differ depending on the individual's (or sample's) perceptions as well as the context. It seems likely that what one individual (or a group) deems to be sensitive or private, another may view as mundane or routine. In addition, contextual factors may also be important. This study focused on medication adherence in HIV, and our participants were asked to report their medication adherence to study personnel. Some participants who were reluctant to report poor adherence to their physician may have been similarly reluctant to report it to study personnel, given the similarities in social context and social roles between these situations. It is also possible that this specific question invokes a moral context that is not as strongly primed in the other questions. In this item, participants are asked explicitly if they have told their doctors something that is untrue. By essentially asking participants whether or not they have “lied” to their treatment provider, this item may evoke a stronger feeling of judgment, which may make admitting to this behavior more uncomfortable for participants than admitting to other behaviors that feel less socially undesirable. Therefore, the context may make this behavior particularly salient, and perceived as a more “sensitive” or potentially embarrassing topic than even behaviors such as unprotected sex or needle sharing. This would suggest that behaviors may not always be inherently sensitive, but rather that one's perception of sensitivity or sense or privacy may change depending on contextual cues. This should be investigated further, and would have interesting implications for future research; such findings could suggest that researchers should avoid priming or cueing participants about target sensitive behaviors.

It is unclear why UCT estimates were lower than DSR estimates for several items. This may simply reflect the degree of variance introduced by the UCT technique. UCT estimates necessarily have larger confidence intervals than SR estimates obtained within the same population (Engel, Jann, Lynn, Scherpenzeel, & Sturgis, 2014). The UCT may have greater cognitive demands than individual questions (i.e., requiring individuals to retain multiple items and formulate an aggregate response). Greater variance in rates of non-sensitive behaviors can reduce the statistical efficiency of the procedure (Biemer, Jordan, Hubbard, & Wright, 2005), and these rates are unknown. It is also possible that some participants detected the quasi-covert nature of the UCT approach, leading to defensive or guarded responding and reduced endorsement of sensitive behaviors. Finally, the UCT could have primed participants to report the risk behavior when asked using DSR.

Several factors limit the conclusions that can be drawn from this study. Contextual factors may be significant when assessing sensitive behaviors, and the nature of this study (i.e., examining medication adherence) may affect our estimates. Therefore, our results may differ from those obtained from a study in which target behaviors were not primed. On the other hand, this same limitation may increase our confidence in the results obtained for behaviors that were not primed, such as unprotected sex.

Although it is impossible for us to know how participants behave when they are not being observed, we were able to provide a proxy for “true” behavior through MEMS cap adherence rates. This provides a benchmark for comparing the UCT and DSR results. However, it is important to note that participants used the MEMS cap for 30 days following their study visit, while our self-report methods (DSR and UCT) inquired about medication adherence in the 30 days prior to the study. While we would expect these estimates to be highly correlated, and we can still consider the MEMS data to be a proxy for recent adherence, it is important to note that the exact timeframe differed. Additionally, as discussed earlier, our sample included HIV+ individuals who were willing to volunteer and participate in clinical research. As such, the generalizability of our results may be limited to those who volunteer for research or other disclosing situations. Finally, reporting could be affected by other factors that may vary across studies, such as whether or not the study is completely anonymous, whether the UCT is interviewer- or self-administered, and the length of the UCT questionnaire.

To our knowledge, this is the first study examining the UCT in assessing health risk behaviors specifically among HIV+ individuals, a population in which health risk behaviors can have particularly important consequences. Importantly, our results suggest that indirect assessment approaches such as the UCT may be unnecessary when assessing many sensitive behaviors in HIV+ adults. This is particularly salient given that may studies rely on self-report of risk behaviors and UCT techniques may be potentially difficult to carry out in some contexts. Because the UCT precludes exploration of behavior at the individual level, it limits the types of questions that can be explored and the ability to link individual behavior to other factors. These questions can be explored more extensively with direct self-report. However, it may be desirable when individual-level data is not needed. For instance, similar paradigms are often employed when screening potential research participants for eligibility (e.g., reading a set of exclusionary questions and asking if their response is yes to at least one item). Researchers, institutional review boards, and participants alike may favor approaches that minimize disclosure in these scenarios. While more work is needed to determine the contexts and populations that may call for the use of indirect methods such as the UCT, our results support that direct self-report is an appropriate method of estimating sensitive behaviors, including sexual risk behavior, among HIV+ participants.