ACASI Gender-of-Interviewer Voice Effects on Reports to Questions about Sensitive 
Behaviors Among Young Adults

Abstract

Although previous research indicates that audio computer-assisted self-interviewing (ACASI) yields higher reports of threatening behaviors than interviewer-administered interviews, very few studies have examined the potential effect of the gender of the ACASI voice on survey reports. Because the voice in ACASI necessarily has a gender, it is important to understand whether using a voice that is perceived as male or female might further enhance the validity associated with ACASI. This study examines gender-of-voice effects for a set of questions about sensitive behaviors administered via ACASI to a sample of young adults at high risk for engaging in the behaviors. Results showed higher levels of engagement in the behaviors and more consistent reporting among males when responding to a female voice, indicating that males were potentially more accurate when reporting to the female voice. Reports by females were not influenced by the voice’s gender. Our analysis adds to research on gender-of-voice effects in surveys, with important findings on measuring sensitive behaviors among young adults.

Introduction

Audio computer-assisted self-interviewing (ACASI)—by which respondents answer pre-recorded questions during an in-person interview—is the preferred method for administering sensitive questions in face-to-face interviews because it often yields higher reports of sensitive behaviors compared to CAPI or paper-and-pencil questionnaires (Tourangeau and Smith 1996; Turner, Ku, et al. 1998; Dykema, Basson, and Schaeffer 2007). Nevertheless, little research explores how the gender of the ACASI voice affects disclosure about sensitive information. This study evaluates the impact of the ACASI voice’s gender on responses about threatening behaviors among high-risk young adults.

Depending on a question’s topic, respondents may refer to gender-based stereotypes, conversational norms, or identities when responding (Tannen 1996; see also Schaeffer 2000). According to self-disclosure theory, individuals are expected to be more honest and disclose more to someone they trust and with whom they feel comfortable (Jourard 1971). Insofar as respondents hold stereotypes that women are more sympathetic (Pollner 1998) or nonjudgmental (Nass et al. 2003), respondents may disclose or report higher levels of sensitive behaviors to female interviewers (Dindia and Allen 1992). In contrast to self-disclosure theory, other researchers have offered “explanations of exaggeration,” which hold that higher levels of reporting may be less valid. For example, the “macho hypothesis” of Catania et al. (1996, p. 371) explains the higher levels of some sexual behaviors that males report to male interviewers as an effort to seem more virile and manly. In a related vein but predicting a different outcome, Weisel (2002, p. 102) argues that “a female interviewer may [have] inadvertently encourage[d] male interviewees to put on a macho bravado and exaggerate some points” in her study of contemporary gangs.

The gender of a live interviewer has had only limited effects in the studies in which it has been examined (Davis et al. 2010; Schaeffer, Dykema, and Maynard 2010).1 Most studies reported that disclosure among female respondents was greater when responding to a female interviewer (Catania et al. 1996; Dailey and Claus 2001; DeLamater 1974; Pollner 1998; but see Johnson and Parsons 1994). The pattern of reporting among male respondents is less consistent. In some studies, males disclosed more threatening behaviors to female interviewers (Dailey and Claus 2001; Pollner 1998), but in other studies they disclosed more to male interviewers (Catania et al. 1996; Fendrich et al. 1999; Johnson and Parsons 1994; Wilson et al. 2002).

Only one study, conducted exclusively with male respondents, directly tested the effect of the ACASI voice’s gender on survey reports.2 Fahrney, Uhrig, and Kuo (2010) explored the impact of a male versus female voice on reports of sexual activity among men who have sex with men (MSM). Their results were consistent with more accurate reporting among the males who heard questions read by a female voice.

Based on our review, we generated two predictions. First, we expected that both male and female respondents would be more likely to disclose to a female voice. Although findings for males in interviewer-administered studies are somewhat mixed, males and females appear to report more to females, and the one experimental study of gender-of-voice effects using ACASI demonstrated higher reporting among males to the female voice. Second, we expected that male respondents would be more affected by the voice manipulation than would female respondents, based on studies that demonstrate that gender-of-interviewer effects are more pronounced among men than women, indicating that there might be a ceiling effect for reporting among women (e.g., Catania et al. 1996; Pollner 1998). We believe that ours is only one of two studies that directly examines the effect of the ACASI voice’s gender on survey reports and is the first to compare this for male and female respondents.

Methods

Data

Data were provided by the Midwest Evaluation of the Adult Functioning of Former Foster Youth (Midwest Young Adult Study), a longitudinal CAPI study (Courtney, Terao, and Bost 2004). Youth in Illinois, Iowa, and Wisconsin were eligible for the study if they were in out-of-home care supervised by a public child welfare agency, between the ages of 17 and 17½, and had been in the care of the state for one or more years prior to their 17th birthday. Respondents received $25 for participating. Of the 957 eligible youth, 776 were interviewed (all by female interviewers), for an overall response rate of 81 percent (RR1, AAPOR 2011). Ten respondents were dropped because of missing data. Our analysis used reports by 384 male and 382 female respondents collected during wave 1 in 2002–2003.

Experiment

A section of potentially sensitive questions was administered using ACASI. The questions and response categories were pre-recorded using a female and male interviewer in their early twenties. Respondents were randomly and automatically assigned either the female or male voice for the entire module. Each question and its response categories were presented on the screen and read aloud through headphones. Answers could be entered only after the entire question and response categories were read. The audio presentation could not be muted, and only interviewers could adjust the volume.

We evaluated the characteristics of the voice by embedding audio clips containing a sample of each ACASI voice and asking raters questions about the voices. Raters included 56 (out of 96 invited) telephone interviewers employed at UWSC. All raters correctly identified the voice’s gender, and their evaluations of the male versus female voice’s race/ethnicity, educational attainment, and regional accent were not significantly different. Although raters assessed age higher for the male voice (24.6 versus 20.5 years, t = 6.95, p = .000), ratings for both voices were in their “early twenties.” Using a five-point scale, raters described the male voice as significantly less trustworthy (2.46 versus 3.29 scale points, t = -6.19, p = .000) and less friendly (2.84 versus 3.25 scale points, t = -2.77, p = .008) than the female voice.

Measures

ACASI questions were adopted from the National Longitudinal Study of Adolescent Health. We examined the effect of the voice’s gender on reports about 28 behaviors that captured sensitive topics ranging from less stigmatizing and frequently engaged-in behaviors (e.g., lying to parents), to illegal and rare violent and nonviolent criminal behaviors (e.g., shooting someone) (see table 1).

Table 1.

Question Wordings, Midwest Young Adult Study, 2002–2003

Questions using the following response categories: never, 1 or 2 times, 3 or 4 times, and 5 or more times

In the past 12 months, how often did you paint graffiti or signs on someone else’s property or in a public place?

In the past 12 months, how often did you deliberately damage property that didn’t belong to you?

In the past 12 months, how often did you lie to your parents or guardians about where you had been or whom you were with?

How often did you take something from a store without paying for it?

*How often did you get into a serious physical fight?

How often did you hurt someone badly enough to need bandages or care from a doctor or nurse?

How often did you run away from home?

How often did you drive a car without its owner’s permission?

In the past 12 months, how often did you steal something worth more than $50?

How often did you go into a house or building to steal something?

How often did you use or threaten to use a weapon to get something from someone?

How often did you sell marijuana or other drugs?

How often did you steal something worth less than $50?

*In the past 12 months, how often did you take part in a fight where a group of your friends was against another group?

How often were you loud, rowdy, or unruly in a public place?

During the past 12 months, how often did each of the following things happen? You saw someone shoot or stab another person.

Someone pulled a knife or gun on you.

Someone shot you.

Someone cut or stabbed you.

*You got into a physical fight.

You were jumped.

You pulled a knife or gun on someone.

You shot or stabbed someone.

Questions using the following response categories: yes and no

Have you ever been arrested?

Have you ever been convicted of a crime?

Have you ever spent a night in jail, prison, juvenile hall, or other correctional facility?

Continued

Table 1. Continued

Question using the following response categories: none, 1 day, 2 or 3 days, 4 or 5 days, 6 or more days

During the past 30 days, on how many days did you carry a weapon—such as a gun, knife, or club—to school?

Question using an open-ended response format

*During the past 12 months, how many times were you in a physical fight in which you were injured and had to be treated by a doctor or nurse?

Note.—In the analysis, respondents were coded as having engaged in the behavior if they answered “1 or 2” to “5 or more” times for the questions with grouped response categories; “yes” for yes/no questions; or 1 or more days or times for the questions about carrying a weapon and physical fights that had to be treated by a doctor or nurse. We mark questions about physical fighting that are included in the analysis of consistency in reporting about fighting with an asterisk (*).

For the analysis of the individual questions, we examined the effect of the ACASI voice’s gender (coded “1” if female versus “0” if male) on the proportion of respondents who reported engaging in the behavior. Because the questions used different response categories and some asked about rare behaviors, we dichotomized responses and coded answers as “1” if the respondent engaged in the behavior (versus “0” if they did not) (Turner, Ku, et al. 1998). Like previous researchers, we interpret reporting about engaging in the behavior as indicating more accurate reporting (Brener et al. 2006; Turner, Ku, et al. 1998). We also created an index that tallied the number of different behaviors respondents engaged in to test the overall effect of the voice across questions (Nass et al. 2003; Tourangeau, Couper, and Steiger 2003). Finally, we desired an indicator of improved reporting other than higher levels of reporting. The indicator we created captured the respondent’s consistency in reporting about fighting. Four questions asked explicitly about fighting (see table 1), and we examined how consistently respondents reported about these behaviors (with “1” coded as consistent, as shown in table 2).

Table 2.

Coding for Consistency in Reporting About Fighting, Midwest Young Adult Study, 2002–2003, N = 766

Questions and Response Categories
Take part in group fight		Got in physical fight		Get into serious physical fight		Fights treated by doctor/nurse		Response consistency		Sample size ( n )
Never		Never		Never		0 times		Consistent		168
Never		Never		Never		1 or more		Inconsistent		1
Never		Never		1 or more		0 times		Inconsistent		47
Never		Never		1 or more		1 or more		Inconsistent		4
Never		1 or more		Never		0 times		Consistent		40
Never		1 or more		Never		1 or more		Inconsistent		3
Never		1 or more		1 or more		0 times		Consistent		201
Never		1 or more		1 or more		1 or more		Consistent		28
1 or more		Never		Never		0 times		Inconsistent		10
1 or more		Never		Never		1 or more		Inconsistent		1
1 or more		Never		1 or more		0 times		Inconsistent		8
1 or more		Never		1 or more		1 or more		Inconsistent		1
1 or more		1 or more		Never		0 times		Consistent		13
1 or more		1 or more		1 or more		0 times		Consistent		175
1 or more		1 or more		1 or more		1 or more		Consistent		66

Note.—See table 1 for exact question wordings. Questions implied different levels of severity such that a respondent’s answer is inconsistent if he reported not engaging in a less serious incident of the behavior (e.g., got in a physical fight) but then reported affirmatively to a more serious incident of the behavior (e.g., got into a serious physical fight).

Analysis

We used logistic regression to estimate the difference in reports of the behavior by gender of the voice and report the odds ratios to test for significant differences. Models were estimated separately for males and females because of group differences in reporting of the behaviors and in the expected effects of the voice’s gender (Catania et al. 1996; Mensch, Hewett, and Erulkar 2003). For the index of behaviors, we performed a two-way ANOVA with gender of the voice and of the respondent as factors; we tested the interaction in this analysis because of the greater amount of information in the index. For the analysis of consistency in reporting about fighting, we regressed consistency on the voice’s gender; we examined effects among the full sample and also for a subsample who reported engaging in at least one of the fighting behaviors.

Results

Analysis of Survey Responses

Results for the individual items are presented in table 3. The levels of behaviors varied considerably between male and female respondents and across the behaviors. Males were more likely to report engaging in behaviors to the female voice for 27 of the 28 behaviors, though only six of these comparisons were significant or marginally significant: getting into a serious fight (OR = 1.66, p < .05), going into a building to steal (OR = 1.59, p < .10), threatening to use a weapon (OR = 1.64, p < .10), selling drugs (OR = 1.66, p < .05), spending a night in jail (OR = 1.50, p < .05), and getting into a physical fight (OR = 1.61, p < .05). There was no discernible pattern in reporting among females, and only reports about shooting or stabbing were higher when reporting to a female voice (OR = 2.48, p < .10).

Table 3.

Percent Reporting Engaging in the Behavior and Odds Ratios by Gender of Voice for Male and Female Respondents, Midwest Young Adult Study, 2002–2003, N = 766

Behaviors	Male Respondents				Female Respondents
	Female Voice	Male Voice	Odds Ratio	95% CI	Female Voice	Male Voice	Odds Ratio	95% CI
Paint graffiti	11.1	11.4	0.97	0.51–1.83	4.6	6.7	0.67	0.27–1.63
Damage property	30.7	24.3	1.38	0.88–2.16	13.2	19.2	0.64	0.37–1.12
Lie to parents/guardians	66.8	63.8	1.14	0.75–1.74	64.9	61.5	1.16	0.76–1.76
Take something from store	51.8	44.9	1.32	0.88–1.97	36.8	38.5	0.93	0.61–1.41
Get into serious fight	78.4	68.6	1.66*	1.05–2.62	62.6	66.3	0.85	0.56–1.30
Hurt someone	54.3	45.9	1.40	0.93–2.09	28.7	27.4	1.07	0.68–1.67
Run away	43.2	43.2	1.00	0.67–1.50	44.8	47.1	0.91	0.61–1.37
Drive without permission	20.6	20.0	1.04	0.63–1.71	11.5	15.4	0.71	0.39–1.30
Steal something greater than $50	18.1	12.4	1.56	0.88–2.74	5.7	5.3	1.09	0.45–2.64
Go into building to steal	22.1	15.1	1.59+	0.94–2.69	5.2	7.7	0.65	0.28–1.52
Threaten to use weapon	19.6	13.0	1.64+	0.94–2.84	4.0	7.7	0.50	0.20–1.25
Sell marijuana/drugs	36.2	25.4	1.66*	1.07–2.58	11.5	16.8	0.64	0.36–1.16
Steal something less than $50	46.2	42.7	1.15	0.77–1.73	31.6	33.2	0.93	0.61–1.43
Take part in group fight	46.2	38.9	1.35	0.90–2.03	31.6	26.4	1.29	0.82–2.00
Be in a loud public place	49.7	47.0	1.12	0.75–1.66	52.3	47.1	1.23	0.82–1.84
Been arrested	65.3	60.5	1.23	0.81–1.86	42.5	42.3	1.01	0.67–1.52
Convicted of a crime	35.2	28.6	1.35	0.88–2.08	15.5	16.3	0.94	0.54–1.63
Night in jail	52.3	42.2	1.50*	1.00–2.25	24.1	27.4	0.84	0.53–1.34
Saw someone shoot/stab	28.1	22.2	1.38	0.86–2.19	16.1	17.3	0.92	0.53–1.57
Knife/gun pulled on respondent	37.7	36.8	1.04	0.69–1.57	18.4	18.8	0.98	0.58–1.64
Respondent shot by someone	8.0	4.3	1.93	0.81–4.63	0.6	0.5	1.20	0.07–19.27
Respondent stabbed by someone	21.6	20.0	1.10	0.67–1.81	10.3	9.6	1.08	0.55–2.12
Got in physical fight	77.9	68.6	1.61*	1.02–2.54	59.8	67.3	0.72	0.47–1.10
Respondent was jumped	45.2	42.2	1.13	0.76–1.70	20.7	23.1	0.87	0.53–1.42
Respondent pulled knife/gun	19.6	15.7	1.31	0.77–2.22	8.6	8.7	1.00	0.49–2.04
Respondent shot/stabbed someone	9.5	5.4	1.85	0.84–4.08	5.7	2.4	2.48+	0.83–7.39
Carried weapon to school	7.5	4.9	1.59	0.68–3.74	5.2	4.8	1.08	0.43–2.72
Fight treated by doctor	15.6	12.4	1.30	0.73–2.32	13.2	13.0	1.02	0.56–1.85

Note.— *p < .05; +p < .10. Behaviors are coded so that “1” indicates the behavior was engaged in at least once, and “0” indicates it was not. Significant values are in bold.

The index showed a significant interaction between the gender of the voice and that of the respondent, F(1, 762) = 4.82, p < .05 (see figure 1). Post-hoc analyses (ANOVA) indicated that males reported engaging in significantly more behaviors when reporting to a female voice, F(1, 382) = 5.28, p < .05, but the gender of the voice did not affect disclosure among females, F(1, 380) = 0.37, p > 0.30. There was also a main effect for gender, F(1, 762) = 58.08, p < .001, with males reporting engaging in significantly more behaviors than females.

Figure 1.

Mean Number of Questions for Which Respondents Indicated Engaging in the Behavior at Least One Time.

Analysis of Consistency in Reporting about Fighting

Table 4 presents results for the analysis of consistency in reporting about fighting. In the full sample (shown in the first row), the odds of reporting consistently were higher for males reporting to the female voice, but the effect was not significant. The reverse pattern was shown for female respondents but was not significant. When the sample was limited to respondents who reported engaging in at least one of the fighting behaviors (second row), the odds of reporting consistently were marginally higher (OR = 1.82, p < .10) for males who reported to the female voice. Females reporting to the female voice again showed the reverse pattern, which was not significant.

Table 4.

Percent Reporting Consistently About Fighting and Odds Ratios by Gender of Voice for Male and Female Respondents, Midwest Young Adult Study, 2002–2003

	Male Respondents				Female Respondents
	Female Voice	Male Voice	Odds Ratio	95% CI	Female Voice	Male Voice	Odds Ratio	95% CI
All respondents (N = 766)	92.5	88.1	1.66	0.83–3.30	87.9	91.8	0.65	0.33–1.27
Respondents who reported fighting 1 or more times (N = 598)	91.2	85.0	1.82+	0.91–3.65	83.1	89.2	0.60	0.30–1.19

Note.—+p < .10. Significant results are in bold.

Discussion

Our prediction that both male and female respondents would be more likely to disclose to a female voice received support only for the male respondents. Among male respondents, 27 of the 28 behaviors examined showed a consistent pattern of higher rates of reporting when males provided answers to the female voice. The effects were significant (p < .05) or marginally significant (p < .10) for six behaviors (i.e., getting into a serious fight, going into a building to steal, threatening to use a weapon, selling marijuana or other drugs, spending a night in jail, and getting into a physical fight). Further, a test of the effect of the voice’s gender on the index summed across questions indicated a significant interaction between the gender of the respondent and that of the voice, with a higher average number of different behaviors reported by male respondents when the recorded voice was female rather than male. Thus, we found support for the prediction that male respondents would be more affected by the voice manipulation than would female respondents.

Speculation about why we found gender-of-voice effects among only male respondents must address both why females were not affected and why males were. There may be less opportunity for the gender of the interviewer (or their voice) to have an effect among females for two reasons. First, females engaged in most of these behaviors at relatively lower levels, and second, as Catania et al. (1996) argued, women tend to disclose at higher levels (e.g., Dindia and Allen 1992). On average, males both have more behaviors to report and disclose them at lower levels, so that if males have higher levels of self-disclosure to females than to males (Dindia and Allen 1992; Hill and Stull 1987), that difference is more likely to be detected. One mechanism for males reporting differently to females is suggested by research in law enforcement and criminal justice that finds that female interrogators may be more effective in eliciting information about criminal activity from males (Hunt 1984; Weisel 2002). At the time of the study, many male respondents had already had encounters with law enforcement personnel or spent at least one night in jail. It is conceivable that the young, at-risk males in our experiment may have ascribed greater authority to the male ACASI voice, which in turn caused them to be wary about reporting the behaviors for fear of punishment. Support for this conclusion is also provided by our evaluation of the voice’s characteristics in which raters judged the male voice as less friendly and trustworthy than the female voice. We find no support for the macho hypothesis.

Based on the belief that sensitive behaviors are underreported, we have adopted the approach that higher levels of reporting produced under one experimental condition are likely to be more accurate (e.g., Turner, Ku, et al. 1998).3 In a recent study, Kreuter, Presser, and Tourangeau (2008) reported that higher levels of reporting for socially undesirable behaviors and characteristics were associated with more accurate reports when responses were compared to validation data. In our data, we examined the impact of the voice’s gender on consistency in reporting about fighting. Among males who reported fighting, the odds of reporting consistently were 82 percent higher when responding to a female voice, which we take as evidence of more accurate reporting. Males reported more behaviors to the female voice, and their responses were more consistent.

Our study was limited in several regards. First, respondents were youth from foster care and had engaged in many of the high-risk behaviors. Although the levels of their behaviors may have increased our ability to detect statistically significant differences, our results may not be generalizable to other populations. Second, we manipulated only the ACASI voice’s gender and have no way of knowing what impact other social (e.g., age) or voice (e.g., pitch) characteristics might have had. Given that our independent raters deemed the male voice less trustworthy and friendly, more research is needed to disentangle the effects of gender from other characteristics. Third, we do not have data to validate our assumption that higher reports of sensitive behaviors indicate better data quality.

Most ACASI studies use a female or synthetic voice (Couper, Singer, and Tourangeau 2004). Our results provide support for this convention. As noted by Fahrney, Uhrig, and Kuo (2010), however, studies of MSM match male respondents with male ACASI voices to improve reporting (see Wolitski et al. 2005). Their findings and our results suggest that this may lead to underreporting. More research is needed to determine whether gender-of-voice effects in ACASI vary across topics and populations. Such research is important for populations (such as foster care youth and MSM) that are at high risk of engaging in sensitive behaviors. Coupling methodological inquiries with data to validate responses and determine whether higher reports are more accurate is critical.