SAME-GENDER SEX IN THE UNITED STATES IMPACT OF T-ACASI ON PREVALENCE ESTIMATES

Abstract

Well-conducted telephone surveys provide an economical means of estimating the prevalence of sexual and reproductive behaviors in a population. There is, however, a nontrivial potential for bias since respondents must report sensitive information to a human interviewer. The National STD and Behavior Measurement Experiment (NSBME) evaluates a new survey technology—telephone audio computer-assisted self-interviewing (T-ACASI)—that eliminates this requirement. The NSBME embedded a randomized experiment in a survey of probability samples of 1,543 U.S. and 744 Baltimore adults ages 18 to 45. Compared with NSBME respondents interviewed by human interviewers, respondents interviewed by T-ACASI were 1.5 to 1.6 times more likely to report same-gender sexual attraction, experience, and genital contact. The impact of T-ACASI was more pronounced (odds ratio = 2.5) for residents of locales that have historically been less tolerant of same-gender sexual behaviors and for respondents in households with children (odds ratio = 3.0).

Until recently, telephone surveys could not offer respondents a fully private mode of response. Even in surveys of sensitive topics, such as sexual behavior and drug use, respondents in telephone surveys were required to report their behaviors to a human telephone interviewer. In-person surveys, however, have used paper self-administered questionnaires (paper SAQs) and, more recently, audio computer-assisted self interviewing (audio-CASI) to provide a fully private interview mode when asking questions about sensitive topics. A growing literature indicates that respondents in such surveys are more likely to report engaging in illicit, stigmatized, or embarrassing behaviors when questions are asked on a paper SAQ or by audio-CASI rather than by a human interviewer (e.g., Jones and Forrest 1992; Miller et al. 1999; O’Reilly et al. 1994; O’Reilly and Turner 1992; Tourangeau and Smith 1996; Turner, Lessler, and Devore 1992; Turner, Ku et al. 1996; Turner et al. 1998).

While telephone surveys have not been able to offer a fully private mode of response and have other drawbacks—most notably, lower response rates than well-conducted in-person surveys—they do offer several advantages over in-person surveys. In particular, they

• are considerably less expensive than in-person surveys,

• can be conducted more rapidly,

• do not require geographic clustering of samples to reduce interviewer travel, as found in the multistage area probability samples typically used in personal visit surveys (with a resultant loss in sample precision), and

• permit systematic monitoring of the quality of interviewing.

For these reasons, many large-scale surveys of AIDS-related sexual behaviors and drug use have been conducted by telephone in the United States and elsewhere (e.g., Analysis of Sexual Behavior in France 1992; Catania et al. 1992; Davis et al. 1993; Lau, Tang, and Tsui 2003). Careful work by Aquilino (1994) and Gfroerer and Hughes (1992) suggests, however, that telephone surveys elicit less frequent reports of illicit drug use and potentially other sensitive behaviors than in-person surveys that use private modes of questioning such as paper SAQs.

Telephone Audio-CASI

Telephone audio-CASI (T-ACASI) technology implements audio-CASI over the telephone.¹ In the winter of 1994–95, Cooley and other scientists at Research Triangle Institute (RTI) extended their in-person audio-CASI system to allow it to conduct complex call-in or call-out T-ACASI surveys (Cooley et al. 1996, 2000; Cooley and Turner 1998). In 1996 Turner, Miller et al. reported promising results from pilot testing of T-ACASI, including the increased reporting of a variety of sensitive sexual behaviors such as anal sex. Subsequently, scientists at the University of California, San Francisco and RTI used this technology in the 1996–98 Urban Men’s Health Study—a survey of probability samples of men who reported sex with men in four U.S. cities. Respondents assigned to T-ACASI in this study were more likely to report illicit drug use than those who were interviewed by human interviewers, but there was also a higher rate of interview break-offs in the T-ACASI condition (Cooley et al 2000; Gribble et al. 2000).

Later studies generally confirmed the finding that T-ACASI appears to improve the accuracy of reporting of sensitive behaviors while increasing the risk of respondent break-offs (Blumberg et al. 2003; Corkrey and Parkinson 2002; Millard and Carver 1999; Tourangeau, Steiger, and Wilson 2002). However, with the exception of Corkrey and Parkinson’s work in Australia, generalizations from these studies are restricted by methodological limitations—for example, small sample sizes, use of specialized populations, and low response rates. More recently, Currivan et al. (2004) and Moskowitz (2004) have reported results of randomized experiments that found increased reporting of smoking among probability samples of adolescents in California and Massachusetts when respondents were interviewed by T-ACASI. Inferences from the California experiment, however, are compromised by the finding that the reported prevalence of youth smoking was equivalent when sample weighting was used to control for differences in sample composition (Moskowitz 2004, table 4, p. 580).

Table 4.

Estimated Prevalence among U.S. Men and Women Ages 18 to 45 of Same-Gender Sexual Attraction, Behaviors, and Attitude

Behavior or Attitude	Total			Males			Females			Female:Male
	%	SE	n	%	SE	n	%	SE	n	OR	pa
Any same-gender attraction	16.1	1.6	(617)	11.9	2.2	(269)	20.3	2.3	(348)	1.9	< .05
Any same-gender sexual contact	12.6	1.4	(617)	9.1	1.9	(269)	16.2	2.2	(348)	1.9	< .05
Any same-gender genital contact
Ever	8.9	1.2	(617)	8.2	1.7	(269)	9.6	1.8	(348)	1.2	0.570
Past five years	5.4	1.0		4.3	1.3		6.6	1.4		1.6	0.239
Past year	3.7	0.7		2.6	0.9		4.7	1.2		1.8	0.176
Attitude toward same-gender sexb
Always wrong	54.1	2.3	(579)	60.5	3.3	(253)	47.7	3.1	(326)	0.6	< .05
Almost always wrong	7.6	1.2		8.5	1.8		6.7	1.7		0.8
Wrong only sometimes	11.9	1.4		9.2	1.9		14.6	2.0		1.7
Not wrong at all	26.5	2.0		21.8	2.7		31.1	2.8		1.6

Note.—Calculated using weighted data from T-ACASI interviews in national sample stratum of 1999–2000 NSBME. Sample data are weighted to adjust for varying probabilities of selection and include a poststratification adjustment to ensure that survey sample distribution matched the 1999 census population estimates for the national population (ages 18 to 45) by gender, race, and age. Standard errors and p-values were calculated using algorithms (STATA Corporation 2000) that take account of the impact of weighting and complex sample design on variance estimates. (Ns shown in the table are unweighted sample sizes for denominators.)

^aP-value for Pearson chi-square test of the hypothesis that response distributions were equivalent for males and females.

^bORs for attitude toward same-gender sex contrast female and male responses to one response category (versus all others).

The present article reports results from the 1999–2000 National STD and Behavioral Measurement Experiment (NSBME), which extended evaluation of T-ACASI to surveys of the adult population of the United States. The NSBME randomly assigned a probability sample of U.S. adults ages 18 to 45 to have sensitive questions asked either by a human telephone interviewer in a traditional computer-assisted telephone interview (CATI)² or by T-ACASI. In this article we report NSBME findings for survey questions asking about same-gender sexual attraction, experience, genital contact, and attitudes toward same-gender sex.

Data and Methods

SAMPLE DESIGN

The NSBME was embedded in a telephone survey of a probability sample of women and men aged 18 to 45 years residing in U.S. households with working landline telephones. The survey was conducted between September 1999 and April 2000. Two sample strata were recruited for this survey measurement experiment: (1) a sample of the telephone-accessible U.S. household population aged 18 to 45 (national stratum), and (2) a parallel sample of the population of the city of Baltimore, MD (Baltimore stratum).³ A list-assisted random digit dial (RDD) sample was drawn for each stratum using the Genesys Sampling System (2002).

SAMPLE EXECUTION

For the national stratum, 14,250 telephone numbers were generated, and 12,322 telephone numbers (86.5 percent) were successfully screened for eligibility (Roman 2000).⁴ Of these screened telephone numbers, 2,183 were found to be residential numbers with one or more eligible English-speaking respondents aged 18 to 45. One eligible household member of these households was randomly selected for participation in the survey (without substitution). Of the 2,183 target respondents in the national strata, 1,452 completed interviews (66.5 percent),⁵ and 91 respondents (4.2 percent) completed partial interviews that included at least one substantive questionnaire section. A maximum of 91 calls per household were made to screen households and complete an interview in the national stratum. Ten percent of all completed interviews required 25 or more calls. Of 709 completed interviews assigned to be conducted in the T-ACASI experimental condition, 24 (3.4 percent) were administered in CATI mode because respondents did not have a touchtone telephone. (Data for these 24 cases are excluded from our analyses.) Interview completion rates were higher in the CATI condition than in T-ACASI (71.5 percent versus 61.3 percent). Data for 10 of the 1,452 completed interviews in the national stratum were lost due to a data processing error.

The second sample stratum was drawn to represent the adult population of Baltimore, MD. For the Baltimore stratum, 7,498 telephone numbers were generated, and 6,326 (84.4 percent) were successfully screened for eligibility. Screening identified 1,072 households with an eligible respondent, and 697 of these eligible respondents completed interviews (65.0 percent). An additional 47 respondents (4.4 percent) completed partial interviews. A maximum of 82 calls per household were made to screen households and complete an interview in the Baltimore stratum. Ten percent of all completed interviews required 26 or more calls. Of 332 completed Baltimore interviews assigned to be conducted in the T-ACASI experimental condition, 15 (4.5 percent) were completed in CATI mode because respondents did not have a touchtone telephone. (These 15 cases are excluded from our analyses.) As in the national stratum, interview completion rates were higher in the CATI condition, but the difference was smaller in magnitude (67.7 percent versus 62.3 percent).

Roman (2000, pp. 23–29) has calculated the American Association for Public Opinion Research (AAPOR 2000) response rates (formula RR3) to be 62 percent for the CATI condition and 53 percent for the T-ACASI condition in the national strata.⁶ In the Baltimore strata these response rates were 56 percent for the CATI condition and 50 percent for the T-ACASI condition.

INTERVIEW MODES

Telephone numbers were randomly assigned to the CATI or T-ACASI conditions prior to their release to the telephone survey unit. Following screening and recruitment into the study, telephone interviewers at the University of Massachusetts conducted the survey either by asking the respondent questions and recording their answers (CATI condition) or by transferring the respondent to a T-ACASI system at the Research Triangle Institute in North Carolina. Both male and female interviewers were used in both conditions, and no attempt was made to match the gender of interviewers and respondents.

For T-ACASI interviews, the telephone interviewer established a three-way connection between the respondent, the T-ACASI system, and the interviewer. After the T-ACASI interview began, the interviewer remained on the line for the first nine, nonsensitive questions on demographic characteristics and antibiotic use to ensure that the system was working properly and that the respondent was comfortable using the touchtone telephone’s keypad to answer questions. The interviewer then informed the respondent that he/she was leaving the three-way connection so that the respondent could complete the survey in private. Respondents in the T-ACASI mode could return to an interviewer for assistance or access a help menu using the telephone keypad at any time during the interview. In both T-ACASI and CATI conditions respondents were able to refuse to answer individual items, and substantive responses were not probed in either condition.⁷ (The full text of the survey introduction is presented in the online appendix.)

NONINTERVIEWS AND INTERVIEW BREAK-OFFS

Of the household respondents screened and selected for interview, 442 persons in the CATI condition and 526 in the T-ACASI condition were classified as noninterviews. These noninterviews included interview refusals, noncontacts with the sampled person, cases exceeding the survey callback limit, and interviews that were discontinued without completing the initial set of 14 items on alcohol and drug use. Appendix table B1 (available online) provides a detailed accounting of noninterviews and interview break-offs for both experimental conditions.

Table B1.

Noninterviews and Interview Break-Offs by Mode of Interview

Survey Stagea	N		Interview Break-Offs
	CATI	T-ACASI	CATI	T-ACASI
Screened and eligible for interview	1,656	1,599
Noninterviews (refused, no contact, call limit reached, interview break-off before end of alcohol and drug module, questions 16–17)	442	526
Interviews excluded because case assigned to T-ACASI but conducted by T-IAQb	0	39
Interview data lost due to T-ACASI computer malfunction	0	10
Competed alcohol and drug module (maximum of 14 questions)	1,214	1,024	N/A	N/A
Completed sexual behavior module (maximum of 34 questions)	1,193	986	21	38
Completed STD module (maximum of 33questions)	1,188	948	5	38
Completed heterosexual behavior module (maximum of 8 questions)	1,172	947	16	1
Completed same-gender sex module (maximum of 4 questions)	1,166	938	6	9
Completed forced and paid sex module (maximum of 4 questions)	1,164	938	2	0
Completed violence and crime module (maximum of 7 questions)	1,163	928	1	10
Completed next 11 questions on other sensitive behaviors and attitudes	1,159	880	4	48
Answered final substantive question (attitude toward same-gender sex)	1,120	872	39	8

^aAll counts of number of questions are the maximum number asked. Because of skip patterns, the actual number of questions asked of a given respondent may be less than this maximum. For example, if a respondent reported that they had never had an alcoholic drink, they would not be asked more detailed questions about alcohol use.

^bThis occurred because the respondent did not have a touchtone telephone.

A total of 1,214 respondents in the CATI condition and 1,024 respondents in the T-ACASI condition began the survey interview and completed the initial set of 14 questions on alcohol and drug use. These cases were classified as completed interviews (either partial or full), and they were retained in the survey analysis file. Of these respondents, 42 in the CATI condition and 77 in the T-ACASI condition discontinued their interviews prior to reaching the questions on same-gender sexual attraction and behaviors. An additional 6 respondents in the CATI condition and 9 respondents in the T-ACASI condition discontinued their interview while being asked these questions. A detailed breakdown of interview break-offs and noninterviews in both experimental conditions is presented in appendix B (available online).

RECONTACTING T-ACASI BREAK-OFFS

Survey staff attempted to recontact all T-ACASI respondents who broke off or were disconnected from the T-ACASI portion of the interview.⁸ (Details of the recontact procedures are presented in appendix B, online.) Two hundred and seventy-eight T-ACASI break-offs were successfully recontacted and were requested to complete their interview. Responses of these respondents indicate that approximately one-half the T-ACASI break-offs were due not to respondent termination of the interview but rather to an apparent T-ACASI or telecommunications malfunction or to the respondent taking another call using call-waiting. Appendix B (online) provides additional details of the reported reasons for interview break-offs in the T-ACASI condition.

INCENTIVES

As the study progressed, it became obvious that the T-ACASI break-off rates were going to be considerably higher than CATI break-off rates. To address this concern, it was decided to offer T-ACASI break-offs a $25 incentive to complete their interview. We selected 130 break-off cases, from among those cases available late in the study, who (1) had been connected to T-ACASI, (2) had not yet refused three times, and (3) had not been declared “final” due to extensive prior efforts to have them to complete the survey. We were able to successfully contact 93 of the 130 (71.5 percent) respondents. Of those contacted, 48 (51.6 percent) took the money and completed the survey, 34 (36.6 percent) refused the payment and would not complete the survey, and 11 (11.8 percent) did not refuse but still never completed the survey.

Since the payment of an incentive to T-ACASI break-offs introduces a non-equivalence in the experimental design, table 1 (note e) presents estimates of the survey mode effect in which these respondents are excluded from the analysis. The exclusion (or inclusion) of these cases does not have a substantial impact on estimates of the survey mode effect in the measurement of reported same-gender attraction and contact.

Table 1.

Percentage of Respondents Reporting Same-Gender Sexual Attraction, Experience, Genital Contact, and Attitude by Interview Mode

	T-ACASIe		CATI		Crude OR	Adjusted ORb
	%	nf	%	nf
Any same-gender attraction	17.8	933	12.8	1,158	1.5**	1.6***
Any same-gender sexual experience	14.2	931	9.1	1,164	1.6***	1.8***
Any same-gender genital contact
Ever	10.3	931	7.0	1,164	1.5**	1.7**
Past five years	7.0	931	4.5	1,164	1.6*	1.8**
Past year	5.3	931	3.4	1,164	1.6*	1.9**
Attitude toward same-gender sexc
Always wrong	49.1	872	46.0	1,120	1.1	1.2
Almost always wrong	7.5		5.3		1.4*	1.5*
Wrong only sometimes	12.7		11.0		1.2	1.2
Not wrong at all	30.7		37.8		0.7**	0.7***

Note.—Unweighted data from 1999–2000 NSBME, combined national and Baltimore sample strata.^a,d,e

^aAnalysis excludes 39 respondents assigned to T-ACASI who were interviewed in CATI because they did not have a touchtone telephone.

^bAdjusted ORs for mode effect derived from logistic regression model controlling for sample strata, gender, age group, educational level, marital status, presence of children in the household, race/ethnicity, region of country, and level of urbanization. (Covariate controls entered as categorical dummy variables using categories as shown in table 3; all respondents in Baltimore sampling strata were coded as residing in the South Atlantic and in one of the 21 largest MSAs [Baltimore-Washington MSA].)

^cLikelihood ratio chi-square test of the null hypothesis that response distributions across the four answer categories were equivalent for T-ACASI and CATI (using the 2 × 4 cross-tabulation of attitude by interview mode) is rejected with p = .005. Crude and adjusted ORs for attitude toward same-gender sex contrast responses for T-ACASI versus CATI for one response category (versus all others).

^dOther analyses (not shown) indicate that when the sample is restricted to respondents who completed the final section of the questionnaire, which required a touchtone phone in both experimental conditions, estimates of the impact of T-ACASI on reporting of same-gender sexual contact vary only slightly (crude ORs of 1.5 for any same-gender experience, 1.4 for genital contact in lifetime, 1.4 for genital contact in past five years, 1.5 for genital contact in past year).

^eEstimates of survey mode effects vary insubstantially when our analysis eliminates the 48 T-ACASI break-off cases who were paid $25 to complete their interviews. Dropping these cases, adjusted OR estimates are 1.6 for same-gender attraction, 1.7 for same-gender sexual experience, 1.6 for same-gender sexual experience in lifetime, 1.8 in past five years, 1.8 in past year, and 0.7 for reporting that same-gender sex is not wrong at all.

^fExcluding interview break-off (see appendix B), item nonresponse frequencies were 5 in T-ACASI and 8 in CATI for reporting of same-gender attraction; 7 in T-ACASI and 3 in CATI for reporting same-gender sexual experience; and 7 in T-ACASI and 3 in T-IAQ for reporting of same-gender genital contact. These frequencies exclude interview break-offs (i.e., cases in which the respondent discontinued providing substantive responses). The question on attitudes toward same-gender sex was the final substantive question in the survey. In the T-ACASI condition, 8 respondents who completed the preceding question either refused to answer this question or discontinued the interview. In the CATI condition, 43 respondents were coded by the interviewers as “refusals.”

^*p < .05.

^**p < .01.

^***p.< .001.

SURVEY CLOSEOUT

After the main survey was completed, respondents in both CATI and T-ACASI conditions were asked to complete a brief series of questions on their reactions to the survey. These questions were administered by T-ACASI. Since this series of questions was not central to the survey experiment, respondents who broke off the interview at or after this point were not recon-tacted to complete this section of the survey. Of the 2,139 respondents who completed the main survey (excluding 10 cases with lost data), 1,821 completed this final section that asked respondents about their reactions to the survey.

ENSURING SAMPLE EQUIVALENCE ACROSS INTERVIEW MODES

While randomization should ordinarily yield approximately equivalent groups, the T-ACASI condition had higher noninterview and interviewer break-off rates than the CATI condition—particularly in the national sample strata. In addition, some divergence in sample composition might be expected because the CATI sample included an estimated 3 percent of cases that did not have a touchtone phone.⁹ Although these cases have been excluded from the T-ACASI sample in our analysis, they could not be excluded from the CATI sample. (A last-minute survey modification resulted in the accidental elimination of a screening question that asked about the availability of touchtone telephones.)

Four strategies were employed to deal with potential nonequivalence of samples in the two modes: (1) the demographic equivalence of the samples was tested; (2) mode effects were estimated separately using the 1,821 CATI and T-ACASI respondents who completed the T-ACASI follow-up section and thus were known to have touchtone phones (see table 1, note d); (3) the statistical tests for mode effects (table 1) use both crude odds ratios (OR) and estimates of mode effects derived from logistic regressions incorporating controls for nine sociodemographic factors (sample strata, gender, age group, educational level, marital status, presence of children in household, race/ethnicity, region of country, and level of urbanization); and (4) a sensitivity analysis (table 2) was performed to estimate the minimum size of the T-ACASI mode effect under fairly extreme assumptions about the potential impact of T-ACASI interview break-offs and noninterviews.

Table 2.

Sensitivity Analysis: Estimates of Mode Effects (crude ORs) for Reporting of Same-Gender Sexual Attraction and Experience under Alternative Assumptions about the Impact of Nonresponse in T-ACASI Condition

Measurement	Model 1: Reported Data					Model 2			Model 3
	CATI	Observed	T-ACASI	Observed	Crude	T-ACASI	Assumed	Crude	T-ACASI	Assumed	Crude
	Estimate	N	Estimate	N	OR	Estimate	N	OR	Estimate	N	OR
Any same-gender sexual attraction	12.8%	1,158	17.8%	933	1.4	17.2%	968	1.3	15.8%	1,052	1.2
Any same-gender sexual experience	9.1%	1,164	14.2%	931	1.6	13.7%	966	1.5	12.6%	1,050	1.4
Any same-gender genital contact	x	x	x	x	x	x	x	x	x	x	x
Ever	7.0%	1,164	10.3%	931	1.5	9.9%	966	1.4	9.1%	1,050	1.3
Past five years	4.5%	1,164	7.0%	931	1.6	6.7%	966	1.5	6.2%	1,050	1.4
Past year	3.4%	1,164	5.3%	931	1.6	5.1%	966	1.5	4.7%	1,050	1.4

Note.—Model 1 (the reported data) assumes that nonrespondents in T-ACASI would have reported the same prevalence of same-gender attraction, experience, and genital contact as respondents. Under model 1 the prevalence estimates derived from the reported data are not adjusted. Model 2 assumes that all T-ACASI respondents who discontinued the interview after the first sensitive module of questions on alcohol and drug use would have reported no same-gender attraction or behaviors if they had not broken off their interview. Model 2 derives prevalence estimates by inflating the T-ACASI denominators by 35 to equalize the number of persons in CATI (n = 42) and T-ACASI (n = 77) who completed the first survey module but then broke off the interview before reaching the questions on same-gender sex. Model 3 derives prevalence estimates by inflating the T-ACASI denominator by 119 to equalize the total number of noninterviews and break-offs in T-ACASI (526 + 77 = 603) and CATI (442 + 42 = 484), again assuming that none of these 119 additional T-ACASI cases would have reported same-gender attraction or behaviors if they had been interviewed.

SURVEY MEASUREMENTS

The survey questionnaire used in the NSBME was identical, with a few exceptions, to that used in the 1997–98 Baltimore STD and Behavior Survey (BSBS; Turner et al. 2002). Survey questions were adapted from past large-scale surveys of sexual behavior, including the 1992 National Health and Social Life Survey (NHSLS) (Laumann et al. 1994), the 1987–1995 National Surveys of Adolescent Males (Ku, Sonenstein, and Pleck 1994; Sonenstein et al. 1998; Sonenstein, Pleck and Ku 1989), the 1990–91 British National Survey of Sexual Attitudes and Lifestyles (BNATSAL) (Johnson et al. 1994), the National AIDS Behavioral Survey (Catania et al. 1992), and others. The questions selected for the BSBS and NSBME research programs asked about a wide range of sensitive issues, with a majority of the questions pertaining to sexually transmitted diseases (STDs), sexual behaviors, and the use of licit and illicit drugs. We focus analyses in this article on the NSBME measurements related to same-gender sex.

Sexual attraction was assessed in the NSBME by asking a question adapted from the 1990–91 BNATSAL (Johnson et al. 1994) that asked respondents about their “level of sexual attraction to both males and females.” Response categories were provided for exclusively heterosexual or homosexual attraction, three levels of bisexual attraction, and no attraction to anyone. (See appendix A for the complete wording of all sexual behavior questions.) For our analyses, we combined response categories to obtain a binary indicator of whether the respondent reported any same-gender sexual attraction.

Three questions on same-gender sexual behaviors were also adapted from the BNATSAL; these questions asked about (1) same-gender “sexual contact or experience … includ[ing] just kissing and cuddling, not necessarily leading to genital contact or intercourse”; (2) same-gender genital contact; and (3) the recency of same-gender sexual contact, if any. Responses to these three questions are referred to as reports of same-gender sexual experience, genital contact, and recency of genital contact, respectively.

Respondents’ attitudes toward same-gender sexual contact were assessed using a question that has been asked in the General Social Survey (GSS) since 1973 (Davis and Smith 2002). This question asks whether “sexual relations between two adults of the same sex” are “always wrong, almost always wrong, wrong only sometimes, or not wrong at all.”

Data on attitudes toward same-gender sex from the GSS are used to aid in interpretation of the results of our experiment. These data were collected in 18 in-person surveys of probability samples of U.S. adults conducted by the National Opinion Research Center between 1973 and 2000 (Davis and Smith 2002).

STATISTICAL ANALYSIS

Our analyses of the NSBME are intended to answer five questions. First, are the social and demographic characteristics of the samples who completed the T-ACASI and CATI interviews equivalent (i.e., are the across-sample fluctuations no greater than would be expected to occur by chance in a random assignment)? Second, does the mode of interview affect the likelihood that a respondent will report same-gender sexual attraction or behavior? Third, could the different rates of interview break-off in the T-ACASI and CATI conditions have produced any mode effect that we detect? Fourth, is the impact, if any, of interview mode on reporting of same-gender sexual behaviors homogeneous across subpopulations defined by gender, ethnicity, place of residence, and so forth, or are some subpopulations particularly sensitive to interview mode? And finally, if T-ACASI provides more complete reporting of same-gender sexual behaviors, what is the estimated national prevalence of same-gender attraction and contact using this new survey methodology?

To address our research questions on the impact of interview mode on reporting of same-gender behaviors and attraction (tables 1–3), we combine the national and Baltimore sample strata. The combined sample strata are treated as a population that has been randomly allocated to one of two experimental conditions: T-ACASI or CATI interview mode. Data in these analyses are unweighted, and our statistical analyses assess the likelihood that observed fluctuations in survey responses across the two interview modes arose by chance from the random allocation of respondents to one of the two experimental groups. When this null hypothesis is rejected, we conclude that the mode of interview had an effect on survey response.

Table 3.

Percentage of Respondents Reporting Any Same-Gender Sexual Experience, by Demographic Characteristics and Interview Mode.

Demographic Characteristic	T-ACASI		CATI		Mode Effect (within strata OR)	Interaction Testsa
	%	n	%	n	OR	p
Gender
Male	10.6	386	8.1	480	1.3	0.276
Female	16.7	545	9.8	684	1.8
Age
18–25	14.9	215	10.7	289	1.5	0.565
26–35	13.4	350	9.4	424	1.5
36–45	14.6	364	7.8	447	2.0
Education
Less than high school	9.6	73	6.2	97	1.6	0.428
High school	14.3	217	8.2	280	1.9
Trade/some college	13.5	334	11.1	397	1.2
College graduate	16.0	306	8.2	388	2.1
Marital Status
Married	9.0	398	3.3	480	2.9	0.167c
Cohabiting	23.0	100	16.7	108	1.5	0.055d
Divorced, separated, widowed	17.2	128	9.2	163	2.0
Never married	16.8	304	13.4	411	1.3
Race/Ethnicity
Black (non-Hispanic)	11.4	219	8.5	284	1.4	0.862
White (non-Hispanic)	14.7	573	9.2	675	1.7
Hispanic/Latino origin	14.7	68	10.3	97	1.5
Other (non-Hispanic)	18.8	69	9.6	94	2.2
Children in Household
No	16.0	375	15.4	474	1.0	<.001
Yes	13.0	554	4.8	688	3.0
Demographic Characteristic	%	n	%	n	OR	p
Sample Strata
National	13.8	617	8.0	797	1.8	0.304
Baltimore	15.0	314	11.4	367	1.4
Level of Urbanizationb
21 largest MSAs	12.1	214	10.3	281	1.2	0.132e
Counties with 85,000+ households	17.7	186	7.1	268	2.8	0.047f
Counties with 20,000–84,999 households	13.2	106	8.1	136	1.7
Counties with < 20,000 households	11.4	88	3.2	95	3.9
Regionb
Northeast	14.6	96	11.3	159	1.3	0.118g
North Central	11.0	172	5.6	197	2.1	0.046h
South Atlantic	9.9	111	7.6	144	1.3
South Central	16.2	105	4.0	126	4.7
Mountain	28.3	46	8.8	57	4.1
Pacific	12.6	87	12.3	114	1.0

Note.—Unweighted data from 1999–2000 NSBME, combined National and Baltimore sample strata.

^aP-values derived from log-linear models testing for three-way interaction of response by interview mode by demographic characteristic.

^bTabulation includes only respondents in the national sample.

^cP-value for interaction using all four marital statuses.

^dP-value for interaction comparing currently married versus not currently married.

^eP-value for interaction using all four categories of urbanization variable (national sample).

^fP-value for interaction using two-category urbanization variable: 21 largest MSAs versus other (national sample).

^gP-value for interaction using all six categories of geographical regions (national sample).

^hP-value for interaction using collapsed geographical regions: Northeast and Pacific versus all others (national sample).

To assess the strength of the association between interview mode and reporting of same-gender attraction, behaviors, and attitudes, we compute the crude odds ratios for reporting such attraction, behavior, and attitudes in T-ACASI versus CATI interview modes. Since the availability of touchtone phones and differential sample loss in the two conditions introduced possible divergence in the sociodemographic composition of the respondents assigned to the two modes, we also report adjusted odds ratios, testing the impact of survey mode on response while controlling for the impact of deviations in sample composition. Adjusted ORs are calculated using multivariate logistic regression models that control for sample strata, gender, age, educational level, marital status, presence of children in the household, race/ethnicity, region of the country,¹⁰ and level of urbanization.¹¹ All of the foregoing analyses were carried out using STATA, version 6.0 (STATA Corporation 2000).

Log-linear modeling (Goodman 1968a, 1968b, 1978) was used to test whether the impact of T-ACASI on the reporting of same-gender sexual contact was equivalent across subpopulations defined by gender, race, region of residence, and other social and demographic factors (table 3). Our analyses fit log-linear models to three-way tables of same-gender sexual contact (S: yes or no) by mode of interview (M: CATI or T-ACASI) by sociodemographic factor (F: for example, male or female). Log-linear models were constrained to fit all two-way marginals [(SM) (SF) (MF)] in these three-way tables, and likelihood ratio chi-square statistics were calculated for the fit of these models to the observed data. Failure to obtain a statistically adequate fit indicated the presence of a statistically significant three-way interaction. This is to say that the impact of T-ACASI on the reporting of same-gender sexual behavior was not homogeneous across subpopula-tions defined by the sociodemographic factor (e.g., gender). Log-linear modeling was carried out using statistical software from SPSS, version 6.0.1 (SPSS, Inc. 1993).

In our final analysis (table 4), we generate national estimates of the prevalence of same-gender sexual attraction, experience, genital contact, and attitudes for U.S. males and females aged 18 to 45 who were interviewed using T-ACASI. For these analyses, sample weighting is applied to all observations to adjust for unequal probabilities of selection¹² and differential nonresponse by gender, race, and age. Calculation of standard errors and hypothesis testing in these analyses use procedures in STATA that take account of the impact of sample weighting.

Results

SAMPLE EQUIVALENCE

We began our analysis by testing the equivalence of the sociodemographic characteristics of respondents completing CATI versus T-ACASI interviews. These results indicate that respondents in the two conditions were statistically equivalent in age, gender, marital status, education, children living in the household, race/ethnicity, region of the country, and level of urbanization of the place of residence (see Villarroel et al. 2006).

MAIN EFFECTS OF INTERVIEW MODE

The first three panels of table 1 test the impact of interview mode on the reporting of same-gender sexual attraction, sexual experience, and genital contact during the respondent’s lifetime, the past five years, and the past year. In every instance, respondents who were interviewed using T-ACASI were 1.5 to 1.6 times more likely (crude ORs; ps < .001 to .05) to report same-gender sexual attraction and behaviors than respondents who were interviewed using traditional telephone survey procedures (CATI condition). Thus, 17.8 percent of T-ACASI respondents versus 12.8 percent of CATI respondents reported some same-gender sexual attraction; 14.2 percent versus 9.1 percent reported same-gender sexual experience; and 10.3 percent versus 7.0 percent reported same-gender genital contact ever. Odds ratios adjusted for sample composition factors vary little from the crude ORs, and all of the crude and adjusted ORs are statistically significant.

Interview mode also had a significant impact on the reporting of attitudes toward same-gender sexual behavior. The null hypothesis of no impact is rejected (p = .005) for the 2 × 4 table of interview mode by the four categories of response (always wrong, almost always wrong, wrong only sometimes, and not wrong at all). The largest impact of T-ACASI is seen for use of the response category indicating that same-gender sex is “not wrong at all.” Significantly more respondents gave this “tolerant” response to human interviewers (37.8 percent) than to T-ACASI (30.7 percent).

In considering this last result it should be borne in mind that this attitude question was not asked in the questionnaire section on same-gender sexual behaviors. Rather, it was the last substantive question on the survey. (Appendix B, online, provides information on the structure of the survey instrument and break-offs during the interview.) In the T-ACASI condition 8 respondents who completed the preceding question either refused to answer this question or discontinued the interview.¹³ In the CATI condition, however, 43 respondents were coded by the interviewers as “refusals.” (All 43 of these respondents had provided a substantive response to one or both of the two preceding questions.) This high level of refusal in the CATI condition was unexpected, and it may suggest that attitudes toward same-gender sexual behaviors are more sensitive than we anticipated.

SENSITIVITY OF ESTIMATES OF T-ACASI MODE EFFECT TO ASSUMPTIONS ABOUT INTERVIEW BREAK-OFFS AND NONINTERVIEWS

The T-ACASI condition suffered a larger number of interview break-offs than the CATI condition. More T-ACASI respondents who completed the first 23 questions (demographics, alcohol and drug use) discontinued the interview before reaching the questions on same-gender sexual attraction and contact (77 versus 42). Model 2 of table 2 presents estimates of the mode effect made under the assumption that nobody among this excess (n = 35) of interview break-offs in the T-ACASI condition would have reported same-gender sexual attraction or behaviors if they had completed these questions.¹⁴ The estimated mode effect (crude OR, T-ACASI:CATI prevalence) under model 1 is only slightly attenuated, e.g., from 1.6 to 1.5 for reporting of both lifetime same-gender sexual experience and genital contact in the past year.

The T-ACASI condition also had a larger number of noninterviews due to respondent refusals, failures to establish contact with the sampled household member, and interview discontinuation before the 23rd question. Model 3 assumes that nobody in this excess of interview break-offs and noninterviews in the T-ACASI condition (603 versus 484) would have reported same-gender sexual attraction or behavior if they had been interviewed. Even under the extreme assumption of model 3, the estimated mode effect would be 1.4 for both the reporting of lifetime same-gender sexual experience and genital contact in the past year.

INTERACTION EFFECTS

Table 3 presents the results of analyses that test whether the interview mode effects are homogeneous across subpopulations. Our analysis focuses on the reporting of lifetime same-gender sexual experience, and it is intended to answer questions such as, “Is the impact of survey mode on reporting of same-gender sexual experience equivalent for men and women or for persons with differing levels of formal education?” Table 3 indicates that a statistically adequate fit ( p > .15) to the observed data can be obtained by a model that assumes that the effect of interview mode on the reporting of same-gender sexual experience is equivalent for males and females and persons at each of four educational levels (0–11 years, 12 years, trade school or some college, and college graduate), as well as across four categories of race/ethnicity (black, white, Hispanic/Latino origin, other), three age ranges (18–25, 26–35, 36–45), and the two sample strata (national, Baltimore).

There are, however, four instances in which we reject the hypothesis that survey mode has an equivalent impact on reporting of same-gender sexual experience for different sociodemographic groups. Respondents who had children present in their household were three times more likely to report same-gender sexual contact in a T-ACASI interview than when interviewed by a human telephone interviewer (13.0 percent versus 4.8 percent, crude OR = 3.0). Respondents without children, however, gave statistically equivalent responses in the two interview modes (16.0 percent versus 15.4 percent, crude OR = 1.0). Using a log-linear model to test for nonequivalence of these survey mode effects yields a highly significant test statistic for this interaction (p < .001). A parallel interaction test finds suggestive evidence ( p = .167) of variation in the impact of mode across the four marital statuses (married, cohabiting, divorced or separated, widowed). When currently married respondents are contrasted with respondents who are not currently married, we find that T-ACASI had a stronger effect among currently married respondents (OR = 2.9 versus 1.5, p = .055).

The impact of T-ACASI is attenuated in highly urbanized areas such as the 21 largest MSAs (crude OR = 1.2), compared with sparsely populated areas of the country (crude OR = 3.9 for residents of counties with fewer than 20,000 households). When respondents from the 21 largest MSAs are compared with respondents from all less populated areas, we reject the null hypothesis that survey mode has an equivalent effect on reporting of same-gender sexual experience for these two groups of respondents (p < .05).

Log-linear modeling also revealed an interesting pattern of variation in the impact of T-ACASI across the six geographic regions. The impact of T-ACASI was strongest in the South Central and Mountain states (crude ORs = 4.7 and 4.1) and weakest in the Pacific, Northeast, and South Atlantic States (ORs = 1.0, 1.3, and 1.3). An interaction test of the homogeneity of the impact of T-ACASI across the six regions is marginal ( p = .118). When the Northeast and Pacific states are contrasted to the rest of the country, however, there is evidence of significant heterogeneity in the impact of T-ACASI on the reporting of same-gender sexual experience ( p = .046).

Region of the country is, of course, confounded to some extent with urbanicity. For example, only one of the 21 largest MSAs is located in the Mountain region. To explore the joint effect of region and urbanicity, we calculated interaction effects for four combinations of region and urbanicity: (1) residents of the 21 largest MSAs in the United States located in the Northeast or Pacific states, (2) residents of other locales in Northeast and Pacific states, (3) residents of the 21 largest MSAs in the United States located in the North Central, South Atlantic, South Central, and Mountain states, and (4) residents of other locales in these same states. Tests of the homogeneity of the impact of T-ACASI across subjects from these four groupings weakly suggest an interesting interaction (χ² = 2.36, df = 1, p = .12). It arises because the estimated impact of T-ACASI is statistically nil (OR = 0.7) in those of the 21 largest MSAs located in Northeastern and Pacific states, while it is substantial and roughly equivalent in MSAs located in other regions (OR = 2.7) and in less urbanized areas in both the Northeast and Pacific regions (OR = 2.5) and other regions (OR = 2.5).

PREVALENCE ESTIMATES FOR THE UNITED STATES USING T-ACASI

Table 4 provides weighted estimates of the prevalence of same-gender sexual attraction, experience, and genital contact and same-gender attitude in the United States, derived using T-ACASI measurements made in the national sample strata. Since our previous analyses suggest that T-ACASI encourages more complete reporting of same-gender sexual behaviors, we report these results as new national estimates of the prevalence of same-gender sexual behaviors in the U.S. population. Given our sampling universe, these estimates apply to English-speaking adults ages 18 to 45 who reside in telephone-accessible households with at least one touchtone telephone. These estimates indicate that 16.1 percent (SE = 1.6 percent) of the population reports some same-gender sexual attraction; 12.6 percent (SE = 1.4 percent) of the population reports some same-gender sexual experience; 8.9 percent (SE = 1.0 percent) of the population reports some same-gender genital contact in their life; and 3.7 percent (SE = 0.7 percent) of the population report some same-gender genital contact in the past year.

National T-ACASI estimates also indicate that U.S. females ages 18 to 45 report a markedly higher prevalence of same-gender sexual attraction (20.3 percent versus 11.9 percent, OR = 1.9, p < .05) and experience (16.2 percent versus 9.1 percent, OR = 1.9, p = .05) than U.S. males. National T-ACASI estimates of attitudes toward same-gender sex show a parallel and statistically reliable pattern. U.S. females are estimated to be 1.6 times more likely than U.S. males to say that same-gender sex is “not wrong at all” (31.1 percent versus 21.8 percent, OR = 1.6, p = < .05), and 0.6 times less likely to say that same-gender sex is “always wrong” (60.5 percent versus 47.7 percent, OR = 0.6, p < .01).

Table 4 also reveals that the relationship between reporting of nongenital versus genital same-gender contact is different for men and women. Most U.S. males who report same-gender sexual experience (defined to include “kissing and cuddling”) also report genital contact. In contrast, the estimated number of U.S. women who report only nongenital sexual contact (not shown) is substantial and significantly higher for women than men (6.5 percent versus 0.9 percent, OR = 7.8, p < .05).

Conclusion

If our experience in the United States is generalizable to other developed countries, prevalence estimates derived from traditional telephone surveys may understate the prevalence of same-gender sexual behaviors by a factor of 1.5 or more. This understatement appears to be particularly severe in locales that have traditionally been less tolerant of same-gender sexual behavior.

Throughout the last three decades of the twentieth century, residents of rural counties and small towns in the United States expressed attitudes that were considerably less tolerant of same-gender sex than the nation as a whole. Residents of major metropolitan areas, in contrast, were considerably more tolerant.¹⁵ The impact of T-ACASI on estimated prevalence appears to track this observed variation in the “tolerance” of the population. T-ACASI has its strongest impact in “less tolerant” rural counties (OR = 3.9) and its weakest impact in the “more tolerant” major metropolitan areas (OR = 1.2).

The impact of T-ACASI across geographic regions also appears to track the levels of tolerance expressed by the population. Thus, T-ACASI has its strongest effect in the less tolerant South Central states and its weakest effect in the more tolerant Northeast and Pacific states.¹⁶ When the joint impact of region and metropolitan residence is considered, our results indicate that the impact of T-ACASI is particularly attenuated in those of the 21 largest metropolitan areas (MSAs) located in the Northeast and Pacific states, like New York, Los Angeles, and so forth.

Overall, the variation of results from the NSBME by region and urban-rural residence could be interpreted as reflecting the interdependence of community attitudes toward same-gender sex and the relative impact of privacy in encouraging reporting of such experience in a survey interview. From the observed results, one might conclude that the greater the proportion of a population who disapproves of a behavior, the greater the impact of privacy on the likelihood the behavior will be reported.

NATIONAL PREVALENCE ESTIMATES

In addition to the foregoing methodological results, national prevalence estimates derived using T-ACASI data contain some important substantive results. First, in contrast to past studies in the United States and Britain,¹⁷ our new prevalence estimates indicate a higher prevalence of same-gender sexual attraction and experience among women and a statistically equivalent prevalence of genital contact with a weak suggestion of higher prevalence of genital contact in the past year among females (2.6 percent, SE = 0.9 percent for males versus 4.7 percent, SE = 1.2 percent for females; p = .176).

The estimated prevalence of female-female genital contact in the past year is considerably higher than estimates published by Laumann and colleagues (1994) for the percentage of U.S. women (ages 18 to 59) who reported having one or more female sexual partners in the past year (1.3 percent). There are many potential sources of variation—in addition to survey mode—that might account for these differences in prevalence estimates between the 1992 NHSLS and the 1999–2000 NSBME. Turner et al. (2005a, 2005b) suggest that changes over time in the reporting of same-gender contact during the 1990s—particularly by females—are a major contributor to this divergence in estimates.

Comparison of the U.S. T-ACASI prevalence estimates for male-male versus female-female sexual contact also uncovers an interesting pattern of sexual behaviors. Almost all of the men who reported male-male sexual experience in the T-ACASI condition (9.1 percent) also reported male-male genital contact(s) (8.2 percent). In contrast, we estimate that, in addition to the 9.6 percent (SE = 1.8) of U.S. females who report female-female genital contact, 6.6 percent (SE =1.4) of U.S. females report female-female sexual experience(s) without genital contact. While the sample sizes for males and females in the T-ACASI condition are relatively small (N = 269 and 348) and the standard errors are correspondingly large, this difference is statistically reliable (p < .05), and it suggests a possible qualitative difference in the patterns of same-gender sexual behaviors among males and females.

ADVANTAGES AND DISADVANTAGES OF T-ACASI

The foregoing evidence strongly suggests that T-ACASI encouraged more complete reporting of same-gender sexual attraction, experience, and genital contact. This assumes, of course, that increased reporting is more accurate reporting. Given the history of repression of same-gender sexuality, we believe this is a reasonable assumption. The patterns of variation in our results also encourage this interpretation. The privacy proffered by T-ACASI appears to have had its most pronounced effects in locales that express the least tolerance of same-gender sexual behaviors.

We note, however, that the apparent decrease in reporting bias produced by T-ACASI does not come without costs. First, T-ACASI interviews cannot be conducted in approximately 3 percent of U.S. households because there are no touchtone telephones in the household. Second, interview response rates¹⁸ achieved in the NSBME T-ACASI condition were 10 percentage points lower than those achieved by human interviewers in the national sample stratum (71.5 percent versus 61.3 percent) and 4.4 percentage points lower in the Baltimore sample stratum (67.7 percent versus 63.3 percent). This difference in interview response rates arose, in part, from higher rates of interview break-off in the T-ACASI condition. Approximately one-half of T-ACASI break-offs in the NSBME occurred from perceived computer problems or call-waiting interruptions. The other half of break-offs were initiated by the respondent because, for example, respondents thought the “interview was too long,” “didn’t like the questions,” or had to tend to an emergency (see appendix B).

Human interviewers can often forestall or recover from break-offs in such situations (Nyman, Roman and Turner 2001). They can, for example, request that call-waiting be disabled or schedule an appointment to complete an interview if the incoming call is urgent or if an emergency in the household requires the respondent to interrupt the interview. Human interviewers can also remind reluctant respondents of the public health and scientific aims of the study and explain why we need data on sexual behavior from a representative sample of the population. The mere social presence of a human interviewer (rather than a computer) may also make it more difficult for respondents to hang up the telephone.

Comparison of sociodemographic characteristics of the unweighted T-ACASI and CATI samples in the NSBME indicate that sample loss in the T-ACASI condition did not induce differences in sample composition along the dimensions we measured (Villarroel et al. 2006). That does not, however, rule out differences on other, unmeasured dimensions. The sensitivity analysis presented in table 2 reassures us, however, that the impact of any such differences would not be large enough to erase the observed T-ACASI mode effects.

We believe, nonetheless, that if T-ACASI is to be widely adopted to collect data on sexual and other sensitive behaviors, T-ACASI interview response rates will need to be improved. Progress in that direction has been reported (Link, Johns, and Cooley 2000),¹⁹ but a sustained program of applied research toward this end would seem warranted, given the large reductions in reporting bias that we have observed using T-ACASI technology.

Appendix A

NSBME SURVEY MEASUREMENTS RELATED TO SAME-GENDER SEX

Same-gender sexual attraction (adapted from the 1990–91 BNATSAL [Johnson et al. 1994]):

The next question asks about your level of sexual attraction to BOTH males and females. Please consider the response choices carefully, as it is important that you understand them and are as honest as you can be in your answer. To whom have you felt sexually attracted, even if you did not take any action based on feeling attracted?

1. Only to females, never to males

2. Mostly to females, and at least once to a male

3. About equally often to females and to males

4. Mostly to males, and at least once to a female

5. Only to males, never to females

6. I have never felt sexually attracted to anyone at all

Note.—For female respondents, answer categories one to five were presented in reverse order.

Same-gender sexual experience (adapted from Johnson et al. 1994):

The next question asks about any sexual contact or experience. This is a wide term and can include just kissing and cuddling, not necessarily leading to genital contact or intercourse. Have you ever had any kind of sexual experience or sexual contact with a female [male]? [Yes/No]

If yes:

Same-gender genital contact:

(For females) Have you ever had sex with a woman involving genital area/vaginal contact? [Yes/No]

(For males) Have you ever had sex with a man involving genital area/penis contact? [Yes/No]

If yes:

When was the last occasion?

1. In the last 7 days

2. Between 7 days and 4 weeks ago

3. Between 4 weeks and 6 months ago

4. Between 6 months and 1 year ago

5. Between 1 year and 5 years ago

6. Longer than 5 years ago

Attitudes toward same-gender sex (adapted from the General Social Survey [Davis and Smith 2002]):

We would like to know your opinion about sexual relations between two adults of the same sex—do you think it is always wrong, almost always wrong, wrong only sometimes, or not wrong at all?

Appendix B

Some sections of this appendix are adapted or excerpted from the survey field report prepared by Roman (2000) and conference presentation by Nyman, Roman, and Turner (2001).

INTRODUCING SURVEY TO RESPONDENTS

After household screening and within-household random selection of the target respondent, the interviewer explained the study to the target respondent as follows (in both the T-IAQ and T-ACASI conditions):

• Your household is one of only 2,500 nationwide that have been randomly selected for this research study to help us gather better data on the factors that put people at risk of AIDS and other important health problems such as drug use and violence.

• The survey will last only about 15 to 20 minutes. We are conducting this survey with support from the National Institutes of Health in collaboration with the Research Triangle Institute. Your participation is voluntary.

• Because of the health topics we are studying, some questions may be sensitive. You can choose to skip any question in the interview, although we encourage you to answer as many as you can.

• Of course, all your answers are strictly confidential. Portions of this interview may be monitored by my supervisor for quality control purposes only.

• Do you have any questions before we begin?

• Later, if you have any questions about the survey, you may call [Name of Survey Director] 1-800-492-NNNN. If you have any questions about your rights as a research participant, you may call [Name of Institutional Review Board Chair] 1-800-334-NNNN, ext. NNNN (he works for RTI in North Carolina).

• Should you want to call us back, tell the supervisor you are calling about study 648 and refer to case number [Case ID].

• In the CATI condition, interviewers then began the survey. In the T-ACASI condition, respondents received the following additional introduction:

  Let me explain how this interview works. I will be asking some of the questions, while others will be prerecorded and asked by a computer. You can answer the questions by pushing the number buttons on your telephone. In order to do this, you will need a touchtone phone.

  Do you have a touch-tone phone?

The 3.4 percent of T-ACASI interview respondents who did not have a touchtone telephone were switched to the T-IAQ condition, and their data are excluded from the analysis reported in this article. If the respondent was unsure whether they had a touch-tone telephone, interviewers asked them to press the number three on the telephone keypad to allow the interviewer to determine whether the phone was a touchtone telephone that was correctly set to generate tones rather than pulses. If the phone did not generate tones and could not be reset, respondents were switched to the T-IAQ condition, and their data are excluded from our analysis.

All T-ACASI respondents were then instructed:

• We recommend using a phone that has the number pad separate from the part of the phone that you hold in your hand. Also, we want to be sure you feel like you are in a place where you have enough privacy.

• Do you need a minute to go to a different phone? [wait if respondent needs time]

• I’m going to put you on hold for just a minute while I connect us to the computer.

The interviewer established a three-way connection between the T-ACASI computer, the respondent, and the interviewer. The T-ACASI computer then played the following script:

• The computer interview is ready to begin. The Respondent ID is NNNN.

• We are going to begin by having you answer several introductory questions to help you get used to working with the computer. The interviewer will stay on the line as you answer these questions. You will both hear these questions, and you can ask the interviewer for help if you do not understand how to use the system. Please give accurate answers to these questions; the information is important.

The computer then asked nine questions on the respondent’s social and demographic characteristics (gender, age, marital status, education, race, ethnicity, and number of children in the household) and their use of antibiotics. During this time, the interviewer was available to answer respondents’ questions and ensure that the respondent could interact successfully with the computer. At the conclusion of these questions, the interviewer announced:

I am going to disconnect from the call now so that you may answer in complete privacy. If at any time you need help, press the STAR key for instructions on backing up, listening to the question again, answering “don’t know,” refusing to answer a question, or accessing a help line.

The interviewer then disconnected from the call, and the T-ACASI respondent continued the interview in private.

INTERVIEW BREAK-OFFS

The T-ACASI experimental condition had a larger number of interview break-offs than the interviewer-administered T-IAQ condition. Table B1 details sample loss in each experimental condition from the point at which an eligible household member had been selected for interviewing through the final substantive survey question. The break-off point in an interview was defined as the last question for which a respondent gave a substantive answer, that is, excluding “refuse,” no answer, and so forth. Although this is not necessarily the point at which the respondent (or computer) terminated the telephone connection, it is effectively the point at which the respondent (or computer) discontinued the substance of the interview.

RECONTACT OF T-ACASI BREAK-OFFS

If a T-ACASI interview was broken off, an attempt was made to recontact the respondent. Each night, the T-ACASI computer at RTI in North Carolina would e-mail the UMass survey unit a list of cases that were contacted that day, identifying those that were completed and those that broke off prior to completion. For break-offs, this report also indicated how much of the interview had been completed. Break-offs were scheduled to be recontacted the next day to determine why the respondent did not complete the interview and to try to get a complete the interview from the point where the respondent had left off.

After the interviewer recontacted the respondent and identified him/herself and the study, the interviewer stated:

• Recently, we attempted to interview you about AIDS and other important health issues. The computer portion of the interview was not completed entirely.

• Was this because of problems with the computer or because you decided to end the interview yourself by hanging up?

• Were the computer problems caused by: call-waiting disrupting your interview; the computer hanging up on you; or something else? [If something else, specify.]

• About how many minutes went by between the computer asking you a question and you noticing the computer had hung up? [Probe: As best you can recall]

• Could you briefly describe what was happening when the computer hung up?

• Did you decide to stop because the interview was too long, you didn’t like the questions, there was an emergency, you did’t like being interviewed by a computer, or something else [specify]?

• We would appreciate it if you could complete the remaining interview questions.

• I’m going to put you on hold for just a minute while I connect us to the computer.

The interviewer then connected the respondent to the T-ACASI computer and hung up from the call.

REPORTED REASONS FOR T-ACASI BREAK-OFFS

A total of 278 respondents answered the T-ACASI break-off questions. Of the 278, there was nearly an exact split between those who claimed technical problems caused the break-off (50.4 percent) and those who claimed they ended the interview themselves (49.6 percent).

For the 140 respondents who claimed a computer problem, we attempted to get them to describe what happened. This proved quite difficult for respondents to do. Of these respondents, 51.4 percent could not add any more details other than the computer seemed to hang up on them. The next largest category (20 percent) consisted of people who took another call through call-waiting and had the computer disconnect while they were on the other call. (T-ACASI interviews could not be paused. If the respondent did not respond to a question within approximately 30 seconds, the system repeated the question. This was done a maximum of three times. If no valid response was entered after three repetitions of the question, the computer terminated the call.) The next two largest categories were respondents who believed the interview was done because no more questions were asked (10 percent) and problems in which the computer would no longer accept their keypad entries (7.1 percent).

It was much easier to get the 137 respondents who hung up to report why they did so. The two largest complaints were that the interview was too long (36.3 percent of those from whom we obtained a reason) and that they didn’t like the questions (31.5 percent). The next largest category was people who said they hung up to take care of an emergency (13.7 percent). Very few people (3.2 percent) claimed that they didn’t like answering questions asked by a computer.