Reports of Alcohol-Related Problems and Alcohol Dependence for demographic subgroups using Interactive Voice Response (IVR) versus Telephone (CATI) Surveys: The 2005 U.S. National Alcohol Survey

Abstract

Introduction and Aims

Interactive voice response (IVR), a computer-based interviewing technique, can increase a sense of privacy and a willingness to report personally sensitive attitudes and behaviors accurately. Previous research using the 2005 National Alcohol Survey indicates no significant differences between IVR- and personally administered responses to alcohol-related problems and indicators of alcohol dependence. It is not clear if this result holds for specific gender, ethnic, age and income subgroups that may respond differently by mode of data collection. The purpose of this study is to compare the prevalence rates of lifetime and current (last 12 months) alcohol-related problems by gender, ethnicity, age and income subgroups obtained by IVR as compared with traditional telephone interviews (CATI).

Design and Methods

As part of the data collection effort for the 2005 National Alcohol Survey, subsamples of English speaking respondents were randomly assigned to an IVR group that received an IVR module on alcohol-related problems (n=450 lifetime drinkers) and a control group that were asked the alcohol-related problem items using CATI (n=432 lifetime drinkers).

Results

Overall, there were few significant associations. Among lifetime drinkers, higher rates of legal problems were found for white and higher income respondents in the IVR group. For current drinkers, a higher percentage of indicators of alcohol dependence was found for Hispanic and women respondents in the CATI group.

Discussion and Conclusion

Data collected by CATI interviews provide largely comparable results to IVR. Thus, the expense of incorporating IVR in alcohol surveys for alcohol-related problems is not indicated by these results.

Introduction

Misreporting of alcohol problems in sample surveys is problematic and has plagued researchers and challenged them to derive new and better ways to obtain information from respondents. The manner in which data are collected is critical to assuring that self-reports of sensitive behaviors are both valid and reliable. Previous face-to-face U.S. National Alcohol Surveys until 1995 had asked alcohol intake questions using booklets handed to the interviewee, seen as increasing privacy.

The touchtone data entry and voice recognition system was developed as a way of collecting data over the telephone ^1,2 that was termed “Touch-Tone Data Entry” by the U.S. Bureau of Labor Statistics.³ This system allowed for interviewing on a “call-in” (respondent calls into the system to begin the interview) or a “call-out” (respondent is called by an interviewer who transfers the call at some point during the interview to the computer system or the call is completely computerized) basis. Eventually, known as Interactive Voice Response (IVR), this method of data collection has many advantages such as a potential increased of privacy thereby allowing greater accuracy of sensitive items, the ability to branch through a complex questionnaire, consistency checks for appropriate ranges, the ability to analyze the data as they are being collected,⁴ economy, autonomy, confidentiality, the ability to access specific groups of people, multilingual interfaces, and long-term assessments of behavior.⁵

Beginning in the mid 1990s, IVR studies in the alcohol field began to appear in the literature that used daily call-in methods to monitor alcohol use.^6–9 In general, findings indicate that IVR was successful in tracking drinking behavior over a longer time period, and it was well accepted by the respondents. It should be noted that these earlier studies used small samples of 30 and 55 male social drinkers. This 30-day call-in method was also used as an intervention to lower alcohol consumption in a sample of 31 HIV positive patients from a large, hospital-based outpatient HIV primary care clinic¹⁰ and was extended to 12-weeks in a pharmacotherapy study of alcohol use with 9 patients.¹¹ Simpson et al.¹² have also used this monitoring procedure to measure reactivity of daily reports on reports of alcohol use and PTSD symptoms in a sample of 98 respondents with a current alcohol use disorder.

It is important to recognize that more current research using IVR has the advantage of learning from earlier studies on script development, programming, and managing databases with IVR data.¹³ Thus, for example, the length of follow-up has been extended to 2 years for a sample of 80 nonclinical alcohol users.^14,15 One interesting aspect of this study covered in these articles was the ability to monitor the effect of September 11^th on the drinking levels of individuals who lived approximately 300 miles from New York City. This application of relating daily use of alcohol to contextual events is an important and useful aspect to daily monitoring.

There have been several methodological studies of IVR (call-in) with other forms of data collection in the alcohol field that provide evidence of IVR’s usefulness in collecting alcohol data. In comparisons with the TLFB, IVR seems to provide similar or even higher prevalence rates of alcohol use;^16–20 this was also noted by Schroder et al.²¹ for sexual behavior and substance use. Other studies comparing IVR with different data collection methods have demonstrated that IVR is a reliable and valid way to administer alcohol screening instruments such as the AUDIT²² when compared to interviewer administration; the Teen-Addiction Severity Index and the Addiction Severity Index comparing IVR with an Internet application;^23–24 and, a wide range of alcohol scales comparing IVR with a self-administered questionnaire.²⁵ Overall, findings from these studies support the use of call-in IVR as a reliable and valid way to collect alcohol data from either social drinkers or clinical samples.

In the alcohol field, most IVR research has used call-in methods. One exception is a study that incorporated IVR into a national alcohol survey using a randomized comparative design and found no differences between the IVR and standard CATI administration-mode groups on prevalence rates of alcohol-related problems and alcohol dependence both for lifetime drinkers and current drinkers.²⁶

While researchers such as Corkrey and Parkinson⁵ have suggested that more methodological studies should be conducted on the application of IVR for specific subgroups, such as the elderly and minority language groups, few studies have examined whether using IVR is appropriate for these groups and/or can be revised to be more sensitive to the needs of specific demographic subgroups.

The purpose of this study is to assess the relationship and examine potential modality differences by ethnicity, gender, age and income on reports of alcohol-related problems and indicators of alcohol dependence obtained either by IVR or a standard CATI approach.

Methods

2005 National Alcohol Survey

The data used for this study are from the 2005 National Alcohol Survey (NAS), funded by the National Institute on Alcohol Abuse and Alcoholism as part of the National Alcohol Research Center (P30 AA05595). Since the 1960s, the NAS has been utilized to conduct trend analyses in several areas related to alcohol use including drinking patterns, alcohol related problems, use of health services, and potential risk and protective factors among current drinkers.^27–32

The 2005 NAS was conducted for ARG by DataStat, Inc., of Ann Arbor, MI, and involved a nationally representative probability sample of English and Spanish speaking U.S. adults aged 18 years and older residing in households with land-line telephones within all 50 states and Washington, D.C. Using list-assisted Random Digit Dial sampling, telephone interviews were conducted from November 2004 to June 2006 by trained survey interviewers using Computer Assisted Telephone Interview (CATI) methods. Interviews were conducted seven days a week, throughout the day and evening hours, and took approximately 1 hour to complete. Numerous callbacks and refusal conversion attempts were conducted according to a scheme that alternated days and times of calls. Prior to each interview, all respondents were advised of the purpose of the interview, the confidentiality of the results, the voluntary nature of their participation, as well as their right to refuse to answer any question or to end the interview at any time. While no written consent forms were obtained, all participants provided verbal informed consent by an Institutional Review Board approved protocol. Interviews with Hispanics were conducted in either Spanish or English based on the preference of the respondent.

IVR Sample and Procedures

The embedded Interactive Voice Response (IVR) study of sensitive behaviors is based on three separate but comparable randomly selected samples from the 2005 National Alcohol Survey. Originally, respondents were randomly assigned to three groups using CATI system programming; two of these samples received a limited series of questions through IVR during the CATI interview. A female voice in English only (for cost and practical reasons) was programmed for the IVR questions in both modules. Thus, Spanish speaking respondents were not included in this study. Eligible respondents were informed by the interviewer that a brief portion of their interview would be conducted by IVR. Respondents were instructed on how to answer questions using their touch-tone phones and how to return to an interviewer if they needed assistance or to discontinue the interview. For the purposes of this study, the group that was asked the alcohol-problems/alcohol dependence items using IVR (called the IVR group) will be compared with the group that were asked the same items in the regular telephone survey (called the CATI group).

Initially, 562 respondents were randomly assigned to the IVR Group. In this group, 101 respondents were lifetime abstainers and 11 respondents did not have touchtone phones; thus the IVR sample size was 450 lifetime drinkers (309 current drinkers). Approximately, 85 percent of these respondents (384/450) completed the IVR portion of the interview without assistance. Initially 563 respondents were randomly assigned to the CATI group. In this group, 88 were lifetime abstainers and 43 did not have touchtone phones; thus the CATI sample size was 432 lifetime drinkers (297 current drinkers).

Table 1 presents comparisons of key demographic variables between the 2 randomly selected groups. None of the demographics were significantly different between the groups.

Table 1.

Demographics of the IVR and CATI Groups (%)

	IVR (N=562)	CATI (N=563)	X2
Gender - Male	41.5	40.8	P=0.836
Ethnicity			P=0.277
White	27.2	27.2
Hispanic	22.1	25.9
Black	48.2	43.5
Other	2.5	3.4
Age			P=0.661
18–29	18.3	17.6
30–49	33.6	36.3
50+	48.1	46.1
Income			P=0.984
≤ 10k	14.2	13.7
10–30k	25.4	24.9
> 30k	46.6	47.4
Missing	13.7	14.0

Measures

Alcohol-Related Problems

Alcohol-related problems were divided into two scales: Social Consequences and Alcohol Dependence. Social Consequences is derived from 15 incidents in 5 problem areas surrounding alcohol use that respondents may have experienced during their lifetime and during the 12 months prior to the interview. Problem areas include: fights (2 items); legal issues (5 items); health (3 items); work life (3 items); and, negative reactions from spouse or partner (2 items) (for further details see Midanik & Greenfield²⁶).

Alcohol Dependence

Alcohol dependence was derived from diagnostic criteria reported in the Fourth Edition of the American Psychiatric Assocation.³³ Based on a 17-item scale, respondents who had positive symptom reports on at least three of the seven DSM-IV domains during their lifetime and in last 12 months were classified as having alcohol dependence.^34,35 For both Alcohol Problems and Dependence, only those respondents who reported alcohol use in their lifetime were asked questions about lifetime alcohol-related problems; similarly, only respondents who had consumed alcohol in the last 12 months and affirmed a given lifetime problem were asked whether it had occurred during the 12-months (used for defining the 12-month variables.

Demographic Variables

Specific demographic subgroups that were used in this analysis based on self-reported items, in some cases recoded, were: Gender, Ethnicity (White, Hispanic, Black Other), Age (<=45 years, >45 years) and Income (<=$30,000/year, >$30,000/year).

Data Analysis

To compare IVR and CATI for alcohol-related problems and alcohol dependence, we assessed both lifetime and current (last 12 months) prevalence rates for each of the four demographic subgroups. Chi square and t-tests were used to compare outcome measures separately.

Results

Table 2 presents comparisons between the IVR and CATI groups for lifetime drinkers within demographic subgroups for lifetime alcohol-related problems and indicators of DSM-IV alcohol dependence. There were no significant differences between the modality groups within demographic subgroups for 2 or more alcohol-related consequences or the 3-symptomatic-domain indicator of alcohol dependence. However, within the consequences items, reports of alcohol-related legal problems were significantly higher among the IVR group for whites (20.9 versus 10.5, p<0.05) and for higher income respondents (15.1 versus 8.1, p<0.05).

Table 2.

Comparison of lifetime drinking problem measures among lifetime drinkers (%)

			Ns	2+ Consequences	Fights	Legal	Health	Work	Reaction	DSM-IV Dependence
Ethnicity	White	IVR	134	22.4	29.1	20.9	11.9	5.2	12.7	22.4
		CATI	124	25.8	30.6	10.5*	12.9	3.2	16.1	15.3
	Hispanic	IVR	105	19.0	26.7	8.6	6.7	4.8	9.5	13.3
		CATI	118	22.0	28.0	14.4	9.3	6.8	16.1	21.2
	Black	IVR	200	12.0	20.0	8.0	10.5	3.5	6.0	10.5
		CATI	176	9.1	14.2	6.3	6.8	2.8	5.7	8.5
Gender	Women	IVR	250	12.0	20.8	6.0	8.0	3.2	4.8	8.4
		CATI	242	10.7	15.7	5.4	5.8	1.7	7.4	11.2
	Men	IVR	200	23.0	28.0	20.6	12.0	6.0	14.0	22.5
		CATI	190	26.8	32.1	15.3	13.7	7.4	17.9	17.4
Age	≤ 45	IVR	198	18.7	30.3	13.1	7.6	3.0	8.6	16.7
		CATI	205	21.5	27.3	10.7	10.7	3.9	12.7	18.5
	> 45	IVR	240	15.8	18.8	12.6	12.1	5.8	9.6	13.3
		CATI	213	15.5	20.2	9.4	8.5	4.7	11.7	10.3
Income	≤ 30k	IVR	167	21.0	25.1	9.0	11.4	6.0	10.2	14.4
		CATI	154	19.5	22.7	13.0	14.3	5.8	12.3	16.9
	> 30k	IVR	232	15.1	22.8	15.1	8.6	3.9	8.2	14.7
		CATI	223	17.0	24.2	8.1*	6.7	3.6	11.7	12.6

^*p<0.05

Similarly, there were few differences for alcohol-related consequences and alcohol dependence within the four demographic subgroups between the IVR and CATI groups for current drinkers (Table 3). However, in a direction contrasting with the last results, a higher prevalence rate for alcohol dependence was found using CATI than IVR for Hispanic (7.6 versus 1.2, p<0.05) and women (5.8 versus 0.6, p<0.05) respondents.

Table 3.

Comparison of last-12-month drinking problem measures among current drinkers (%)

			Ns	2+ Consequences	Fights	Legal	Health	Work	Reaction	DSM-IV Dependence
Ethnicity	White	IVR	96	2.1	2.1	1.0	3.1	1.0	3.1	3.1
		CATI	97	5.2	4.1	0	4.1	1.0	6.2	5.2
	Hispanic	IVR	82	1.2	9.9	2.4	1.2	1.2	4.9	1.2
		CATI	92	1.1	3.3	0	2.2	0	3.3	7.6*
	Black	IVR	124	4.8	9.7	2.4	3.2	0	4.0	4.0
		CATI	97	5.2	5.2	3.1	3.1	0	1.0	4.1
Gender	Women	IVR	158	1.3	5.7	1.9	1.3	0.6	0.6	0.6
		CATI	156	3.8	3.8	0.6	3.8	0	3.8	5.8*
	Men	IVR	151	5.3	9.3	2.6	4.0	1.3	7.9	5.3
		CATI	141	3.5	4.3	1.4	2.1	0.7	3.5	5.0
Age	≤ 45	IVR	152	5.3	10.6	3.9	2.6	1.3	6.6	4.6
		CATI	164	6.1	6.1	1.8	3.7	0.6	5.5	8.5
	> 45	IVR	149	1.3	4.7	0.7	2.7	0.7	2.0	1.3
		CATI	127	0.8	1.6	0	2.4	0	1.6	1.6
Income	≤ 30k	IVR	100	4.0	9.0	1.0	4.0	2.0	7.0	3.0
		CATI	92	5.4	5.4	1.1	4.3	1.1	4.3	5.4
	>30k	IVR	175	2.9	7.5	3.4	1.1	0.6	2.9	2.9
		CATI	174	2.3	4.0	0.6	2.3	0	2.9	5.7

^*p<0.05

Discussion

While our previous research indicates that there were no significant overall differences between the IVR and CATI groups on alcohol-related consequences and alcohol dependence,²⁶ only a few significant subgroup differences emerged here. For lifetime drinking, significantly higher reports of legal problems were found in the IVR group, compared to the standard CATI administration, as also true in this analysis for White respondents and for respondents with incomes above the median. For current drinkers, significantly higher reports of indicators of alcohol dependence were found in the CATI group for Hispanic respondents and for women. These differences suggest that within the general domain of social and health consequences of alcohol use, specifically alcohol-related legal issues may be more sensitive and thus were better assessed, with less downward bias, using IVR, which may be considered to provide more privacy. This seems plausible since such reports among lifetime drinkers might be considered stigmatizing or socially inappropriate. However, it appears that the opposite occurs for current drinkers at least in relation to symptoms of alcohol dependence and for certain demographic groups—women and Hispanics. By reporting current alcohol consumption these individuals may already have indicated some openness to drinking issues, and so these women and Hispanic respondents may be more responsive to an actual interviewer. One advantage of an in-person interviewer is to develop rapport, and while the IVR module was administered within an overall CATI interview, it may nonetheless create a sense of “impersonality”, possibly in some instances offsetting some advantages of not requiring reporting to a person. While for reporting income the computer generated voice and impersonal response might allow more accurate reports, for some types of questions the lack of an “empathic listener” might actually reduce openness. More methodological work would be needed to verify such conjectures.

There are several limitations in this study that need to be addressed. First, the study relied on self-reported data. While the validity of self-reported alcohol use has been discussed in detail elsewhere,^36–38 it should be noted that both the CATI and the IVR methods of data collection offer a high level of autonomy and a certain sense of anonymity for respondents; thus, the context in which these items are being answered are less likely to be conducive to errors or bias in reporting. Second, respondents who did not speak English or were not comfortable with having their interview in English were omitted from this study in both IVR and regular CATI control. Individuals who spoke Spanish, however, were included in the CATI interview for the larger 2005 National Alcohol Survey. A third limitation is that the sample sizes within subgroups analyzed here is small and thus more subject to random errors.

Implications

Overall, the modest degree of differences seen by subgroup between IVR and CATI for alcohol-related problems and alcohol dependence among lifetime and current drinkers supports earlier general population findings.²⁶ One might reasonably conclude that in general, use of IVR within the context of a CATI-based epidemiological survey of alcohol use patterns and problems is not critically needed to assure reasonably accurate reports from most participant groups.