Agreement between Prospective Interactive Voice Response Telephone Reporting and Structured Recall Reports of Risk Behaviors in Rural Substance Users Living with HIV/AIDS

Abstract

Sound measurement of risk behaviors is essential to guide tailored risk reduction strategies as HIV infection patterns shift toward rural minorities, particularly in the South where HIV disease remains highly stigmatized. Interactive Voice Response (IVR) systems appear to enhance reports of sensitive behaviors and can support telehealth applications to extend the reach of care in rural, underserved areas. This study evaluated the feasibility and data quality of an IVR telephone reporting system with rural substance users living with HIV/AIDS. Community-dwelling patients were recruited from a non-profit HIV medical clinic in rural Alabama (N = 35 men, 19 women). Participants engaged in daily IVR reporting of substance use and sexual practices for up to 10 weeks. IVR reports were compared with retrospective Timeline Followback (TLFB) interview reports for the same period. IVR and TLFB reports showed good to excellent agreement for summary measures of alcohol consumption and sexual activity. Agreements for illicit drug use reports were less satisfactory. Reports of monetary spending on alcohol and drugs were significantly higher on the IVR. Most individuals showed good agreements for reports of day-to-day alcohol and drug use and sexual practices. The study established the utility of IVR assessment with rural, disadvantaged adults living with HIV/AIDS who are priority targets for risk reduction interventions.

HIV rates in the Southeastern United States have increased dramatically, particularly among persons of color and other disadvantaged groups (Southern AIDS Coalition, 2008). Seven states with the 10 highest AIDS rates in the U.S. are located in the South, and proportionally greater increases in Southeastern HIV prevalence have been found for persons living in rural, versus suburban or urban, population centers (Centers for Disease Control & Prevention, 2008). Access to medical and behavioral health care is poor in the region due to geographic isolation and poverty, and HIV disease remains highly stigmatized. Relatively little is known about risk and protective behavior patterns among rural Southern HIV populations, which is a necessary first step to designing population sensitive and specific interventions.

Interactive Voice Response (IVR) telephone systems show promise as a way to reach and obtain sensitive information from rural, geographically isolated populations (Schroder & Johnson, 2009). Phone access is near universal, even in rural settings, and IVR systems support private, near real-time reports of HIV risk behaviors such as drinking, drug use, and unprotected sex (Schroder, Johnson, & Wiebe, 2007; Tucker, Foushee, Black, & Roth, 2007). Prospective IVR assessment yields reports of risk behaviors that show good correspondence with established structured retrospective assessment methods, such as the Timeline Followback (TLFB) interview (Sobell & Sobell, 1992) in which daily reports of drinking and other risk behaviors are recalled for up to a year (e.g., Carney, Tennen, Afflect, Del Boca, & Kranzler, 1998). When differences have been observed between methods, self-reports using IVR systems and similar electronic diaries yielded higher, presumably more complete reports of sensitive behaviors, such as risky sexual practices (e.g., unprotected sex, transaction sex), compared to the interview data collection (Blumberg, Cynamon, Osborn, & Olson, 2002). In some applications, IVR data collection may be preferable to face–to-face interviews, such as for collecting near real-time reports on daily sequences of different risk behaviors (Schroder & Johnson, 2009).

IVR systems thus may serve as a platform for delivering telehealth risk-reduction programs with assessment and therapeutic functions to hard-to-reach populations living with or at risk for HIV disease. However, past IVR studies concerned with HIV have focused almost exclusively on urban areas (e.g., Aharonovich et al., 2007), and research on IVR feasibility and data quality is needed using rural persons living with HIV. In the present study a community-dwelling sample of sexually active clinic patients living with HIV/AIDS in the rural South used a computerized IVR telephone system to report prospectively their daily drinking, drug use, and sexual activity for up to 70 days. Reliability interviews using an expanded TLFB were conducted at the end of the IVR assessment, and were compared with the IVR reports. The present study thus replicated with a disadvantaged, hard-to-reach population Tucker et al.’s (2007) prior work that established the utility and quality of IVR assessment of drinking behaviors using recently resolved problem drinkers who were generally educated, employed, middle-class adults.

Based on past studies (e.g., Tucker et al., 2007), good percent agreements (≥ 80%) and correlations (r ≥ .80) were predicted between measures of drinking practices (e.g., frequency of and quantity on drinking days). Higher IVR reporting was predicted for sensitive, complex information, such as sexual activity, compared to TLFB reports (Schroder et al., 2007). Agreements for reports of the temporal patterning of day-to-day occurrences of drinking, drug use, and sexual activity based on longitudinal data from each participant were expected to range from poor (≤ 50%) to excellent (100%), with most expected to be satisfactory (≥ 80%).

Method

Sample Characteristics

Patients living with HIV/AIDS were recruited from a non-profit community-based clinic that is the only provider of HIV medical care to 14 counties in rural northeastern Alabama. The service area is approximately 30% African-American, 16% living below poverty level, with a mean education level of 11^th grade. Patients recruited using fliers and word-of-mouth were told that the study was investigating risk and health behaviors using interview and phone assessments and were screened by an interviewer who was not on clinic staff. Eligibility criteria included: (1) age ≥ 19 years; (2) reported use of alcohol or illicit drugs and sex within the past 3 months; (3) no health problems precluding participation (e.g., active psychosis); and (4) daily phone access.

About half of the 109 screened respondents met eligibility criteria, and 91.5% of those (54/59) were enrolled either in an initial pilot study to finalize procedures that involved up to 28 days of IVR reporting (n = 8), or in the main study that involved up to 70 days of IVR reporting (n = 46). The research was approved by the University Institutional Review Board and covered by a federal Certificate of Confidentiality. All participants gave written informed consent. Table 1 shows demographic and substance use characteristics of the enrolled sample of 54.

Table 1.

Baseline Characteristics of Enrolled Sample (N = 54)

Variable	M	SD	%
Demographic characteristics
Male			65
Female			35
White			54
African American			43
Other race/ethnicity			3
Age (years)	38.39	9.49
Education (years)	10.87	2.36
Unemployed			21
Disabled			46
Annual income < $10,000			76
Annual income < $20,000			19
Annual income > $20,000			5
Live in own house or apartment			48
Partner status and sexual orientation
Married or in long-term partner relationship			32
Heterosexual			52
Homosexual			20
Bisexual, unsure, questioning			13
Undisclosed			14
HIV health markers a
Viral load unknown			61
Viral load > 50			17
Viral load undetectable			22
CD4 unknown			53
CD4 < 200			11
CD4 ≥ 200			35
Baseline risk behaviors (past 90 days at enrollment)
Any alcohol use b			80
Any marijuana use b			48
Any cocaine, methamphetamine, or other stimulant use b			26
ASSIST Global Continuum of Risk score b	61.0	43.5
% days involving alcohol consumption c	16.6	24.1
Quantity per drinking day c	16.6	23.9
% days involving other drug use c	19.0	28.4
% days involving sex c	17.8	24.1
% days involving risky sex c	9.2	20.2

Note.

^aViral load is an index of HIV nucleic acid used to monitor HIV disease and guide treatment; values < 40 copies/mL are considered “undetectable.” CD4 is a measure of immune system function; CDC categorizes values < 200 as AIDS.

^bScreening assessment.

^cBaseline Timeline Followback interview; M quantity per drinking day reported in milliliters of 190-proof ethanol. ASSIST = Alcohol, Smoking and Substance Involvement Screening Test (range = 0 to 208; higher scores indicate greater substance involvement (World Health Organization, 2002).

Procedures

Interviews

Interviews were conducted in private locations. Participants received $20 Visa™ debit cards for initial interviews, $10 for initial IVR training, and $15 for follow-up assessments; main study participants also received a $25 debit card for completing all data collection. Pilot participants were re-assessed once using select baseline measures at the end of their 28-day IVR interval. Main study participants were re-assessed twice at about the mid-point and end of their 70-day IVR interval. At each assessment, alcohol and drug use, sexual activity, and money spent on alcohol and drugs were assessed using an expanded TLFB interview (e.g., Carey, Carey, Maisto, Gordon, & Weinhardt, 2001; Tucker et al., 2007). Baseline assessment covered the past 89 days in order to verify participant eligibility. Follow-up assessments covered the time elapsed since the baseline interview and served as reliability checks on IVR reports for the same interval. Drinking and other drug use were assessed on each reporting day using established TLFB procedures (Sobell & Sobell, 1992). Following Tucker et al. (2007), participants also reported money spent each day on alcohol and other drugs, regardless of when they were consumed.

IVR assessment

After the initial interviews, participants were trained to use the toll-free IVR system, which was programmed using SmartQ Version 5(5.0.141) (Telesage, Inc., Chapel Hill, NC). Training included definitions of all to-be-reported behaviors and events and practice using the IVR system. The same IVR protocol was used in the pilot and main studies, and included a daily survey, reported here, and a weekly survey, not reported here, administered each Monday. The daily survey assessed drinking, other drug use, and dollars spent on alcohol or other drugs during the preceding day (defined for participants as the 24-hours from midnight-to-midnight yesterday). When no alcohol or drug use was reported, participants were asked about other activities on the preceding day (e.g., social activities) to balance call duration.

Daily surveys also asked whether sexual activity occurred on the preceding day and, if so, the type of activity (with anal, oral, or vaginal sex queried separately), type of partner with whom each sex act occurred (main, non-main, or anonymous partner), other sexual risk behaviors (e.g., exchange of sex for money or goods, if alcohol or drugs were used before or during each reported sex act), and protective behaviors (e.g., use of barrier protection). A day was scored as a “risky sex day” if participants reported sexual activity with multiple or anonymous partners, substance use before or during sexual activity, unprotected sex, or exchange of money or other goods for a sex act. Average time for daily survey completion was < 5 minutes. To promote IVR compliance, participants accrued points for completing daily and weekly surveys that were modestly reimbursed using an “electronic bank” (Searles, Perrine, Mundt, & Helzer, 1995).

IVR and interview compliance

Ten of 54 participants interviewed at baseline (2 pilot, 8 main study) did not start IVR; 3 who did start provided very few days (< 3) of useable IVR data; and 5 who provided sufficient IVR data did not complete a follow-up assessment. Attrition was due to post-enrollment illness or death, moving out of the area, legal problems, or lack of interest. Phone access was not related to IVR utilization. IVR-TLFB reliability analyses were based on 36 participants (5 pilot, 31 main study) who completed a follow-up assessment (67% of the enrolled sample) and who provided a sufficient IVR-based data for comparison with the continuous daily TLFB reports covering the same real-time interval (≥ 10 days of IVR reports); having a sufficient IVR behavior sample was essential for evaluating concordance across methods of reports of the temporal patterning of risk behaviors day-to-day for individual participants. The reliability analyses used either the final scheduled re-assessment (n = 29) or the only available re-assessment (n = 5). Participants in the reliability sample tended to be younger (M = 37 years, p = .04), better educated (M = 9.8 years, p = .059), and less likely to report alcohol use at screening (73%, p = .044) than excluded participants (respective Ms = 43.7, 11.3, and 100%). There were no significant differences in gender, race, and economic variables, or in alcohol and drug use assessed at baseline or using TLFB data from the final follow-up.

Data analyses

Following Tucker et al. (2007), the IVR and TLFB data for each participant were matched in real time, and then summary measures of risk behaviors were derived by aggregating the observations over matched study days. Because the numbers of IVR-TLFB matched data points varied across participants, analyses were based on the percentage of matched observation days available for each participant (e.g., % matched observation days involving drinking) or on mean levels per occurrence day (e.g., mean ethanol consumption per drinking day). Pearson correlations were computed across participants using data aggregated over matched observation days, and the aggregated IVR and TLFB reports related to alcohol, drug use, and sexual activity were compared using correlated-groups t-tests.

In addition to analyses using summary measures, the consistency of reports of day-to-day patterns of risk behaviors was evaluated for each participant using percent agreements (based on the sum of days when occurrences were or were not reported on each measure) and the prevalence-adjusted-bias-adjusted kappa (PABAK) statistic (Lantz & Nebenzahl, 1996). Whereas the preceding analyses evaluated the reliability of IVR-TLFB reports at the group level summed or averaged over multiple observations irrespective of order, the latter analyses evaluated whether individuals reported the temporal patterning of risk behaviors in a reliable manner, which is important for event-level analyses of the role of substance use in risky sex (Leigh & Stall, 1993).

Results

Overall IVR compliance for the sample was 61.5%. The 5 pilot participants completed a mean of 20.0 (SD = 7.96) of 28 IVR days and a median of 24 days (M = 71% IVR compliance), and the 31 main study IVR participants completed a mean of 41.9 (SD = 20.94) of 70 IVR days and a median of 39 days (M = 60% compliance). Based on the daily IVR data, the percentages of the sample that reported any drinking, drug use, or risky sex were 58.3%, 69.4%, and 69.4%, respectively. The sample reported drinking very moderately on average (M = 0.77 standard drinks [13.7 ml ethanol] on drinking days), although 33% reported one or more risky drinking days (≥ 5 drinks/day for men, ≥ 4 drinks/day for women).

Concordance of IVR and TLFB Summary Reports of Substance Use and Sexual Behavior

Table 2 presents agreements for summary measures of drinking, drug use, economic variables, sexual activity, and the subset of risky sexual practices. Good to excellent IVR-TLFB correlations were found for variables related to drinking behavior, including the percentage of days involving drinking, quantities consumed per drinking day, the percentage of days involving spending on alcoholic beverages, and dollars spent per alcohol expenditure day. Although both methods produced similar reports of drinking practices, the IVR yielded significantly higher and presumably more complete reports of monetary allocation to buy alcoholic beverages. The mean number of days involving spending on alcohol was about 2.5 times higher on the IVR than TLFB, and the mean dollars spent per alcohol expenditure day was about 6 times higher.

Table 2.

Means and Correlations for Interactive Voice Response (IVR) and Timeline Followback (TLFB) Reports of Alcohol and Drug Use, Spending on Substances, and Sexual Practices (n = 36)

Variable	TLFB M (SD)	IVR M (SD)	r	p <	Md	p
Drinking behavior
% days involving alcohol consumption	15.9 (25.2)	15.7 (21.9)	0.76	0.0001	1.5	ns
Quantity per drinking day (ml ethanol) a	13.4 (30.2)	13.7 (33.4)	0.87	0.0001	1.5	ns
% days involving expenditures on alcohol	4.5 (8.6)	11.3 (20.0)	0.66	0.0001	5.5	0.027
Money spent on alcohol ($) a	0.39 (1.0)	2.5 (6.2)	0.86	0.0001	8.0	0.0009
Illicit drug use b
% days involving drug use	12.5 (22.6)	12.5 (19.8)	0.49	0.002	0	ns
% days involving drug expenditures	1.8 (3.7)	6.5 (11.7)	0.16	ns	4.0	0.096
Money spent on drugs ($) a	0.5 (1.1)	1.1 (2.4)	0.40	0.015	4.5	0.064
Sexual behavior
% days involving sex	19.4 (31.0)	19.4 (25.1)	0.67	0.0001	3.5	ns
% days involving risky sex c	14.9 (30.9)	15.1 (24.9)	0.66	0.0001	3	ns

^aMeans based on days that involved any drinking (mean ml 190-proof ethanol consumed per drinking day) or expenditures on alcohol or illicit drugs (mean dollars spent on ethanol or illicit drugs per expenditure day).

^bDrugs included marijuana, crack cocaine, powder cocaine, prescription drugs not prescribed to participant (TLFB); marijuana, crack cocaine, powder cocaine, methamphetamine, prescription drugs not prescribed to participant, opiates, and club drugs (IVR).

^cRisky sex defined as sexual activity that also included one or more of the following: multiple or anonymous partners, substance use prior to or during sexual activity, unprotected sex, or exchange of money or other goods for a sex act (“transaction sex”).

^dSign test for pairs of matched observations

Significant, but lower correlations were found for reports of days involving drug use and dollars spent per drug expenditure day. Reports of the percentage of days involving expenditures on drugs were not significantly correlated. Like the alcohol-related findings, both methods yielded similar reports of drug use, but the IVR yielded higher reports of money allocated to drug purchases. Reports of the percentage of days involving drug expenditures and mean dollars spent per spending day were about 3.5 and 2.5 times higher on the IVR than TLFB, respectively. Significant agreements were found for the percentage of days with sexual activity and risky sexual activity. There was no apparent tendency for either method to produce higher reports.

Repeated-measures analyses of variance evaluated whether reports varied across the IVR and TLFB methods by gender or race (white or non-white). No significant differences were found on any variable, with one exception. A significant Race x Method interaction, F(1, 34) = 5.44, p = .026, showed that non-whites reported significantly higher illicit drug use on the IVR (M = .118, SD = .143) than TLFB (M = .030, SD = .047; p = .021), whereas whites reported similar drug use on the IVR (M = .006, SD =.014) and TLFB (M = .005, SD = .013, ns).

Agreement between IVR-TLFB Reports of Day-to-Day Substance Use and Sexual Behaviors

Day-to-day agreement statistics were computed for each participant individually for reports of substance use, spending on substances, and sexual activity using all study days for which both IVR and TLFB reports were available. Table 3 summarizes the mean and median percent agreements and PABAKs for each variable and gives the numbers of participants in the sample of 36 who showed acceptable percent agreements (≥ 80%) and PABAK values (≥ .60).

Table 3.

Individual Agreement Statistics for Reports of Alcohol Consumption, Illicit Drug Use, and Sexual Practices (N = 36)

Variable	M Percent (range)		% Agreement				PABAK
	IVR	TLFB	M	Median	Range	% sample agreement ≥ 80%	M	Median	Range	% sample PABAK ≥ .60
% drinking days	16 (0-84)	16 (0-88)	81.5	92.5	34.8 – 100	61	.63	.85	-.30 – 1.0	61
% days with alcohol expenditures	11 (0-98)	5 (0-36)	87.9	95.3	36.9 – 100	75	.76	.91	-.26 – 1.0	81
% drug use days	13 (0-87)	13 (0-100)	83.4	97.7	12.9 – 100	67	.67	.95	-.74 – 1.0	67
% days with drug expenditures	7 (0-43)	2 (0-16)	91.9	98.4	56.9 – 100	81	.84	.97	.14 – 1.0	81
% days sex	19 (0-100)	19 (0-100)	79.1	84.9	2.4 – 100	58	.58	.70	-.95 – 1.0	61
% days risky sex	15 (0-100)	15 (0-100)	84.3	93.9	2.4 – 100	72	.69	.88	-.95 – 1.0	72

Note. IVR = Interactive Voice Response self-monitoring, PABAK = prevalence-adjusted bias-adjusted kappa statistic (see text), TLFB = Timeline Followback interview. Table based on 36 participants with ≥ 10 IVR-TLFB matched day pairs; M = 36.1 IVR call days (range = 10 – 70).

On all measures, most participants showed satisfactory to excellent agreements, although agreements ranged from perfect to very poor for a few participants. With few exceptions (see Table 3), the mean and median percent agreements were ≥ 80% for all variables, and the mean and median PABAK statistics were ≥ .60, indicating good to excellent concordance. A small number of participants had negative PABAK values for select variables; values of - 1.0 occur when the percentage agreement is exactly zero. Among participants with negative PABAKs for substance-related variables, most tended to report these behaviors on both the TLFB and IVR, with higher IVR reporting. For reports of sexual behavior, two of three participants with negative PABAKs reported sex and risky sex mainly on the TLFB, with few corresponding IVR reports.

Finally, summed across participants and observation days, we examined the extent to which IVR-TLFB reports agreed fully and the extent to which the agreements reflected reports of non-occurrences of target behaviors (i.e., abstinence) on both methods. For alcohol use, 78.52% of reports (1086/1383) were in full agreement, and 63.89% of agreements involved reports of abstinence. For sexual activity, 77.0% of reports (1071/1391) were in full agreement, and 55.56% of agreements involved reports of abstinence. These findings suggest that participants reported sequences of days involving abstinence and risky behaviors in a reliable, patterned manner.

Discussion

The results supported IVR telephone reporting as feasible for collecting quality data from persons living with HIV/AIDS in underserved, disadvantaged rural communities who have substantial health and functional limitations. The majority of participants called the IVR system at least once, and over half of callers completed more than 60% of scheduled daily calls. Acceptable to high IVR-TLFB agreements were found for summary measures related to alcohol consumption and sexual activity. The individual-level agreements further indicated that the great majority of participants reported the temporal patterning of risk behaviors in a reliable manner. The present study added to evidence that IVR systems are a flexible method for collecting quality reports in field settings on sensitive topics from diverse populations with risky health-relevant practices. The reliable individual reports of temporal patterning are promising with respect to investigating sequences of contextual and behavioral events involved in HIV-related risk-taking (Leigh & Stall, 1998). Such event-level assessment remains an important avenue for HIV prevention planning.

The very good agreements for summary measures of drinking practices approached levels found for problem drinkers (Sobell & Sobell, 1992), even though the sample was not selected for drinking problems. This suggests that reports of drinking practices can be obtained using either method (IVR or TLFB) across diverse samples. Agreements were less satisfactory for summary variables of illicit drug use. The lower frequency of drug use coupled with the greater ease of reporting alcohol intake in standard units may account for the higher drinking-related agreements.

Agreements for summary variables related to sexual activity were significant, although somewhat lower than the corresponding alcohol findings. There was no evidence of greater reporting of sexual activity, including risky sex, on the IVR compared to the TLFB. Both methods were useful over the assessment period covered in this research (≤ 70 days).

Although it remains to be determined if the agreement findings generalize to non-treatment-seeking samples at risk for HIV, the clinic sample disclosed and consistently reported risk behaviors using multiple methods throughout the study. In addition to the good IVR-TFLB agreements for reports of sexual and drinking practices, screening and baseline TLFB measures of alcohol, marijuana, or stimulant use during the past 3 months did not differ significantly.

Drug/alcohol spending was the only domain in which IVR reporting yielded higher and presumably more complete reports than the TLFB. This replicated our study of untreated problem drinkers who reported greater spending on alcohol after a recent recovery attempt using the IVR than TLFB method (Tucker et al., 2007). Although reliable reports of many risk behaviors can be obtained using the retrospective TLFB, prospective IVR reports appear better suited for natural environment studies of behavioral economic variables, such as the relative value of alcohol and drugs within individual environmental contexts (e.g., Tucker et al., 2007).

The present study has limitations. First, caution is necessary in generalizing results obtained with patients in a single rural clinic to other rural populations. Second, despite efforts to verify select participant reports using clinic records, this task was not completed sufficiently for use as a third data source in the reliability analyses. Third, a minority of enrolled participants did not use the IVR system much, if at all, which highlights the importance of identifying predictors of IVR engagement and utilization. Although some attrition and non-compliance were explained by participant circumstances (e.g., post-enrollment illness, death), predictors of IVR engagement need further study so that appealing IVR-based services can be developed and offered to persons who will use them. Related research questions include whether IVR utilization affects (i.e., reduces) risk behaviors and whether utilization rates are related to data quality. Future studies should capitalize on the utility of IVR systems as a platform for assessment and risk-reduction interventions to hidden and hard-to-reach populations living with or at risk for HIV.