Convergence of Online Daily Diaries and Timeline Followback Among Women at Risk for Alcohol Exposed Pregnancy

Abstract

Researchers and clinicians interested in assessing drinking and unprotected sex in evaluating risk for alcohol-exposed pregnancy (AEP) have limited options. The current investigation examined the degree to which data collected from online prospectively collected daily diaries (Diaries) converged with data from interviewer-administered retrospective timeline follow back (TLFB), the standard in AEP intervention studies. 71 women (M_age=27.7, SD=6.2) at risk for AEP were recruited via online advertising and were randomly assigned to an online patient education condition or a tailored, online internet intervention to reduce AEP risk. All participants were administered both Diaries and TLFB at baseline and 6 months after intervention. Key outcomes were variables of drinking rates and unprotected sex that combined to indicate risk for AEP. Zero-order and intra-class correlations (ICC) between Diaries and TLFB were strong for each outcome. Examination of ICC confidence intervals indicated that condition assignment did not have a significant impact on the degree of convergence between Diaries and TLFB. With the exception of proportion of days drinking and proportion of days with unprotected sex at baseline, none of the paired t tests reached significance. Examination of descriptive statistics revealed that 63% of participants reported problem alcohol use and unprotected sex in both the 10-day Diaries and 90-day TLFB at baseline, with 70% agreement at post 6-month follow up. Findings indicate overall strong agreement between TLFB and Diaries in detecting alcohol use and unprotected sex in women at risk for AEP, and each method has benefits and challenges that should be weighed carefully by researchers and treatment providers.

1. Introduction

Alcohol exposure can have a critical impact on the organ systems of a developing fetus, leading to fetal alcohol spectrum disorders (FASD; Abel, 1990; Niccols, 2007). In particular, for women who have an elevated risk of an alcohol-exposed pregnancy (AEP) because of their engagement in frequent drinking and unprotected sexual intercourse, primary prevention of FASDs is a top priority (Floyd et al., 2009). Thus, researchers and treatment providers must have multiple ways to assess drinking and unprotected sex in order to develop and evaluate interventions that attenuate AEP risk.

A variety of retrospective self-report measures are used to assess alcohol use (e.g., Allen & Wilson, 2003; Sobell et al., 2003) and risky sexual behavior (for a recent review, see Mirzaei et al., 2016). Among these, the Timeline Followback (TLFB), a daily estimation instrument that is administered in a single interview, is the standard in AEP intervention studies (e.g., Ceperich & Ingersoll, 2011; Farrell-Carnahan et al., 2013; Floyd et al., 2007; Ingersoll et al., 2013; Parrish et al., 2016). While TLFB allows data to be captured in a single time point, the interview required to collect this data can be lengthy and burdensome to both the interviewer and interviewee. This can be problematic for researchers and treatment providers that want to assess key behaviors in AEP risk while minimizing burden and costs. Thus, identifying alternative methods that can capture the frequency of drinking and unprotected sex, while reducing burden, could prove worthwhile.

One obvious alternative is to utilize a web-based, prospective daily diary approach (Diaries). In 2015, 84% of American adults and 96% of young adults reported using the Internet (Perrin & Duggan, 2015). Although there exists some disparity in internet access, in 2016, 88% of Hispanic adults and 85% of Black adults reported using the internet (compared to 88% of White adults), as well as 79% of those with an income under $30,000 annually (“Internet/Broadband Fact Sheet”, 2017). Like TLFB, Diaries can capture the frequency of drinking and unprotected sex. While one disadvantage is that collecting data using Diaries requires multiple days and time points, increasing the likelihood of missing data, a clear advantage to Diaries is that participants can complete entries quickly and with limited support, and online data collection eliminates traditional obstacles of pencil-and-paper measures (e.g., transporting data, data entry errors).

While previous studies have compared and contrasted TLFB with other self-report measures (e.g., Sobell et al., 2003), no published studies have compared TLFB with Diaries in assessing behaviors linked to AEP. Thus, while TLFB has strong empirical support as a valid and reliable tool in assessing AEP risk (e.g., Ceperich & Ingersoll, 2011; Farrell-Carnahan et al., 2013; Floyd et al., 2007; Ingersoll et al., 2013), it is worth examining Diaries as an alternative method that can be adopted by researchers and treatment providers depending on available resources.

The aim of the current investigation was to examine the degree to which data on AEP risk collected from online Diaries would converge with data from interviewer-administered TLFB. Data were drawn from a study testing the efficacy of an online program designed to reduce AEP risk, and therefore drinking and unprotected sex served as the primary outcome variables. In the present investigation, all outcome variables were examined at baseline and at 6-month follow up, which allowed for determining whether convergence between TLFB and Diary was affected by an active intervention. Because the details of this intervention study are presented in a separate paper (Ingersoll et al., in preparation), and because the current investigation focuses on comparing TLFB and Diary methods, only details relevant to the current study are presented.

2. Method

2.1. Participants

The majority of participants responded to craigslist ads placed in the “seeking volunteers” section in selected cities across the U.S. Applicants answered an initial interest form online and a research coordinator had a follow-up phone call with applicants who met the initial criteria to assess their drinking and intercourse behaviors over the past 90 days. Eligibility criteria included: female; age 18–44; not surgically sterile; reports sexual intercourse with a man in the past 90 days; using no contraception, intermittent contraception, or ineffective methods of contraception; reports drinking above NIAAA recommended levels for women (more than 3 standard drinks per occasion or more than 7 per week, on average); and has regular access to a computer connected to the Internet. The current study was based on data from 71 eligible women (M_age=27.7, SD=6.2) who completed the online study forms indicating interest in participation and ultimately enrolled in the intervention study. Participants had an average of 15.2 years of education (SD=2.3), and were 63.4% White, 19.7% Black, 4.2% Asian, and 12.7% other or Multiracial. The study was approved by the Institutional Review Board at the University of Virginia and participants were treated in accordance with the ethical standards of the Helsinki Accords.

2.2. Conditions

Participants were randomly assigned to one of two Internet conditions. Half were assigned to a patient education condition, which provided accurate but untailored and static information about reducing the risk of alcohol-exposed pregnancy. The other half was assigned to the experimental condition, the Contraception and Alcohol Risk Reduction Internet Intervention (CARRII.org), which featured information that was tailored, personalized, and highly interactive. CARRII includes six cores that are metered out weekly, to mimic the pace of face-to-face counseling, and to allow for between-core behavior change. CARRII content was adapted from CHOICES (Floyd et al., 2007), the most potent AEP risk reduction intervention, and featured components designed to convey the tone of a Motivational Interviewing counseling style as well as behavioral therapy components.

2.3. Instruments

Only the two instruments relevant to this project will be described. First, the TLFB is a self-report, retrospective daily recall procedure that guides the respondent to identify routine and unusual work/school and social events to anchor days to specific activities. TLFB yields a daily count of standard drinks and sexual intercourse episodes, with additional specification of type of contraception used and how it was used to assist the interviewer in determining likely effectiveness. At baseline and 6-months after intervention, TLFB was used to obtain day-by-day estimates of drinking and unprotected sex over the previous 90 days, a timeframe adopted in prior AEP studies (e.g., Ceperich & Ingersoll, 2011; Farrell-Carnahan et al., 2013; Floyd et al. 2007). For TLFB, data is collected via face-to-face or phone interview by a research coordinator who presents participants with a calendar to identify important dates that serve as memory prompts for problem drinking and unprotected sex. TLFB is a well-validated assessment tool for alcohol intake with good psychometric properties (e.g., Sobell & Sobell, 2000, 2003). The version used in this study was previously modified to include intercourse events, contraception behaviors, and menstrual periods (e.g., Ceperich & Ingersoll, 2011; Farrell-Carnahan et al., 2013; Floyd et al., 2007; Ingersoll et al., 2013; Parrish et al., 2016). In the present study, average interview lengths at baseline (not including the consent process) and 6-months after intervention were approximately 75 minutes and 45 minutes, respectively.

In addition to TLFB, all participants completed online diaries at baseline and 6-months after the intervention period. Diaries were administered and completed online, in people’s natural environments, and participants were required to complete 10 diaries over a 14-day span. Diaries were used as a data collection method and those at pre, post, and follow-up were administered to both conditions and were not considered part of the intervention. Self-monitoring was encouraged during the CARRII intervention, but those diaries were not used as data. Because there is no prior literature on timeframe for Diaries in AEP risk, a 14-day span was chosen to minimize user burden while allowing for enough time to assess occurrence of drinking and risky sexual behavior. Prior research using Diaries have utilized a one to two week span (e.g., Buysse et al., 2006; Ritterband et al., 2017). Diaries prompted participants to report on behaviors such as daily drinking and risky sexual behavior, which were also assessed with TLFB. Online diaries are widely used in other areas of health research and have good psychometric properties (e.g., Buysse et al., 2006; Ritterband et al., 2017). Table 1 contains questions in TLFB and Diaries.

Table 1.

Questions asked in TLFB and Diary.

Questions asked for each day	TLFB 90 days	Diaries 10 days
Did you drink alcohol this day?	✓	✓
If drank alcohol, how many standard drinks?	✓	✓
If drank alcohol, what was your main drinking location?	✗	✓
How many times did you have intercourse this day?	✓	✓
If had intercourse, which method of contraception used (if any) for each time?	✓	✓
Was contraception used exactly as directed?	✓	✓
Specify type of birth control pill.	✓	✗
If missed birth control pill, did you double-up the next day?	✓	✗
If missed birth control pill twice in a row, did you double up the following two days?	✓	✗
Were you on your period this day?	✓	✗
For those who reported becoming pregnant, were you pregnant this day?	✓	✗

Because TLFB and Diaries encompassed different lengths of time (90 vs. 10–14 days, respectively), primary outcomes were computed to be time-invariant. Specifically, for both TLFB and Diaries, the main outcomes examined were: (1) proportion of days with at least 1 drink; (2) proportion of binge drinking days (defined as ≥4 drinks in a day, based on NIAAA guidelines); (3) average number of drinks per day; and (4) proportion of days with unprotected sex. Proportions were calculated by dividing the total number of days on which a behavior occurred (e.g., binge drinking) by the total number of days covered by that particular instrument.

2.4. Plan of Analysis

Multiple tests were conducted to examine the degree of convergence between Diaries and TLFB. First, zero-order correlations were computed between these instruments, for each of the drinking and unprotected sex outcomes, at both baseline and 6-month follow up. Second, intraclass correlations (ICC type=absolute agreement, single measure) were computed to examine the degree of consistency in reporting on key outcomes from Diaries and TLFB. ICC values range from 0 to 1, with higher values indicating greater reliability of ratings between instruments. Finally, paired sample t tests (two-tailed) provided an additional rigorous test of the differences in key outcomes. Given the use of repeated univariate tests for multiple variables, a conservative significance level of .001 was used for all tests (Sobell & Sobell, 2003). Previous work has found moderate to large effect sizes (ICC’s=.6–.8; Sobell et al., 2003) representing a high degree of agreement between TLFB and other self-report instruments. The current sample size allowed for reliably detecting moderate to large effect sizes with 80% power, with an error rate of 5%.

3. Results

Descriptive statistics and test results can be found in Table 2. Zero-order correlations between TLFB and Diaries were strong for each of the outcomes (r=.63–.83; all p’s<.001), indicating considerable shared variance between these instruments (r²=.40–.69). Similarly, ICC’s were strong for each outcome (all p’s<.001) and were similar in magnitude to prior literature comparing TLFB with other self-report instruments (e.g., Sobell & Sobell, 2003). In addition, ICC confidence intervals overlapped for each pair of outcome variables at baseline and 6-month follow up (e.g., proportion of days binge drinking at baseline and 6-month follow up), indicating that the intervention did not have a meaningful impact on the degree of convergence between TLFB and Diaries. Finally, with the exception of proportion of days drinking and proportion of days with unprotected sex at baseline, none of the paired t tests reached significance.

Table 2.

Descriptive statistics and test results between TLFB and Diary in assessing drinking and risky sexual behavior, at baseline (top) and 6-month follow up (bottom).

		Descriptive Statistics
Variable		Diary	TLFB	r	ICC	t test
Baseline
Proportion of days drinking		.50 (.26)	.38 (.23)	.68* [.54, .80]	.61* [.32, .77]	4.98*
Proportion of days binge drinking		.21 (.21)	.17 (.18)	.83* [.70, .91]	.81* [.70, .88]	2.67
Drinks per day		1.86 (1.40)	1.59 (1.60)	.78* [.67, .88]	.76* [.64, .84]	2.21
Proportion of days of unprotected sex		.30 (.24)	.21 (.17)	.62* [.42, .77]	.54* [.31, .70]	3.93*
Post 6-Months
Proportion of days drinking		.29 (.23)	.26 (.21)	.78* [.63, .88]	.77* [.64, .86]	1.38
Proportion of days binge drinking		.11 (.13)	.08 (.10)	.80* [.64, .91]	.74* [.57, .84]	2.92
Drinks per day		1.05 (1.06)	.85 (.80)	.65* [.41, .87]	.61* [.43, .75]	1.82
Proportion of days of unprotected sex		.14 (.17)	.08 (.11)	.63* [.39, .80]	.56* [.34, .71]	2.78

^*p<.001.

Note. Means (SD’s), zero-order correlations (r), intraclass correlations (ICC; absolute agreement, single measure), 95% confidence intervals (brackets), and paired sample t tests.

Since a prerequisite for study inclusion was the presence of both unprotected sex and alcohol use above recommended levels based on a 90-day TLFB, all participants reported both behaviors at the baseline TLFB. Based on diary responses at baseline, 45 of 71 participants (63%) reported the presence of both types of risky behavior occurring over a span of 10 days. At post 6-month follow-up, 41 of 59 (70%) participants had consistent reporting of the presence or absence of both alcohol use and unprotected sex across their TFLB and Diary data.

4. Discussion

Findings from the current study indicate overall strong agreement between TLFB and Diaries in detecting alcohol use and unprotected sex in women at risk for AEP. These results suggest that Diaries may be considered as an alternative when TLFB is not possible or feasible. As noted in Table 3, choosing which instrument to use can be based on available resources, including time, staffing, and funds. Specifically, TLFB may be the preferred choice when there is a need to collect a continuous stream of data at a single time point. TLFB may also be preferable when there is a strong need to collect fine-grained information (e.g., qualitative information such as exactly where and when behaviors took place), because experimenters and clinicians have the opportunity to clarify ambiguous responses and ask follow-up questions. However, in order to use the TLFB approach, sufficient time and staff to conduct individual interviews is required, and it is important to keep in mind that interviews can be difficult to schedule and vary in the amount of time each one takes to complete. For example, in the current study, interviews at baseline generally ranged between 15 and 135 minutes to complete, with an average of approximately 75 minutes. This can be a significant barrier for researchers, and in particular for treatment providers, who wish to adequately gauge AEP risk in their clients while having limited ability to commit the necessary time and energy into conducting interviews.

Table 3.

Benefits and challenges associated with TLFB and Diary methods for AEP risk.

Timeline Followback		Online Diaries
Benefits	Challenges	Benefits	Challenges
▪Data captured in a single time point ▪Minimizes likelihood of missing data ▪Opportunity to adapt to responses and ask follow-up questions ▪Previous research establishes norms for period length between assessments (e.g., 90 days) ▪Phone or face-to-face sessions require little technical ability	▪Interviews can be lengthy ▪Difficulty in scheduling sessions/keeping appointments ▪Participants may feel judged by experimenter in disclosing risky behavior ▪Greater reliance on long-term memory of events ▪Requires extra staff/time	▪Less time to complete an entry ▪Does not require an experimenter or clinician present ▪Participants may feel less judgment in disclosing risky behavior ▪More frequent assessments means less reliance on long-term memory ▪Limited administrative oversight	▪Requires multiple time points ▪Increases potential of missing data ▪Questions are preset ▪No established guideline for period length can make it harder to capture lower base rate events ▪Requires internet (if administered online) ▪May require extra motivation

In contrast, Diaries allow researchers and treatment providers to bypass potential issues that arise with scheduling and keeping TLFB appointments, since no interviewer is needed and questions can be delivered automatically online. Increasingly, given that online platforms are increasingly numerous, cost efficient, and many do not require technical expertise, researchers and treatment providers are adopting online approaches to disseminating information and collecting data. Although it is possible that Diaries will not actually save total time for some participants due to ongoing and frequent administration, the variation in time it takes to complete Diaries is likely to be less than that of TLFB since questions are preset. Thus, Diaries may be preferable when individuals are most concerned with assessing the frequency and/or intensity of behaviors, which is typically the case when measuring the impact of an intervention. However, the savings in time and energy associated with online diaries to researchers and treatment providers can be substantial.

Because there are no clear guidelines of period length (e.g., 20 days vs. 10 days) when using Diaries, researchers and treatment providers must consider the intermittency of behaviors they wish to assess. In the current study, the application of a 14-day Diary timeframe may have limited the ability to detect behaviors compared to the 90-day TLFB timeframe. For women at risk for AEP, engaging in unprotected sex generally occurs less frequently than does drinking alcohol. Ideally, the period encompassed by Diaries should be long enough to capture relatively low base rate behaviors (e.g., sexual intercourse), while being short enough to minimize participant burden. For example, based on the current study, unprotected sex occurred on roughly 20–30% of days assessed. Therefore, if participants completed 20 days of Diaries, this would be expected to capture 4–6 days with unprotected sex, on average.

Interestingly, self-reported rates of risky drinking and sexual behavior were actually slightly higher for Diaries relative to TLFB, leading to several possible explanations. One possibility is that because online Diaries are administered on a daily basis, they are less reliant on long-term memory than a typical TLFB. This may be particularly relevant for certain populations, such as those at high risk for AEP, who frequently engage in risky behaviors (e.g., drug and alcohol use) that can erode memory and cognitive abilities over time (e.g., Stavro et al., 2013). Another possibility is linked to how data is collected via Diaries and TLFB. There is ample evidence in the health sciences that people’s behavior is influenced by their awareness that they are being actively observed (e.g., De Amici et al., 2000; McCarney et al., 2007). Participants may under-report risky drinking/sexual behaviors in their TLFB in order to minimize shame and embarrassment. By contrast, in the current study, Diaries were automated and delivered online, which might have reduced potential self-consciousness about potentially embarrassing behaviors. Indeed, studies have shown that individuals tend to be more willing to disclose sensitive behavior (e.g., sexual behavior, drug and alcohol use) in self-report and online questionnaires than with in-person communication with a researcher (e.g., Aquilino, 1994; Buhrmester et al., 2011; Catania et al., 1986; Gribble et al., 1999). This may be particularly relevant for racial and ethnic minorities (e.g., Aquilino, 1994), where mismatch between experimenter and participant demographics can lead to mistrust of the experimenter by participants.

Although the current investigation focused on comparing TLFB and Diary methods in assessing behaviors associated with AEP risk, both of these are self-report instruments, which raise the possibility of common method bias in influencing findings. For example, the observed agreement between TLFB and Diaries may be partially attributed to systematic shared error variance arising from the introduction of a common rater/participant. Thus, future work may wish to test how well TLFB/Diaries correlate with passively and continuously collected data afforded through mobile technology, such as the relation between self-reported binge drinking and continuously monitored blood alcohol levels. Given its convenience and accessibility, mobile technology also has the potential to extend the timeframe for self-reported data collection. Further, because the majority of participants responded to an online Craigslist advertisement, future work should examine the potential of different online recruitment methods to produce biased samples. One issue that future work should continue to address is how to motivate individuals to complete Diaries when they are away from the clinic. For example, research on Internet interventions highlights the importance of user preferences and increasing interactive components (e.g., Ritterband, Thorndike, Cox, Kovatchev, & Gonder-Frederick, 2009), while research in motivation suggests that a system of rewards and positive feedback for completing Diaries may be helpful in maintaining motivation (e.g., Deci, Koester, & Ryan, 1999).

5. Conclusion

Researchers and treatment providers interested in assessing AEP risk need to make important decisions regarding how to balance the pros and cons associated with measures of alcohol use and risky sexual behavior. Importantly, selection of method should depend on several crucial considerations, such as resources available (e.g., time, money) and the importance of capturing sporadic behaviors that might occur over a lengthy period. While TLFB has received much support as a valid and reliable instrument in research studies, and should continue to be used in future work, the present research provides evidence that online diaries may be used to assess behaviors associated with AEP risk in both research and clinical settings. The present work suggests that Diaries may provide researchers and clinicians with a reliable means of assessing AEP risk when administering TLFB is not feasible.

Highlights.

• Examined convergence of online diaries with timeline follow back in AEP risk

• Outcomes were drinking rates and unprotected sex that convey risk for AEP

• Overall strong agreement between methods in detecting key outcomes

• Weighs pros and cons of each method to assist researchers and treatment providers