Patterns in on-time, daily submission of a short web-based personal behavior survey in a longitudinal women’s health study

Abstract

We evaluated compliance with submitting a short web-based personal behavior survey daily during a 10-week study (n= 52 women/3,419 diaries). Timestamped forms revealed 50% of diaries were submitted within 24 hours of the email prompt and 19% were missing or submitted >3 days late. Late submissions may affect data quality.

Summary

In a longitudinal study requesting daily online completion of short personal behavior diaries, 19% of surveys were missing or submitted over three days late. Delayed submissions may indicate recall bias.

Introduction

Timely measurement of self-reported sexual and feminine hygiene behaviors are important in longitudinal studies of sexually transmitted infections (STIs).(1, 2) Biomarkers for sexual exposure, including Y-chromosome and prostate specific antigen detection in women, continue to emerge (1, 3, 4), however they are not in widespread use. Obtaining self-reported data, specifically sexual practices, can be challenging due to under- and over-reporting of socially undesirable and desirable behaviors, respectively.(2, 5) In addition, error in recall increases as more time passes between the events of interest and their reporting.(6, 7) One study found recall error was between 37%-51% when reporting 3-5 days following an event.(8) Daily surveys are often utilized to collect the most accurate and reliable prospective behavioral data.(9, 10)

Electronic data collection is a solution for some study populations as it allows anonymity, reduces social desirability bias, and in longitudinal studies, may facilitate increased frequency of reporting.(9, 11-16) Similarly, Ghanem et al. demonstrated that audio-computer assisted self-interview (ACASI) results in a more reliable assessment of sexual risk than face to face interviews.(17) Web-based surveys have been shown to increase survey completeness and maintain participant satisfaction compared to paper formats; however, there is little data on data accuracy and response rates in longitudinal studies using web surveys.(18)

In an observational longitudinal study of adult women, we sought to evaluate patterns of daily online survey submissions, as well as factors associated with timely submission.

Methods

This analysis reports on 52 participants recruited between May 2017 and December 2018 who completed the Gynecology and Lubricant Effects (GALE) study, a 10-week observational cohort study in Baltimore, MD that seeks to investigate changes in the vaginal microenvironment among women exposed to vaginal lubricants (galestudy.org). Women over 18 years of age who were referred for transvaginal ultrasound (TVUS) were eligible for the study. Possible reasons for referral included abnormal periods, uterine fibroids, ovarian cysts and pelvic pain. Exclusion criteria included diagnosis of diabetes, HIV or other immunosuppressive conditions, current pregnancy, or antibiotic or antifungal use in the month prior to TVUS.

Every evening, participants were emailed a link to a 30-question survey on REDCap (19), a web-based application for building surveys, which included questions on sexual behaviors, feminine hygiene, and partners for the prior 24 hours. Questions were formatted as yes/no and there were three comment fields for text (Supplemental Figure 1). Completion of the diary takes less than five minutes and could be completed on any device with an internet connection. No reminders were sent following incomplete diaries; however, study coordinators sent reminders to participants with consecutive incomplete diary days. Diaries were submitted by ID# and were therefore de-identified. The REDCap portal supplied a timestamp for submission. The GALE study included a $125 bonus at the final visit for 80% compliance with all study requirements including three clinical visits, weekly sample drop-offs, and diary submissions within 3 days of the initial email prompt. Diaries submitted over three days late would be considered “overdue”. This analysis was limited to the 52 women (3,419 submitted diaries) who used the web-based format to complete daily diaries. Six women who chose to submit paper copies of the diaries are excluded from this analysis.

Because there were some variations in study observation follow-up for each participant, we calculated the proportion of diaries that were either missing or overdue. We defined timeliness across the study as ≥70% of diaries submitted within three days of the initial email prompt. Altering this cutoff to 60% or 80% did not significantly alter our findings (Supplemental Figure 2). Diaries completed in batches were determined by subtracting the number of unique logins to REDCap from the total number of completed daily diaries per woman, leaving a proportion of those diaries which were submitted in batches of >1 diary. We sought to describe patterns of daily submission and baseline participant factors associated with timeliness. We conducted univariate Fisher’s exact tests to evaluate baseline variables that might be associated with timeliness. All statistical analyses were completed using StataCorp Software 14. In defining overdue diaries, we chose the cut-off of three days late because women were also tasked with self-collecting vaginal samples twice-weekly for the Parent study. This three-day interval allowed for relatively timely submission of diaries.

The GALE study was approved by the Institutional Review Board at the University of Maryland Baltimore (HP-#00061371).

Results

Participants included in this analysis ranged in age from 20-63 years, and the majority (71.15%) identified as African American, followed by white (15.38%), multi-racial (9.62%) and Asian (3.85%). Most (36.00%) reported an annual income between $20,000- $40,000, and 47.06% of participants did not report education beyond a high school diploma. The median number of lifetime sexual partners was 6.5 (range: 1-60). The majority of women reported never being married (48.08%), and most reported having children (57.69%).

Figure 1 displays the frequency of daily diary submissions per participant. As an example, participant ID# 227 completed diaries a few times per week in the first three weeks, followed by batched diaries submitted less than once a week. The time intervals between her submissions also tended to grow towards the end of study follow-up. Timely submission also appeared to decline at the end of the study for a number of participants (i.e./ID#s:235, 238, 245 and 248).

Figure 1. Patterns of daily diary submission in a longitudinal women’s health study.

The size of the diamond icons indicates the number of days late a diary was submitted, and a blank space indicates no diary was submitted that day. The smallest dots indicate a single diary submission within 24 hours of the initial email prompt. Sporadic gaps represent intervals in which no diaries were submitted, and multiple large diamonds reflect multiple late submissions. Participants are stratified as having greater or less than 70% of their diaries submitted within three days (on-time).

Daily survey data was submitted for over 91% (3419 of 3747) of observation days. Submitted diaries were highly complete, with responses recorded for over 98% of variable fields. Fifty percent of diaries were completed within 24 hours of the email prompt. No women submitted 100% of diaries within 24 hours of the email prompt, but half of participants (n=26) had no diaries more than three days late (overdue). Nine percent of diaries were missing, and 10% were submitted more than three days late. A median of 27.4% of diaries per participant were submitted in batches (range: 1-79%).

Participants who had ≥30% of their diaries missing/overdue were categorized as “<70% timely” in submissions (n=12, 23%). This subset of participants was responsible for the majority of large diary batching. Among the <70% timely participants, an average of 48.1% of diaries were missing/overdue, whereas among the ≥70% timely participants (n=40, 77%), an average of only 8% of diaries were missing/overdue. Among ≥70% timely participants, 20.7% of diaries were completed in batches, but only 1.7% of diaries were overdue, suggesting that the overwhelming majority of these batches were done in small 1-3 day increments. Among <70% timely participants, 64.5% of diaries were completed in batches and 32.6% of diaries were overdue, suggesting larger batching.

Women who reported greater number of lifetime sex partners were more likely to have less timely submissions (p-value: 0.009). Sixty-three percent of the women with <70% timely submission reported 11 or more lifetime partners compared to 23% of respondents with timely diary submission. Women who smoked also tended to be less timely with submission, although the finding was not statistically significant (p-value: 0.2). Other factors, including children in the household, participant age, number of sexual partners in the 2 months prior to enrollment, race/ethnicity, education, income, marital status, TVUS indication/findings and other behavioral factors were not significantly different between groups.

Discussion

In this cohort study of women referred for transvaginal ultrasound, participants had both high compliance with submission (91%), as well as completion of all survey fields, in a longitudinal study with daily online behavioral diaries; however, diaries were often submitted later than the study requested and observation days were often submitted in batches. These findings are consistent with those of previous studies and suggest access to electronic timestamped forms may allow for easier participation and may enable researchers to better discern timeliness and reliability of data.(16, 20)

We evaluated our baseline survey to determine if there were any factors that were associated with timely submission. Such information could inform study staff to be more aware of challenges that participants face in cohort studies and make arrangements so that diary submission is less onerous. We found only one factor (greater number of reported lifetime sex partners) was associated diary submission delay. Other factors were not statistically significant. The strength of this study is that the volume of diary time points (n=3,419) in Figure 1 allowed us to visualize a descriptive presentation of submission patterns. The study was not designed to formally evaluate the web-based instrument.

Late or batched diary data may be more problematic for behaviors which are not usually practiced, such as feminine hygiene practices or specific sexual behaviors, or for self-report of transient urogenital symptoms. The batching of surveys may also reflect that self-collected biological samples, in our case vaginal specimens, may also have been delayed in collection or collected in batches. Based on these findings, future studies should consider utilization of time-stamped forms either by web-based surveys or mobile devices. Weekly meetings between study coordinators and participants during sample drop-offs could benefit from enhanced focus on submission patterns. Studies may also consider utilizing different monetary bonus schemes, such as weekly bonuses or scaled bonuses based on submission participation, to encourage on time submission of diaries and samples.

Supplementary Material

Supplemental Digital Content_1

Supplemental Figure 1. GALE Study Daily Diary. A REDCap link to the daily diary was emailed every 24 hours to each GALE study participant. The survey did not change significantly across the time course of the study, and links did not expire. Participants could access the survey from any device with an internet connection, including a phone, tablet or computer.

Supplemental Digital Content_2

Supplemental Figure 2. Proportion of diaries missing or late across GALE study participants. The majority of participants (94%) completed either >80% of timely submissions (n=37 women, 71%) or less than 70% (n=12 women, 23%). Three participants completed between 70-80% of diaries, two of whom missed the 80% mark by .8% and .5%, respectively. For this reason, we stratified our timeliness submission categories to <70% timely and ≥70%. When analyzing the data at 80% and 60% stratification levels, results did not significantly change.