The Feasibility and Acceptability of Using Technology-Based Daily Diaries with HIV-Infected Young Men who have Sex with Men: A Comparison of Internet and Voice Modalities

Abstract

This study delivered a daily diary to 67 HIV-infected men who have sex with men (MSM) between 16 and 24 years old for 66 days to measure HIV-risk behaviors and other psychosocial variables via two diary modalities: internet (accessible via any web-enabled device) and voice (accessible via telephone). Participants were randomized to complete one diary modality for 33 days before switching to the second modality for 33 days. The study was implemented in three urban HIV health care centers in the United States where participants were receiving services. Through diary data and qualitative interview data, we examined the feasibility and acceptability of the dairies and identified barriers and facilitators of dairy compliance. Results show high participant retention in the daily diary (93.4%) and high compliance for the number of dairies completed (72.4%). Internet diaries were preferred by 92% of participants and completed at a significantly higher rate (77.5%) than voice diaries (67.7%). Facilitators included opportunities for self-reflection and cathartic sharing, monetary compensation, relationships with study staff, and daily reminders. Barriers included being busy or not having privacy at the time of reminders, forgetting, and falling asleep. Participants also described barriers and facilitators unique to each modality. Overall, both modalities were feasible and acceptable for use with our sample of HIV-infected MSM.

INTRODUCTION

Daily diaries designed to measure proximal predictors of HIV-risk behaviors may better allow researchers to identify strategies to decrease HIV transmission and improve the well-being of individuals living with HIV/AIDS. As technology-based methods for delivering daily diaries become increasingly available, researchers must determine if these methods are feasible and acceptable for use with HIV-infected individuals.

Two novel methods for implementing daily diaries include the use of telephones to deliver voice response diaries and the use of internet-enabled devices, including smartphones, computers, and tablets, to deliver text-based diaries. In comparison to pencil-and-paper methods, phone and internet-based technologies allow for data collection strategies that improve anonymity, confidentiality, and the accuracy and validity of data [1-5]. This may be especially important for collecting data on sensitive topics such as sexual behavior and substance use.

Furthermore, cell phone and internet technologies are widespread among adolescents and young adults, making these technologies appropriate candidates for data collection with this population [6]. Cell phone access among those disproportionately at risk for the contraction of HIV, including Black, Latino, and gay individuals [7], is especially high compared to White and heterosexual individuals [8, 9]. This highlights the potential for using cell phones to implement daily diaries with those most impacted by HIV, including young men who have sex with men (MSM) [7].

Diary studies involving electronic assessments also show greater response rates than diaries using pencil-and-paper assessments [10]. In Stalgaitis and Glick’s systematic review of 15 studies implementing web-based dairies to measure sexual risk behavior, diary completion ranged from 61.9% to 99.7%; of studies conducted with MSM, diary completion ranged from 78% to 99.7% [11]. This indicates generally high compliance, but the range of completion rates begs further investigation into which factors impact diary compliance. Stalgaitis and Glick’s review found no consistent relationship between dairy completion and other available factors, such as compensation per day, diary frequency, or the length of the diary collection period [11].

The handful of diary studies that have been conducted with HIV-infected adults suggests the use of electronic diaries with this group is feasible and acceptable for collecting sensitive data relevant to HIV/AIDS research, such as sexual and substance use risk, mood, social support, and coping. For example, Barta et al. used electronic daily diaries with 116 HIV-infected adults over 5 weeks to track vaginal and anal intercourse [12]. Barta and colleagues observed a compliance rate of 81%, and participants reported roughly 1,500 sexual episodes [12]. In addition, a feasibility study of 21 racially and ethnically diverse HIV-infected adults employed personal digital assistants to collect daily diary data on mood, support, and coping [13]. Eighty-one percent of participants completed two months of daily diaries, 67% of participants completed six months of daily diaries, and diaries were completed on 72% of the days of the study, indicating high compliance [13]. Lastly, an 86% response rate was observed in a study with 140 HIV-infected MSM using weekly structured, computer- and internet-based diaries to measure sexual behavior and drug use [14].

Despite research indicating the feasibility and acceptability of implementing electronic daily diaries with HIV-infected adults, little has been done to compare methods within studies and determine specific barriers and facilitators to dairy completion. Cook, McElwain, and Bradley-Springer note potential barriers using digital diaries with their small sample of HIV-infected adults, including problems remembering log-in information and measurement fatigue [13], and two studies using phone-based diaries with young adults have shown lower response rates on weekends [15, 16]; however, additional research is needed for a more nuanced examination of barriers and facilitators among HIV-infected adolescents. Furthermore, while daily diaries also have the potential to be used as tools in behavioral HIV interventions, more research is needed on how to best implement dairies with HIV-infected populations [5, 17]. Though internet and voice-based daily diaries may be ideal candidates for collecting sensitive data about factors related to HIV risk, including sexual behavior, medication adherence, and substance use, they have yet to be empirically tested among HIV-infected young MSM, who are disproportionally impacted by the HIV epidemic [7].

The present study filled this gap by delivering a daily diary to HIV-infected MSM between 16 and 24 years old via two daily diary modalities, interactive web response (i.e., internet) and interactive voice response (i.e., voice), to obtain in-depth quantitative and qualitative information on the strengths and weaknesses of using two widely accessible, but distinct, diary completion modalities. The aims of the present analysis were to 1) explore the feasibility and acceptability of using daily diaries with HIV-infected young MSM; 2) describe patterns in daily diary completion over time; 3) compare completion rates between voice and internet modalities; and 4) identify barriers and facilitators of voice and internet dairy compliance among HIV-infected young MSM.

METHODS

Sample

The present analysis includes data collected by the Adolescent Medicine Trials Network for HIV/AIDS Interventions, a nationwide collaborative research network. All participants received services at least once from adolescent HIV health care centers in Denver, Detroit, or Philadelphia, although we did not ask what services were received. Eligible participants self-identified as male at birth and at the time of screening. Additional eligibility criteria included being HIV-infected through sexual behavior, being between 16 and 24 years old at the time of enrollment, engaging in oral or anal sex with another man in the previous year, having consistent internet and phone access, and being able to read English at a fifth grade level. In order to enroll participants with a measurable level of sexual risk or substance use risk, eligibility criteria included reporting either at least one episode of unprotected anal or vaginal intercourse and/or having four or more alcoholic beverages in one sitting and/or two or more occasions of illicit drug use in the past 90 days.

Procedure

Participants were recruited from three clinical sites participating in the Adolescent Medical Trials Network for HIV/AIDS Interventions. Clinic staff approached potentially eligible clients during clinic visits to inform them of the study and set up a time to complete the eligibility screening.

All participants provided informed consent, and study activities were approved by the Institutional Review Board at all participating institutions. A waiver of parental consent was acquired for participants under 18. For the extra protection of minors, participants completed an assessment of understanding prior to providing consent.

After completing an assessment with a researcher to determine eligibility and to collect demographic data, enrolled participants were randomized on a 1:1 ratio within each study site to complete either the internet or voice diaries as their first modality. Participants received a written training guide and completed an interactive training with study staff on the use of their first modality. Participants then completed a baseline assessment via Audio Computer-Assisted Self-Interview Software (ACASI) to collect information on key constructs and covariates.

Participants completed their first modality for 33 days before switching to their second modality for 33 days. The first three days of the first modality (study days 1-3) and the first three days of the second modality (study days 34-36) were considered “calibration days” to ensure participants were generally able to comply with the schedule of diary assessments and to account for possible reactivity (i.e., participants’ responses might initially change by virtue of beginning to record their moods and behaviors). Participants were required to complete diaries on all three calibration days of their first modality, or report an extenuating circumstance, in order to remain on the study. After successful completion of the first three calibration days, no participants were removed from the study for non-compliance.

Participants met regularly with a researcher in person and over the phone for a total of two phone check-ins and five in-person visits, including training visits before the first day of each modality (days 1 and 31), phone check-ins after each three-day calibration period (days 4 and 34), follow-up visits midway through each modality (days 19 and 52), and a final debriefing visit (after day 66). The daily diary system allowed study staff to monitor diary compliance in real-time, and participants were contacted by study coordinators if they missed three or more consecutive diaries. The final debriefing visit included a one-hour, semi-structured debriefing interview with a researcher, which was conducted via webcam and audio recorded and transcribed for analysis.

Participants were asked to complete one daily diary per day between the hours of 18:00 and 06:00. Participants could log out of any diary and return to it at a later time within the 12-hour time period, but they could not complete a missed diary after 06:00. At 18:00 and 22:00, participants received an automated text-message reminder to complete their diary. Great care was taken to enhance participants’ privacy; for this reason, no information about the nature of the study was available until participants had entered their private login information, and the text-message reminders were vague and did not reveal participation in a research study. Participants also had the option of skipping any question.

The voice and internet technology was implemented by an independent company that specializes in developing software and managing secure databases for diaries used in research and clinical health care settings. The voice diary was accessible via cell phone or landline. Participants were instructed to call a toll-free number and enter a personal identification number and private password. A pre-recorded voice read each question to the participant who was asked to respond by pressing a number on the telephone keypad for multiple-choice questions or by leaving a voice recording for open-ended questions. The internet diary was accessible via any web-enabled computer, telephone, or tablet. Participants were given a link to a secure website and instructed to enter their identification number and private password. Each question was presented in readable text format, and participants clicked on buttons to respond to multiple-choice questions or typed their answers to respond to open-ended questions.

Participants were compensated for diary completion with an incentive structure that utilized tiered incentives, loss aversion, and variable reinforcement. A compensation account was started for each participant with a $25 deposit. Following this deposit, money was added for every diary completed and removed for every dairy missed. The amount added for each completed diary increased with the number of dairies completed, with diaries worth from $2 to $6. For every missed diary, participants lost $1. In addition, for every dairy completed, participants were entered into a lottery with a chance of winning $5. In total, participants earned an average of $197 for diary completion. Participants were also given a $35 gift card twice per month to cover the cost of cell phone and internet service.

Measures

The measures included in this analysis were part of a larger battery of assessments to explore relationships among mood, stressful events, social support, substance use, sexual behavior, and adherence to antiretroviral therapy among HIV-infected young MSM.

Screening and Baseline Assessment

The screening assessment and baseline assessment collected information on eligibility criteria, demographics, key constructs, and covariates. For the purposes of the current analysis, this included race, ethnicity, self-identified gender, recent history of sexual behavior and substance use, reading level, age, and mode of HIV transmission.

Daily Diary

The daily diary included questions on current mood and positive affect, daily stressors, HIV-specific stressors, emotional and practical support, substance use, HIV medication adherence, positive daily events, and anal and vaginal intercourse, including partner characteristics, HIV status disclosure, and substance use prior to intercourse. Participants who reported no sexual behavior were asked a series of questions on daily habits (e.g., exercise, taking a nap), so that these diaries would not be significantly shorter. Measures employed in the daily diary were used in previous diary studies and all items were evaluated with input from two community advisory boards made up of HIV-infected young MSM.

Participants who completed their previous diary were asked to report events that occurred in the time since completing their previous diary, whereas participants who missed a diary were asked to report events that occurred in the past 24 hours. When participants missed a diary, the following diary asked why the previous diary was missed, providing a measure of reasons for non-compliance. Open-ended questions asked participants to describe anything especially good that happened within the time period covered by the diary and to leave a message if there was anything else they wanted to mention to the study staff. Participants were reminded that this should not be used to report emergencies and were provided with alternative contact information for such purposes. The diary automatically recorded the start and end time and date for each dairy entry.

Check-in Logs

Check-in logs were completed by study staff during participant visits and phone check-ins. Included in these reports are reasons for missing the calibration period, reasons for discontinuation from the study, and any problems using the daily dairies, including loss of cell phone or internet service.

Debriefing Interview

The debriefing interview was conducted at a participant’s final visit, including premature discontinuation visits. Participants were asked about the usability of both diary modalities, general thoughts and feelings about completing the diaries, barriers and facilitators to diary completion, suggestions for future revisions, reactivity to the diary, and specific questions about the acceptability of using the diary to measure key constructs. Participants were also asked about reasons for completion or non-completion of the study. Each interview lasted up to an hour and was transcribed for analysis.

Analyses

The qualitative analysis includes debriefing interviews from 61 participants (two interviews were lost to technical errors, and four interviews were not completed). Research personnel used repetitions and cutting and sorting techniques using NVivo software to identify themes in debriefing interview transcripts [18]. In some cases, an independent coder quantified responses to key questions (e.g., the preferred diary modality) to obtain variable frequencies.

For quantitative daily diary data, SPSS was used to examine measures of central tendencies and compare categorical variables with chi-square tests for independence. Diary completion on a daily level was examined as a dichotomous categorical variable (i.e., either “completed” or “not-completed even though the dairy was available”). The six calibration days (days 1-3 and 31-33) were not included in the present analysis. For this reason, only participants who completed the first three-day calibration period and remained on the study until day four of their first modality are included in the daily diary results.

RESULTS

Sample Description

The final baseline sample included 67 HIV-infected young men who have sex with men. Participants ranged in age from 16 to 24, with a mean age of 21. The majority of participants identified as Black or African American (61.2%), and 11.9% of participants identified as Hispanic or Latino. About 16% of participants identified as non-Hispanic White. The majority of participants (95.5%) reported acquiring HIV through male-to-male sexual contact, and 34.3% of participants were taking antiretroviral therapy to treat their HIV. In the 90 days prior to the screening assessment, 67.2% of participants had two or more occasions of illicit drug use, 43.3% of participants reported having four or more alcoholic beverages in one sitting, and 70.1% of participants reported having unprotected vaginal or anal intercourse.

Study Retention

Sixty-seven participants completed the baseline visit and began the first calibration period. Of these 67 participants, six failed the calibration period and were removed from the study, and 61 participants continued on to the daily diary portion of the study. Reasons participants cited for failing the calibration period included falling asleep before completing the diary (2), losing cell phone service (1), being too busy (1), forgetting (1), and not being sure if the diary had already been completed (1). Participants who failed the calibration period did not differ from participants who completed the calibration period in race, ethnicity, age, or drug use and unprotected intercourse in the past three months.

Of the 61 participants beginning the daily diary portion of the study, 57 were retained throughout the 66-day period, for a 93.4% retention rate among those who completed the calibration period. Two participants who completed the 66-day daily diary period did not complete the debriefing interview.

Overall Diary Completion Trends

Overall, participants completed 2,558 of 3,532 expected voice and internet diaries, for a compliance rate of 72.4%. The proportion of completed diaries to available diaries per participant ranged from 0.10 to 1.00 (SD = 0.23). Participants completed significantly more internet diaries (77.5%) than voice diaries (67.4%) (X² (1, N = 3,532) = 44.607, p < .001). Participants who started with voice as their first modality completed 70.2% of voice diaries and 71.1% of internet diaries; conversely, participants who started with internet as their first modality completed 83.6% of internet diaries and 64.6% of voice diaries. In other words, participants starting with voice showed a 0.9 point increase in the percent of diaries completed when switching to internet, whereas participants starting with internet showed a 19 point decrease when switching to voice. In addition, participants with low compliance on one modality tended to also have low compliance with the other modality (16.9% were low on both; 25.4% were average on both; 20.3% were high on both); no participants had low compliance on one modality and high compliance on the other modality. Compliance rates did not show significant relationships with race/ethnicity, age, or whether or not participants used drugs or alcohol or had unprotected anal or vaginal intercourse in the three months prior to the baseline assessment.

By splitting each 30-day modality into five equal periods of six diaries each, we were able to observe that diary completion decreased over time (X² (4, N = 3,532) = 25.709, p < .001), although when completion over time was separated by modality, a significant decrease in completion over time was only present with voice diaries (see Figure 1). Voice completion decreased from 75.7% (time period 1) to 61.7% (time period 5) ( X² (4, N = 1,778) =20.399, p < .001). In contrast, internet completion decreased from 82.5% (time period 1) to 75% (time period 5).

Figure I. Daily Diary Compliance for Voice and Internet Modalities over 30 Days.

The percent of daily dairies completed during 30 days (split into equal time periods of 6 days each), with voice (IVR) and internet (IWR) modalities displayed separately.

A greater percentage of diaries was missed on Fridays and Saturdays (32.5% missed) compared to the rest of the week (Sunday to Thursday, 25.5% missed) (X² (1, N = 3,532) = 18.216, p < .001); this was true for both internet (X² (1, N = 1,754) = 7.981, p = .005) and voice (X² (1, N = 1778) = 10.815, p = .001).

Diaries were completed most frequently within the half hour after reminders were sent at 18:00 (13.4%) and 22:00 (11.1%). The median time between diary start time and diary completion time was 7 minutes 02 seconds. Voice diaries took more time to complete (median = 8 min 06 seconds) than internet diaries (median = 4 min 32 seconds).

Participants filled out their voice and internet diaries using various technologies. The majority of participants used cell phones exclusively to access the voice diaries, although two participants used landlines when cell phone service was unavailable. To complete their internet diaries, participants reported either using internet-enabled mobile phones exclusively (60%), computers exclusively (12%), tablets exclusively (4%), or a combination of devices (25%). Over the course of the study, 21% of participants reported temporarily losing cell phone or internet service, but all participants who completed the first calibration period were able to restore cell phone service before their subsequent check-in visit.

Facilitators of Diary Completion

The debriefing analysis provides evidence that participants found the study to be an enjoyable experience, with one participant noting, “I liked it; I really did, and I would love to continue. You know, I would do it and not even get paid.” See Table 1 for additional quotes. A number of facilitators to diary completion were identified. Participants positively noted that the diary was cathartic (“They might not know it’s you, but at least you told somebody. It makes you feel a little bit better”) and engendered self-reflection and behavior tracking (“It’s just something that you would like to do to understand yourself a little better, about what’s going on in your life”). In addition, participants noted the motivating nature of the compensation scheme, especially the inclusion of the daily lottery (“When you win and you know you are capable of winning, then you’re going to consistently call in”). Participants also described having motivating relationships with study staff (“she definitely has a good way of being on you about doing what you need to do but not making it seem like she’s getting on your case”).

Table I.

Selected Thematic Categories from Qualitative Analysis with 61 HIV-Infected Men Who Have Sex with Men

Main Theme	Quote
	Facilitators
Acceptability	I feel like it was a good thing to take part in, and I definitely would do it again.
Catharsis	It reminds you, hey, if you ain’t got somebody to talk to, at least you can write it down or at least it’s out there…It makes you feel a little bit better.
Self-reflection	I liked the diary. It was a way of letting me be able to keep track of certain things that I wanted to keep track of.
Compensation	I think it was great, especially every time you log on [to the diary] it tells you how much [money] you have, and you're like, "Okay, I'll keep doing this.”
Study Staff	I loved it because with [study coordinator], we had a very good relationship.
Reminders	The reminders really helped.
	Barriers
Busy, forgetting, and falling asleep	I was busy with work, and by the time I get home from work, it’s late at night…So, by the time I get home, I'm super tired from work and I kind of sometimes am like, I'm just about ready to get to bed and I'm like, “Oh, wait, I gotta do my phone diary before I go to bed.” It’s like, I almost forget, just because it’s been so busy throughout the day.
	Barrier Management Strategies
Completing right away	I had to tell myself, “You gotta do the diary when they send you the text…”
Additional reminders	I printed the phone number and my code and I stuck it on a big sheet of… huge poster sized paper.
Memorizing questions	It became such a routine, though, I knew what they were gonna ask me, so I would do it quicker than usual.
Incorporating daily routine	I actually changed my schedule… the 6:00 text is like also a reminder for me to take my meds…
	Voice vs. Internet
Internet more convenient	I was able to read it [the IWR diary] a lot faster than I could hear the phone and… it didn't take up quite as much of my attention as the phone did.
Voice more personal	I thought it was cool because it was kind of like it was running a conversation with somebody else…And I just, I think it was more interactive and kind of personalized.

Barriers to Diary Completion

Of the 61 participants who completed the calibration period, 58 participants missed one or more of the 60 non-calibration diaries. When completing a diary that followed a missed day, participants were asked to pick a reason for why they did not complete their previous diary; those who missed more than one diary could have indicated multiple reasons for missing diaries. Overall, 58 participants provided a total of 511 responses for reasons for diary non-compliance. Approximately 71% of the 58 participants reported at least once that forgetting was the reason they missed a diary, 67% reported at least once that being too busy was the reason they missed a diary, 66% reported at least once that being too sick, injured, or tired was the reason they missed a diary, 64% reported at least once that not having enough privacy was the reason they missed a diary, 55% reported at least once that experiencing a technical issue or not having phone or internet access was the reason they missed a dairy, 22% reported at least once that not feeling like completing the dairy was the reason they missed a diary, and 10% reported at least once that “other” was the reason they missed a dairy.

In addition, qualitative analyses revealed that a common narrative for why diaries were missed was being busy or not having privacy at the time the reminders were received and then forgetting to fill out the diary or falling asleep (“I’d go to school [at] 5:00 and like have school ‘till 10:00, go home, do homework, and I’d forget about it and I’d fall asleep”).

Participant Barrier Management Strategies

Participants noted multiple methods for managing barriers and capitalizing on perceived facilitators. This included completing the diary at the time of the text message reminders, setting additional reminders such as visual reminders (e.g., sticky notes or adding the internet diary website to a device’s desktop) or alarms, and using headphones or speakerphones for voice diaries. Some participants made the diary part of their daily routine by completing it during their commute or when taking medications. In addition, some participants reported memorizing questions and response options; however, although repetitions allowed for memorization, some participants noted being irritated or frustrated by the repetitive nature of the diary questions.

Comparing Voice to Internet

Participants’ reasons for missing the previous diary differed between voice and internet diaries. To compare reasons for missing each diary by modality, eight separate chi-square analyses were conducted with each reason for missing the previous diary separated into a dichotomous variable (e.g., 1 = missed for this reason, 0 = missed for a different reason). Participants more frequently reported missing voice diaries due to being busy (X² (1, N = 511) = 9.50, p = .002), not feeling like it (X² (1, N = 511) = 10.77, p = .001), and not having enough privacy to complete the diary (X² (1, N = 511) = 4.60, p = .032) compared to reasons they missed IWR diaries. In contrast, participants were more likely to miss internet diaries due to forgetting (X² (1, N = 511) = 5.27, p = .022) or having a technical issue/not having phone or internet access (X² (1, N = 511) = 17.58, p < .001) compared to voice diaries.

Ninety-two percent of participants who filled out internet and voice diaries reported a preference for internet diaries. In qualitative interviews, participants noted different facilitators and barriers for internet versus voice. In general, the internet modality is seen as taking less time and attention, allowing for multitasking, and being easier to fill out anywhere, with fewer issues regarding finding a quiet or private location. The voice modality is seen as more personal or interactive, with a slower pace engendering deeper reflection from participants. A participant summarizes, “I definitely like them both, just for different reasons: the voice ones, because I was able to talk and express how I was feeling, and it made it more personal hearing someone respond back; and then for the typing one, because I was able to do it wherever I needed to…I got my reminder at 10:00, like “Hey, do this,” and I was like, “Oh, yeah. Let me do this really quickly.”

Participants noted that completing the diary close to the time when the reminders were received was easier with internet diaries than voice diaries; this was supported by diary data showing that 27.1% of internet diaries were completed within a half hour of the reminders, compared to 21.6% of voice diaries (X² (1, N = 2,558) = 10.3, p = .01). A participant explains how not completing the voice diary at the time of the reminder due to being busy and having issues with noise and privacy can lead to further barriers such as forgetting, being too tired, or falling asleep: “Well, Internet is-it was just more convenient. It was a little faster. I could do that…while on the bus, but with the phone diary, it was harder because it was a lot of noise, and then when I would get home, shower, I will just – by the time I realized that I need to do it, I’d be too tired, sleepy, fall asleep.”

Participants left a greater number of open-ended messages using the internet diary compared to the voice diary. A total of 448 messages were left for study staff, including 147 (32.8%) voice messages and 301 (67.2%) internet messages. A greater number of messages than expected were found for the internet diary compared to the voice diary for the questions “Is there anything else you would like to mention or explain about any of your answers” (X² (1, N = 2495) = 19.656, p < .001) and “Did anything especially good happen today? If yes, what happened?” (X² (1, N = 2487) = 19.250, p < .001).

Participants were asked if they were honest when filling out their diaries. Three participants mentioned being less honest about sexual behaviors when they were using the voice diary; no participants mentioned being less honest when they were using the internet diary.

DISCUSSION

In summary, study retention (93.4%) and overall daily diary compliance (72.4%) rates for the 60-day diary period indicate that voice and internet daily diaries were feasible for use with our sample of HIV-infected young MSM. These findings were similar to previous studies of electronic daily diaries, which further supports our conclusion [12, 13, 14]. Our analysis allowed us to build upon the literature by identifying specific barriers and facilitators to diary completion.

A number of facilitators motivated participants to complete the diary more consistently. Although monetary compensation was an important factor in engaging participants, it was not the only facilitator noted by participants, and this form of compensation alone may not be enough to encourage perfect compliance. Compliance differed across modality, despite the fact that compensation was the same in each condition. Other facilitators noted by participants included the cathartic and self-reflective nature of the diaries, positive relationships with study staff, reminders, and the facilitators unique to each modality, such as the conversational quality of the voice diaries or the ease of multitasking while completing the internet diaries. Future work should make use of potential facilitators. For example, the present analysis suggests that compliance among certain participants may be increased by heightening their sense of benefiting from the diary’s self-reflective and cathartic qualities. This could be addressed when explaining the study to participants or during end-of-study procedures, such as by providing participants with feedback on their moods and behaviors over time.

Barriers to diary completion were often those that prevented the diaries from being filled out at the time the reminders were received, as well as issues related to being too busy, forgetting, and falling asleep. We propose that it is important to ensure participants can complete their dairies within close proximity to the reminder times, which may include increasing participants’ ability to quickly, privately, and discreetly fill out the diaries. In addition, a greater percentage of diaries were missed on the weekend, which is supported by previous research [15, 16]; this suggests that future studies should focus on improving weekend compliance, perhaps by providing additional monetary incentives or staff check-ins for weekend diaries.

Participants often developed barrier management strategies, such as setting addit ional reminders, as a way to overcome the perceived barriers for each diary method. Future daily diary studies should include action plans based on the barrier management strategies identified in Table I to improve compliance. For example, participants who cannot fill out the diary during work hours may be prompted to set a reminder to complete the diary during their commute home from work, and participants can be made aware of the higher potential for missing diaries during the weekends.

The majority of participants (92%) preferred the internet modality, and internet diaries were completed at a higher rate (77.5%) than voice diaries (67.4%). Our study provides evidence that internet diaries are more likely to be completed close to the time of the reminder texts, take less time to complete, and are often perceived as being more convenient and discreet than voice diaries. Furthermore, a greater number of open-ended messages were left with the internet diaries compared to the voice diaries, and three participants were less honest when filling out the voice diaries compared to internet diaries. As such, internet diaries may be the preferred method for collecting sensitive data with HIV-infected young MSM.

However, qualitative interviews provided evidence that there are benefits to both modalities, with voice diaries engendering deeper reflection and a more powerful feeling of catharsis among some participants. Consequently, voice dairies may be more useful in intervention studies where critical reflection and self-monitoring are key outcomes. Researchers who implement voice diaries may need to take greater efforts to ensure compliance; for example, it may be beneficial to reduce the time it takes to complete voice diaries and provide multiple reminders to complete the diary.

Furthermore, our comparison of internet and voice diaries suggests action plans to avoid low-compliance situations can be tailored to each diary modality. Our findings provide evidence that participants filling out internet diaries may need to rely on strategies to ensure internet access and avoid forgetting the diary, whereas participants filling out voice diaries may need to employ methods to improve motivation, set aside time to fill out the diary, and identify private locations in which to fill out the diary.

Overall, our clinic-based sample of young men receiving care may limit the generalizability of our findings. However, our diverse sample of young MSM is generally reflective of the population most impacted by the HIV epidemic, and our compliance and retention rates are similar to other studies employing electronic diaries to measure sexual behavior among MSM [11].

The applicability of our findings to non-research settings was limited in that participants received monetary compensation for completing their dairies and were contacted regularly by study staff. We propose that using electronic diaries was less resource intensive than using pencil-and-paper diaries would have been, as electronic diaries allowed for seamless consolidation of data across three sites into one central database, monitoring of data in real-time, and automatic data entry. This consolidation process limits the potential for human error during data entry, thereby improving the accuracy of data analysis and reducing the amount of personnel time spent entering and checking data. Filling out electronic diaries may also reduce participant burden, as participants do not need to carry around paper surveys or remember to save or return them to the clinic. However, we do not want to imply that using electronic diaries completely eliminates the need for human and monetary resources to improve compliance. Furthermore, we did not complete a financial cost-benefit analysis comparing the internet and voice modalities, but future researchers may need to take different costs into consideration; currently, free or low-cost internet-based platforms are more ubiquitous than voice-based platforms. Further studies are needed to examine daily diary compliance with different levels of support and monetary compensation.

One potential limitation of our study is that we excluded participants who did not have perfect completion of the diaries during the first three “calibration days.” As such, we may have selected for participants who were more likely to complete the daily diaries, although unlike other studies, we did not discontinue participants for low compliance once they had completed the calibration period. Thus, our study gives direction for improving compliance with individuals who have at least some capacity and desire to fill out daily diaries. Different methods may be necessary to implement daily diaries with participants who are at risk for extremely poor compliance. We should also note that the use of a calibration period is also useful in addressing potential reactivity that can result in commencing a diary study. In ongoing outcome analyses, we have observed initial elevation effects in participants’ reports of their mood. Following the first few days after starting each modality, we observed a decline in average scores for mood variables. We attribute these initial effects to participants adjusting to the diary modality. Thus employing the calibration period and excluding calibration days from analyses allowed us to address and adjust for reactive effects.

Future work should aim to explore compliance for longer than 66 days, as it was seen that diary compliance decreased over time. Furthermore, participants who started with internet as their first modality had higher internet completion rates and lower voice completion rates than participants who started with voice as their first modality. A study in which only internet or voice diaries are used may yield different compliance rates.

Additional investigation is necessary to determine if the less frequently, more slowly completed voice diaries provide more or less accurate data than the more frequently, more quickly completed internet diaries. Researchers deciding between voice and internet modalities may need to determine whether their studies will benefit from a greater number of dairies overall or from more carefully filled out responses. Voice diaries may be more advantageous for use with low-literacy individuals [5], and our study is limited in that all participants could read at 5^th grade level or above.

Also, further research is needed to examine the effect of feedback on future diary compliance and behavioral reactivity and to determine if participants show improvements in mood or behavior as a result of the cathartic nature of the diaries. If feedback from daily diaries positively impacts psychological and behavioral outcomes, this may have implications for the use of daily diaries in interventions.

Researchers who apply our findings by addressing barriers to diary completion may avoid the potential pitfalls of inefficiently implemented daily diaries, which may delay a study or result in a paucity of quality data. In addition, these results provide evidence that daily diary studies implemented via ubiquitous internet and phone technologies are feasible and acceptable for use with young MSM. As such, daily diaries may be of key importance to HIV/AIDS researchers as they delve into nuanced explorations of state-based constructs and examine repeated daily events such as medication adherence. It is our hope that future studies will take advantage of novel daily diary technology to better understand ways to improve the health and wellbeing of people living with HIV/AIDS.