Formative Evaluation of a Text Messaging Intervention to Promote Varenicline Adherence Among Tobacco-Dependent Persons with HIV

Abstract

Few published studies describe processes in the development of mobile health interventions. This study reports data from a formative evaluation of a text messaging intervention being developed to increase adherence to smoking cessation medication (varenicline) among tobacco-dependent persons with HIV/AIDS. Four focus groups were conducted (N = 29) using a mixed-methods approach to assess: (a) beliefs and preferences regarding the use of varenicline, (b) preferences for receiving tobacco-related texts, and(c) the acceptability of draft text messages. Themes that emerged from the focus groups were that (a) participants were cautious and wanted to discuss varenicline carefully with health care providers, (b) participants preferred simple messages that were positive and encouraging, (c) messages should emphasize tobacco cessation and not varenicline adherence, and (d) texts would serve as a reminder about goals and foster support and connectedness with the health care team. Overall, 47 out of the 100 messages received a grade of C or less (rated on a 5-point grade scale: A, B, C, D, or F), the majority of which focused on medication adherence. All participants reported that they were likely to read the messages. The majority (64%) indicated that they preferred receiving 2 or more messages per day. Gathering systematic participant feedback provides critical input in intervention planning.

Smoking rates among people living with HIV/AIDS (PLWHA) are 2 to 3 times that of the general population. HIV/AIDS and tobacco use share multiple common risk factors, which likely accounts for this strong relationship (Burkhalter, Springer, Chhabra, Ostroff, & Rapkin, 2005; Gritz, Vidrine, Lazev, Amick, & Arduino, 2004). The tobacco-related illnesses of cardiovascular disease and lung cancer are, respectively, the second and third leading causes of non-HIV/AIDS-related deaths among PLWHA (Sackoff, Hanna, Pfeiffer, & Torian, 2006). In addition, cigarette smoking places PLWHA at increased risk for serious HIV-related comorbidities and premature death compared with HIV-positive nonsmokers (Reynolds, 2009). Despite the burden of tobacco use, few studies have evaluated the delivery of smoking cessation interventions for this population (Harris, 2010). Intervening to improve rates of successful smoking cessation has the potential to significantly improve the health and long-term survival of HIV-positive smokers.

A few studies have indicated that PLWHA are interested in quitting and can achieve abstinence, particularly when pharmacotherapy is used as recommended (Lloyd-Richardson et al., 2009; Vidrine, Arduino, Lazev, & Gritz, 2006). In the general population, higher rates of adherence are strongly associated with an increased likelihood of smoking cessation (Hays, Leischow, Lawrence, & Lee, 2010). However, similar to findings in the general population, use of smoking cessation medication among PLWHA is low (Ingersoll, Cropsey, & Heckman, 2009). Several barriers to cessation medication use specific to PLWHA have been described, including an already complex medication regimen and negative beliefs about smoking cessation medications (Burkhalter et al., 2005).

Given the pervasiveness, low cost, and convenience of the technology, text messaging may be particularly well suited for supporting health behavior change. Text messaging can facilitate more communication with patients and offers the opportunity to deliver health-related messages at exactly the times and places where these messages can have the greatest impact, such as medication reminders consistent with a patient’s dosing schedule. Two recent pilot studies have demonstrated significant increases in HIV-related medication adherence using daily text message reminders (Hardy et al., 2011; Lewis et al., 2013). Despite the promise of this method, no studies to date have specifically addressed adherence to smoking cessation medication in this population (Rodgers et al., 2005; Whittaker, McRobbie, et al., 2012).

The current study presents formative evaluation data from the first phase of a planned randomized controlled trial comparing text messages alone to text messages plus six sessions of telephone-delivered counseling. Consistent with recommendations for developing mHealth interventions (U.S. Department of Health and Human Services Text4Health Task Force, 2012; Whittaker, Merry, Dorey, & Maddison, 2012), we first conducted focus groups with the target population to determine the feasibility and acceptability of the text messaging intervention concept and gather feedback on a series of draft text messages. This article reports mixed-methods results from the formative evaluation phase.

Methods

Participants and Recruitment

This study was conducted at a large comprehensive urban HIV care clinic in collaboration with an academic medical center. The overall clinic population as of 2013 was 66% male, 47% Black, 13% White/not Hispanic, 35% Hispanic/Latin, 1% Asian, and 4% more than one race/race unknown. The modal age was 30–49 years old (58%), with 26% ages 50+. In terms of comorbidities, 60% had a psychiatric diagnosis, 56% used tobacco, 60% had current or past other substance use, and 38% had hepatitis or liver disease. The majority of the population had Medicaid for insurance (68%), 13% had Medicare, 13% had the AIDS Drug Assistance Program from New York State, 5% had private insurance, and 2% were uninsured. Participants were recruited by one of two means: They responded to flyers posted in the clinic waiting rooms or were approached in the waiting room by the study research assistant. In each case they completed a screener for current tobacco use. Those who agreed to participate and met inclusion criteria (tobacco use in the past 30 days and a diagnosis of HIV/AIDS) were scheduled for one of four focus groups. To decrease barriers and misgivings related to research participation among this population, we did not collect personal information and used a verbal consent procedure. The study was approved by an institutional review board.

Message Prototyping

The planned intervention was based on the information–motivation–behavioral skills (IMB) model of antiretroviral adherence (Fisher, Amico, Fisher, & Harman, 2008). This model incorporates factors from social cognitive theory and the theory of planned behavior that are associated with medication adherence and account for a large proportion of the variance in any given deliberate behavior (Bandura, 1989; Fishbein, 2008; Johnson et al., 2007). The behavioral intervention components were meant to address factors hypothesized by the IMB model to influence the primary study outcomes of varenicline (Chantix^®) adherence and smoking cessation (Marlatt, 2005; Osterberg & Blaschke, 2005).

The development of the text messaging intervention was based on findings indicating the efficacy of text messaging for tobacco cessation (Free et al., 2009, 2013) as well as findings from other studies that “simply forgetting” is the most common self-reported reason for nonadherence (Osterberg & Blaschke, 2005). Therefore, the texting protocol was meant to address both cessation medication adherence and tobacco cessation themes. Given that varenicline is prescribed for 12 weeks and that our previous work had shown that participants would likely prefer receiving two texts per day (Lloyd-Richardson et al., 2009; Opoku-Nti, Burkhalter, Krebs, & Ostroff, 2015), the intervention required a series of 168 text messages. As it was not practical to gather feedback on all messages, we chose 100 messages for focus group review, a number we found feasible in previous work. Messages created by our study team were based on IMB constructs, including motivation, social support, and expectancies, and included basic medication reminders. We also drew from the National Cancer Institute’s QuitNowTXT library (U.S. Department of Health and Human Services, 2013) and from our collaborator’s HIV medication adherence study (Lewis et al., 2013).

Focus Groups and Evaluation Plan

Four 2-hour focus groups with five to nine HIV-positive smokers each were conducted, a sample size sufficient to achieve data saturation in qualitative analysis (Krueger & Casey, 2008). The main aims of the focus groups were to(a) assess beliefs, experiences, and preferences regarding the use of varenicline; (b) assess the acceptability of options of receiving text messages during the study; and (c) obtain feedback on the text message library to determine which messages would be most acceptable and useful.

A mixed-methods approach was used whereby participants were given paper surveys on which they rated 100 messages using a letter grade scale from A to F. They also responded to a series of open- and closed-ended Likert-type questions regarding message acceptability and study feasibility, a process previously used to develop messages for HIV medication adherence (Furberg & Willoughby, 2013; Lewis et al., 2013). This was followed by a 60-minute structured focus group session that began by exploring participants’ experiences with and barriers to varenicline use. During this time research assistants scored the messages. After the structured questions for the varenicline discussion were completed, we gathered open-ended feedback on messages that participants rated a C or lower and explored their preferences for message types and content. Focus groups were audio-recorded and transcribed verbatim, and two research assistants took notes during group discussions. Following qualitative analysis procedures (Silverman, 2006), two independent raters coded transcripts and conducted consensus meetings to determine emergent themes.

Results

Evaluation of Draft Text Messages and the Feasibility of Texting

A total of 29 clinic patients (7 women and 22 men) participated in the focus groups. Survey responses to evaluations of the draft text messages are presented in Table 1. The average message rating was 4.0/5 (a B on the letter scale). Almost half (48.1%) of participants reported that the messages were written in too serious of a tone. A total of 47 messages obtained an average score of less than a B. The majority (88.5%) of participants reported that the messages were not offensive. In terms of feasibility of the texting intervention, the majority of participants (85.2%) responded they would be very or moderately likely to read messages sent to them. Almost all participants had their own phones (96.0%). Most (84%) had experience with texting, but a large portion (79.2%) did not have a texting plan (i.e., they paid per text). The modal number of texts participants reported wanting daily from the study was two (46.4%), followed by one (25%; see Table 2).

Table 1.

Feedback regarding draft messages

Question	Response	n	%
How likely are you to read these messages?	Very likely	13	48.2
	Moderately likely	10	37.0
	Somewhat/mildly likely	4	14.8
	Not all likely	0	0.0
The messages were offensive.	Strongly agree	2	7.7
	Moderately agree	1	3.8
	Somewhat disagree	9	34.6
	Strongly disagree	14	53.9
The messages were too serious.	Strongly agree	4	14.9
	Moderately agree	9	33.3
	Somewhat disagree	5	18.5
	Strongly disagree	9	33.3
The messages were designed for me or other people like me.	Strongly agree	12	44.4
	Moderately agree	9	33.3
	Somewhat disagree	4	14.9
	Strongly disagree	2	7.4

Table 2.

Cell phone use and texting preferences

Question	Response	n	%
On average how many messages do you send each day?	0	8	33.3
	1–10	8	33.3
	11–19	1	4.2
	20+(M=22.0, SD=33.3)	7	29.2
On average how many messages do you receive each day?	0	5	20.8
	1–10	9	37.5
	11–19	1	4.2
	20+(M=25.5, SD=36.4)	9	37.5
How many texts do you prefer to receive each day?	0	3	15.8
	1–10	11	57.9
	11–19	1	5.3
	20+(M=17.7, SD=31.2)	4	21.0
Do you have a cell phone?	Yes	24	96.0
	No	1	4.0
Prepaid or on a monthly plan?	Monthly	17	70.8
	Prepaid	7	29.2
Have you used it for text messaging?	At least once	21	84.0
	Never	4	16.0
How many messages each day would you want to receive from the clinic?	None	3	10.7
	1	7	25.0
	2	13	46.4
	3 or more	5	17.8

Qualitative Analysis

Qualitative analysis of focus group sessions identified four themes:

1. Participants were cautious and wanted to discuss varenicline carefully with health care providers. Statements concerning side effects were common across the groups, with participants either mentioning firsthand accounts or relating stories from friends concerning experiences with vivid dreams, personality changes, and interactions with current medications. Participants did not like texts with information about varenicline or its side effects, as they felt these were “not relevant.” They stated that they would have already “done our homework beforehand” with their providers prior to starting the medication.

2. Participants wanted simple messages that were positive and encouraging. In general, participants rated longer messages poorly and found them confusing. They expressed a preference for messages that were positive and focused on benefits of quitting, such as “Stay on track” or “Look how much money you saved this year.”

3. Messages should emphasize tobacco cessation and not varenicline adherence. Participants gave poor ratings to detailed messages about medication taking, with statements such as “Talking about the pill sounds like your mother” and “Don’t stuff it down our throat with every word being Chantix.” This seems to reflect a strong sense of free choice regarding taking medication; they would take it if they wanted but reacted strongly to messages that seemed “patronizing” or told them what they should be doing. They expressed a preference for messages that “talk about the problem, not the cure”—that is, messages that focused on tobacco use and cessation rather than cessation medications. Participants noted that they were already skilled at maintaining a regular medication regimen, and adding an additional prescription would not pose a new organizational challenge. Some medication reminders would be welcome but should be “straight and to the point,” such as “Have you taken your Chantix today?”

4. Texts would serve as a reminder about goals and foster support and connectedness with the health care team. Participants highlighted perceived benefits of texting, such that it would be useful “to keep you on track, just like if your doctor called you out of the blue. I love that.” They also felt that the study should frame the texts as originating with the clinic rather than an independent system or one affiliated with a drug company: “Coming from here would mean, you know, more like you care. Somebody cares about me.” Some of the draft texts mentioned Chantix by name, but participants felt that this sounded as if the texts were “pushing meds” “like a commercial.” Participants also wanted to have a clinic number to call if they were experiencing side effects or had questions.

Discussion

The formative evaluation data indicate a number of important points useful for developing mHealth tobacco cessation programs for PLWHA. Most important, it appears that PLWHA would welcome a text messaging program that provides support for tobacco cessation. Participants in the focus groups were enthusiastic about the possibilities for texting to assist in meeting their cessation goals. Almost all participants indicated that they would read the messages and that they were written for people like themselves. The formative results provided further data to inform implementation, indicating that two messages per day would be the preferred number to send and that texting would be highly feasible for this population, as most clinic patients have phones and are open to receiving messages.

Participant feedback was also helpful for targeting the planned intervention to the needs and preferences of this population. It was particularly surprising that participants gave poor ratings to many of the adherence-focused messages. Messages were written based on IMB constructs that contribute to low adherence, such as concern regarding side effects, information seeking, and general forgetting. Participants responded that these factors were important in their decision making but were not helpful to include in text messages. They noted that, as experienced medication users, they would have addressed these questions with their providers before starting varenicline. Given personal experiences that may have involved judgment or stigma, HIV-positive smokers may be particularly sensitive to messages that could be seen as paternalistic. Also helpful was feedback indicating that many of the messages were too long, complex, or serious. It is interesting that Gold, Lim, Hellard, Hocking, and Keogh (2010) reported similar focus group feedback in their study of texting for sexual health, in which participants wanted positive, brief, and humorous messages. Many texts were borrowed from the National Cancer Institute library and other programs we were developing, such as an employee health program. One drawback of using existing text libraries is that formative work has not been published regarding the acceptability of these messages. Our data indicate that texting libraries may not generalize well across contexts. For instance, messages suitable for highly educated employees may not be equally welcome among an urban public health clinic population.

Intervention Changes Based on Focus Group Feedback

Using data gathered on specific messages as well as overall focus group feedback, we revised the texting library prior to commencing the next pilot phase in which we sent messages to a small sample of 10 participants. This involved replacing all messages with a rating lower than a B with new versions and modifying the medication adherence messages to focus on simple reminders. After reviewing the data, members of our research team each devised new messages and met to rate them as a group prior to their inclusion. For instance, instructive messages such as “Place the Chantix bottle near your other morning meds so you take them all together” were replaced with straightforward reminders such as “Good morning, just a reminder to take your quitting meds.”

Future Directions

Our findings highlight the value of conducting formative work prior to launching an intervention. The mHealth evaluation framework as outlined by Whittaker, Merry, and colleagues (2012) provides guidance regarding steps for developing technology-based interventions. However, technology-based interventions provide a challenge to typical information-gathering protocols in social science. Our evaluation protocol had participants rate texts that were printed on paper in table format. The modality of presentation may have influenced the perception of the message content. We discovered that longer messages appear less lengthy on a phone, perhaps leading to some of the feedback we received. Another potential limitation of our approach is that questions asked people about their preferences, which may not necessarily correlate with efficacy. This would be of more concern if questions were asking about what would help them quit smoking (concept development) rather than having them review messages that they would be receiving from an already-developed intervention (concept testing). We would suggest creating a system whereby focus group participants could be given phones and the system programmed to send them messages one at a time with the opportunity to respond with their ratings directly, which our technical consultants agree is possible and would also facilitate data collection.

Our findings also indicate that although the IMB model may be useful for conceptualizing nonadherence from a theoretical perspective, direct use of patient messaging based on the model may present a challenge in terms of participant acceptability. Poor ratings for IMB-based messages suggest two possibilities. First, constructs of the IMB model (Fisher, Fisher, Amico, & Harman, 2006) may better be addressed in a live interaction rather than via a text message, as texts may be more likely to be interpreted as instructional rather than as part of a collaborative problem-solving process. Second, it is also possible that the constructs proposed by the model may not apply to varenicline adherence; patients may find the behavior less complex and that such messaging is overkill. In a pilot study Konkle-Parker, Erlen, Dubbert, and May (2012) found a small positive effect for IMB-based Highly Active Antiretroviral Therapy (HAART)–adherence counseling but did not collect acceptability data. Thus, knowledge is limited regarding participant responses to IMB-based interventions. Such data emphasize the need and importance of collecting formative qualitative data during intervention development.

In sum, few interventions have been deployed to assist PLWHA with tobacco cessation (Harris, 2010), and those have achieved relatively low abstinence rates (10%–13%; Lloyd-Richardson et al., 2009; Vidrine, Marks, Arduino, & Gritz, 2012). However, the authors of one study (Lloyd-Richardson et al., 2009) did conclude that brief contacts focused on cessation medication use may be beneficial and more cost effective than prolonged counseling. Thus, potential exists for text messaging to provide a low-intensity yet effective intervention.

The formative work we conducted indicates that text messaging may be a useful, low-cost strategy for delivering cessation support and medication reminders but that such interventions likely benefit from the use of participatory research to properly target message content and timing.