Development of a multi-behavioral mHealth app for women smokers

Abstract

Objective

This paper describes the development of the See Me Smoke-Free™ (SMSF) mobile health application, which uses guided imagery to support women in smoking cessation, eating a healthy diet, and increasing physical activity.

Materials and Methods

Focus group discussions, with member checks, were conducted to refine the intervention content and app user interface. Data related to the context of app deployment were collected via user testing sessions and internal quality control testing, which identified and addressed functionality issues, content problems, and bugs.

Results

Interactive app features include playback of guided imagery audio files, notification pop-ups, award-sharing on social media, a tracking calendar, content resources, and direct call to the local tobacco quitline. Focus groups helped design the user interface, and identified several themes for incorporation into app content, including positivity, the rewards of smoking cessation, and the integrated benefits maintaining a healthy lifestyle. User testing improved app functionality and usability on many Android phone models.

Discussion

Changes to the app content and function were made iteratively by the development team as a result of focus group and user testing. Despite extensive internal and user testing, unanticipated data collection and reporting issues emerged during deployment due to the variety of Android software and hardware, but also due to individual phone settings and use.

Conclusion

Focus group interviews helped thematically frame the intervention and maximize user engagement. Testing caught many bugs and usability concerns, but missed some of the data transmission problems encountered during deployment.

Objective

A multi-disciplinary research team developed and tested the See Me Smoke-Free™ (SMSF) mobile health application (mHealth app) for women smokers. SMSF uses guided imagery to support women in smoking cessation, eating a healthy diet, and increasing physical activity. Guided imagery is a mind-body therapy using controlled visualization of specific mental images (Kabat-Zinn, 1982). Herein, we describe the development of the SMSF app, a multi-step process of content development, formative assessment, programming, testing, and deployment to the Google Play Store. In order to improve evidence-based practice (Short, James, & Plotnikoff 2013), we used process evaluation to document and evaluate the development and deployment of this mHealth intervention. Feasibility testing findings are reported elsewhere.

Background and Significance

Lung cancer is the most common cause of cancer death in women (American Cancer Society, 2015), causing as many deaths as breast and gynecological cancers combined (Patel, Bach, & Kris, 2004). Despite smoking being the most significant risk factor for lung cancer, 15.3% of American women smoke (Jamal et al., 2014). mHealth apps present opportunities for reaching diverse groups to promote health and assist women in quitting smoking. Of the 64% of U.S. adults who own a smartphone, 62% have used their phone to look up health information (Smith, 2015). Smartphone users, particularly women, are interested in using apps for tracking or managing health, with exercise, diet, and weight apps being the most popular (Fox & Duggan, 2012). Moreover, nearly one fifth of smartphone users may rely on their phone for internet access (Smith, 2015). Therefore, apps have the potential to reach diverse smokers with innovative tobacco cessation interventions.

Targeting an intervention to address multiple behaviors and tailoring it to individual needs

For a variety of reasons, women may face particular challenges when quitting smoking (Addicott, Gray, & Todd, 2009; Allen, Hatsukami, Christianson, & Brown, 2000; Brandon & Baker, 1991; Orleans, 1994; Perkins et al., 2001; Pomerleau, Zucker, & Stewart, 2001; USDHHS, 1990) and they may be more prone to relapse (Brandon & Baker, 1991). Women may experience physiological impediments due to nicotine sensitivity during the menstrual cycle (Allen, Hatsukami, Christianson, & Brown, 2000) or psychosocial difficulties due to concern about weight gain upon cessation (Pomerleau & Saules, 2007). Moreover, evidence suggests medical interventions, such as nicotine replacement therapy, may not be as effective in helping women quit (Allen, et al., 2000; Cepeda-Benito, Reynoso, & Erath, 2004; Killen, Fortmann, Newman, & Varady, 1990; Perkins, 1996). Although concern about weight gain may not be the primary deterrent in a woman’s decision to quit smoking (Nichter, Vuckovic, Tesler, Adrian, & Ritenbaugh, 2004), evidence suggests it is a barrier for many women (Perkins, et al., 2001; Pomerleau, et al., 2001). The average weight gain of 8–10 pounds after quitting may contribute to relapse (Pisinger & Jorgensen, 2007). Research indicates that weight-concerned women are more likely to be heavy smokers (Nichter, et al., 2004; Pomerleau, et al., 2001) and preoccupied with looking thin (Pomerleau, et al., 2001). Studies suggest that attaining a more realistic body image will help these smokers quit (Clark et al., 2006; Pomerleau & Saules, 2007).

Growing evidence suggests that many behaviors, such as smoking and drinking alcohol, (Weitzman & Chen, 2005) may cluster (de Vries et al., 2008) and reinforce each other (Nichter, Nichter, Carkoglu, Lloyd-Richardson, & Tobacco Etiology Research Network, 2010). Changing multiple behaviors simultaneously may result in increased confidence in the capacity for change (Prochaska, Spring, & Nigg, 2008) and addressing multiple behaviors in weight-concerned women smokers may lead to more successful smoking cessation (Farley, Hajek, Lycett, & Aveyard, 2012; Hyman, Pavlik, Taylor, Goodrick, & Moye, 2007; Spring et al., 2009; Spring et al., 2012). SMSF uses guided imagery to focus the mind on visualizations that incorporate all of the body’s senses and direct the user toward smoking cessation, healthy eating, and increased physical activity in order to address the weight gain that can result from quitting cigarettes. Following the success of interventions tailored to demographic traits (e.g. gender) using theoretical concepts (e.g. self-efficacy) (Noar, Benac, & Harris, 2007), the team developed an app that provides individualized feedback based on smoking status. Users receive push messages based on their quit date, commending them on their decision to quit and offering tips for managing urges to smoke. If users report a “slip,” the app offers feedback and extra support.

In developing SMSF, the team was attentive to the need for targeting its content to women while tailoring the intervention to individual needs. Researchers have acknowledged the relationship between smoking and identity in tobacco marketing to women (Amos, Greaves, Nichter & Bloch, 2012), and in the use of cigarettes to promote a desired image of oneself (Hoek, Maubach, Stevenson, et al., 2013; Nichter et al., 2006; Nichter et al., 2007; Nichter et al., 2009). Using qualitative methods to identify women’s concerns about and desires for their health, the team revised initial program content developed by the co-investigators who are experts in smoking cessation, nutrition, physical activity, and app development. Investigators were guided by the theoretical framework described below, and evidence-based guidelines for behavior change interventions (Fiore, et al., 2008; Lancaster and Stead, 2005; Michie et al., 2005; Michie 2008; Strecher, 1999). The intervention was refined through an iterative process of focus group discussions and “member checks” (Thompson, Cullen, Boushey & Konzelmann, 2012) with self-identified women smokers.

Theoretical Framework

Investigators constructed the intervention using a theoretical framework that considers the interrelationships between multiple psychosocial mediators of smoking cessation, including body image, concerns about weight gain, and self-efficacy (Figure 1). The imagery audio file components of the intervention were based on previous research showing that mental imagery and cognitive behavioral therapy (CBT) have been used separately to address eating disorders and concerns about weight gain (Cooper, 2011; Hofmann, Asnaani, Vonk, Sawyer, & Fang, 2012; Mountford & Waller, 2006; Tatham, 2011; Vocks et al., 2009). The team predicted that cognitive behavioral therapy-framed guided imagery would change women smokers’ concerns about body weight and image, thus leading to increased tobacco cessation behaviors.

Figure 1.

Theoretical framework

Guided imagery provides opportunities to improve self-efficacy in a range of behaviors and health outcomes (Kim, Newton, Sachs, Glutting, & Glanz, 2012; Maddison et al., 2012; Menzies, Taylor, & Bourguignon, 2006; Ramsey, Cumming, & Edwards, 2008). The team used a dual function framework to develop imagery content. Supported with psychometric analyses (Giacobbi, Tuccitto, Buman, & Munroe-Chandler, 2010), the dual function imagery framework allowed the team to tailor imagery scripts to cultivate cognitive skill building (e.g. managing cravings, establishing routines) and motivation as users achieved greater confidence from the development of these skills. In addition, the team drew from the work of colleagues in the promotion of behavior change (Bandura & Wood, 1989; Perkins, et al., 2001; Prochaska, DiClemente, & Norcross, 1992; Schnoll et al., 2005; Williamson, Perrin, Blouin, & Barbin, 2000). Project investigators developed initial program content aimed at increasing engagement with the app and with the desired behavior change. Content includes imagery scripts, a system that tracks cigarette cravings and money saved from quitting, motivational messages, tips and techniques, and resources centered on smoking cessation, improving diet, and increasing physical activity.

Materials and Methods

Developing the Content

SMSF app content and technology were developed in a step-wise fashion, incorporating feedback from members of the app’s target audience throughout the process (Figure 2). After the guided imagery content, intervention components, and app design were drafted, these were presented to groups of women smokers for feedback. After a prototype was built, the team enlisted participants in testing the system and their input was incorporated into the final version of the app. Participants in all phases of app development were recruited using a variety of methods, including earned media coverage, postings on local websites and social media (e.g. Craigslist, Facebook), and flyers posted in a variety of community locations, such as health clinics, libraries, and laundromats within several miles of the university where the study took place.

Figure 2.

SMSF Development Process

Imagery scripts

The team developed five scripts: one that introduced the concept of guided imagery, three that targeted specific behavior change, and another that addressed general wellness, positive body image, and self-efficacy. The behavior change scripts were designed to increase smoking cessation, physical activity, and fruit and vegetable consumption, and to promote well-being. The scripts were developed with the following goals in mind: 1) increase confidence in ability to quit smoking and belief in capacity for behavior change; 2) minimize concerns about weight gain to increase smoking abstinence; 3) improve body image by reducing dissatisfaction and internalization of thin body ideals; and 4) reduce urges to smoke, eat unhealthy food, or remain sedentary (Table 1). The scripts were audio-recorded for presentation to participants during focus groups and member checks.

Table 1.

Smoking Cessation Script components according to theoretical framework.

Body Image (B)	Affect (A)*	Self-Efficacy (SE)	Cravings (C)*
As you think about being smoke-free, you can vividly see clean air flowing through your body. You see your lungs becoming cleaner, your heart pumping stronger, and feel your whole body energized.	It’s normal to feel stressed or irritable when you’re not smoking. See yourself coping with these feelings by doing healthy things that you enjoy, like taking a walk or listening to music.	Visualize yourself actively quitting smoking. See yourself reviewing your plan for quitting. Think about your goals, and reflect upon your reasons for being smoke-free.	You can expect and anticipate strong cigarette cravings. Know that they won’t last forever. See yourself having an urge to smoke, and accept it, knowing that it will go away.
Imagine the blood flowing to your skin. Your complexion becomes radiant, and wrinkles reduce. See yourself looking younger and healthier when you quit smoking.	You are happy and proud of yourself knowing that you are smoke-free. Feel the excitement of knowing that you can maintain these feelings.	Create in your mind’s eye a clear image of you without a cigarette. You are smoke-free. Practice seeing this image in your mind.	See yourself riding the urge to smoke like riding a wave on a surfboard. You catch the wave and ride it to shore. When you reach the shore, the craving is gone.
It’s normal to gain a few pounds when you quit smoking. Realize that the benefits offset the extra weight. See yourself as powerful, healthy, and beautiful.	Imagine others seeing you as a strong, smoke-free woman. Experience the pleasure of knowing that you feel good on the inside, and look good on the outside.	Create a clear and vivid image of being in a difficult situation without smoking. Imagine yourself in this situation and what you will be doing instead of smoking.	Create a feeling in your body of relief from cigarette cravings. Imagine the urges fading over time, until you barely notice them.

Other intervention components

In addition to the guided imagery audio files, content included various app components to maintain user engagement and offer support. These included resources for tobacco cessation, improving diet quality, and increasing physical activity. We also created a tracking system and awards for goal setting and achievement, daily motivational messages, and messages designed to provide tips for successful behavior change.

App user interface

The selection and design of app elements were informed by evidence-based guidelines for behavior change (Fiore et al., 2008; Lancaster and Stead, 2005; Michie et al., 2005; Michie 2008; Strecher, 1999), co-investigators’ experience creating eHealth interventions (Gordon, Mahabee-Gittens, Andrews, Christiansen & Byron, 2013; Gordon, Severson, Seeley & Christiansen, 2004; Severson, Gordon, Danaher & Akers, 2004; Hingle et al, 2014), and a graphic designer’s recommendations. App components then were presented to women who participated in group sessions.

Focus Groups

Four, 1.5-hour focus groups, including member checks (a qualitative research technique in which interpretation of participant feedback is given to members of the sample to confirm its accuracy), comprised members of the intervention’s target audience: women smokers who are interested in quitting, but have some concern about weight gain. Participants were English speakers and had a history of smoking for a year or more (Table 2). Member checks enabled the research team to ensure the data were being interpreted and applied correctly throughout the iterative development process (Thompson, Cullen, Boushey, and Konzelmann, 2012). We conducted two phases of focus groups, with four groups of 2–5 people each, depending on member availability. Smaller focus group sizes are beneficial when conducting in-depth discussions about sensitive issues, such as cigarette smoking and body image (Bernard 2012). Because the focus groups included member checks, several women participated in more than one group. When we found ourselves with one attendee at a planned focus group, we conducted a one-on-one interview instead.

Table 2.

Focus Group & User Tester Demographics (n=15)

	# of participants
Age
18–29	2
30–39	5
40–49	4
50–59	3
60 +	1
Race
Asian	0
African American	0
Latino	1
Native American/Alaska Native	1
Native Hawaiian/Pacific Islander	0
White	13
Other	0
Education
Less than high school	1
High school graduate or equivalent	0
Some college	8
College graduate	3
Post-college	3

Participants in the first two focus groups listened to the audio files and reviewed preliminary graphics, including logo, for the app, and offered their feedback (Figure 2). Then, the imagery files were refined based on participant feedback, and the prototype graphic design modified for presentation to the remaining groups. Focus groups used structured and semi-structured data collection in which participants completed a brief survey and answered open-ended questions about the intervention. These approaches enabled us to examine the intervention’s a priori content in light of themes that emerged in the focus groups. Participants listened to audio recordings of the imagery scripts, and completed an individual questionnaire regarding their perceptions of the script. Five-point Likert scale questions prompted participants to rate the following: 1) their ability to clearly picture what the script asks them to imagine; 2) how much they like the script; 3) how well the script flows, or “fits together.” A fourth question was tailored to the script content and elicited participants’ thoughts about the script’s effectiveness in addressing concerns they had about guided imagery, quitting smoking, eating more fruits and vegetables, or getting more exercise. Open-ended questions invited participants to share what they liked most and least about the scripts. Once questionnaires were completed, the moderator led a group discussion about the scripts. Individuals in subsequent focus groups listened to audio recordings of revised scripts and were asked questions that would help refine the scripts in light of previous feedback. For example, if previous participants flagged certain vocabulary as preferable (e.g. “active” versus “physically fit”), interviewers asked later focus groups about their preference. These latter groups also reviewed mock-ups of the app and were asked for their feedback about app features and functionality. The last two focus groups and the individual interview were also opportunities for the team to check our findings and resulting content revisions with members of previous groups or with other members of the target population. We conducted focus groups until no new themes emerged in our data (data saturation) and the member-checks confirmed that our content accurately reflected their previous feedback.

Focus groups were audio-recorded and cross-referenced with completed surveys and notes taken by trained staff. The summative data were analyzed for common themes by two members of the project team who have expertise in qualitative data analysis (Ryan & Bernard, 2003). Using an inductive approach to analysis (Thomas 2006), the data summations were systematically reviewed by JA and a research assistant (RA) to identify emerging themes in the data. After the themes were identified and JA and RA achieved consensus about thematic definitions, the documents were reviewed again for additional themes, and an eye toward variation. Detailed qualitative reports then were presented to the entire research team for discussion. Changes to the program content and app functionality were prioritized based on identified themes and participant suggestions for improvement.

Rapid Prototypes and Internal Quality Control Testing

The app was built and refined iteratively with ongoing input from focus group participants and internal research staff. Using an incremental approach, development was divided into design, implementation, and testing (Figure 2). During quality control (QC) testing, functionality issues, content problems, and bugs were identified by the team. Upon identifying issues, the process returned to the design phase, when bugs and concerns were discussed by the research team, alternative designs developed, and new or revised elements were implemented.

Research project and departmental IT staff conducted internal QC testing on study phones or their personal phones, comprising several different hardware types running multiple versions of Android operating systems. All phones ran Android operating system version 4.0 or above. After a review of the most popular models of Android pre-paid phones, members of the research team tested the app on eight identified popular phone models. Testers set up the app as real users and interacted with the app over a 90-day period, which was the duration of the feasibility study.

User Testing and Deployment

User testing

Six Android users (Table 2) who self-identified as smokers and were interested in quitting participated in 1.5-hour individual user test. User testing is intended to assess the ability of users to complete tasks within the app that result in intended outcomes, the ease with which they complete these tasks, and their subjective reactions to the app (Brook, 1996). See Figure 2 for a diagram illustrating the iterative user testing process.

Participants were sent a link to the testing version of the app for download to their personal phones before the meeting. This version included 10 “use cases” that would model the use of the app over time so that users might test and offer feedback on all app features. Testers were asked to perform three to four use cases; all testers completed use case #1 (see Table 3). User tests were conducted by two trained staff, with one in charge of interacting with the participant and the other responsible for taking detailed notes. After walking through each use case and sharing their thoughts about the app’s design and functionality, participants were asked to complete the Brooke’s System Usability Scale (Brook, 1996) and a consumer satisfaction survey used in previous research by the Principal Investigator (JG) (Gordon & Mahabee-Gittens, 2011; Severson, Gordon, Danaher, & Akers, 2008). Following each tester’s feedback, revisions were made to the prototype, and the app was tested with additional users until no new concerns arose.

Table 3.

Use cases for User Testers

Use case	Tester Actions
#1: New app user	View landing page; set-up account; take baseline survey; set-up profile; set quit date; set audio file reminder time; view “quit date set” award; change profile settings; view resources and awards tabs
#2: User completed one day of app use	Track goals for the day; review message in response to user input
#3: User completed one week of app use	Review eight motivational messages; listen to “Get Started” and “Be Smoke-Free” imagery audio files; receive awards for imagery progress and one-week smoke-free
#4: User completed two weeks of app use	Listen to the “Eat Well” imagery audio file; receive an award for imagery progress (week 2)
#5: User completed three weeks of app use	Listen to the “Get Moving” imagery audio file; and receive an award for imagery progress (week 3)
#6: User completed four weeks of app use	Listen to the “Feel Fantastic” imagery audio file; receive an award for imagery progress (week 4) and for 30-days smoke-free; respond to the research study’s 30-day survey; review the daily diary, which had been populated with a month’s goal tracking data; view landing page and graph with daily tracking data
#7: User completed three months of app use	Record your own imagery audio file; respond to the research study’s 90-day survey; receive an award for 90-days smoke-free.
#8: User completed six months of app use	Listen to audio file of your choice; receive six-months smoke-free award
#9: User completed one year of app use	Receive one-year smoke-free award
#10: motivational messages (for internal quality control)	Test all mandatory and optional motivational messages in sequence

Deployment

We created a project web site that described the content and purpose of the SMSF app, and contained a link to the Google Play Store where visitors could download the app. The team’s Android developers uploaded the app to the Play Store through a web platform that collected the software code and information about the app, such as the name, description, and target audience. The Google Play Store platform tracks app use statistics, including the number of downloads and the number of installs by device type and country. Once deployed, SMSF was ready for the feasibility trial, the results of which are reported in a separate manuscript.

Results

Content Refinement: Results of Focus Group Testing

In the first two focus groups, research staff elicited detailed feedback regarding guided imagery audio files using a questionnaire and discussion. The audio files included: “Get Started,” describing guided imagery and how to use it; “Be Smoke-Free,” encouraging tobacco cessation; “Eat Well,” promoting a healthy diet; “Get Moving,” supporting increased physical activity; and “Feel Fantastic,” encouraging positive body image, affect, and self-efficacy. Participants indicated they were able to clearly picture the images described in the audio files, with some recordings needing improvement. Participants suggested improvements for addressing barriers to changing dietary practices and increasing physical activity. In focus group discussions, participants concluded that lengthier audio files would allow more time to evoke the described images. Some participants also suggested adding background music to recordings, which others agreed would be helpful in promoting relaxation. In addition, focus group findings identified several themes that were incorporated into audio file revisions.

Theme #1

Bring attention to the positive. Focus group participants liked the positivity of the audio files, which they felt counteracted the stigma they feel as smokers. Members flagged language that might be perceived as negative or judgmental by listeners (e.g., to get moving when one is “feeling lazy”), stating: “People already see smokers as lazy people;” “They think all we do is sit around and smoke;” and “When you’re calling us lazy, you’re just confirming the fact that society sees us that way.”

Participants also identified language that might unintentionally inspire fear or concern in listeners, such as, “It’s normal to gain a few pounds when you quit smoking, but you are making a change to benefit the health of your entire body.” Participants commented that this reminder about weight gain made them anxious: “You’re trying to relax and all of a sudden it’s like: Weight gain!” Another member said, “I think women…are sensitive to that, any mention of the weight or the wrinkles and we’re like, ‘Oh, this is an issue I have to be worried about!” One participant commented that too much focus on meeting “challenges” (e.g., smoking cessation) makes listeners second-guess their ability to meet those challenges. Instead, they suggested that the audio files reinforce women’s confidence in quitting and center listeners’ attention on its benefits by describing a healthy, smoke-free future.

Theme #2

Highlight the rewards of quitting smoking and taking steps to improve health. Participants wanted imagery that reminds listeners about the improved physical health that comes with lifestyle changes. They requested more visualization related to physiological changes in the body resulting from smoking cessation: “One thing I really like when I’ve quit or when I’ve thought about quitting, is the sort of week-to-week or month-to-month, you know, ‘After two weeks, your nerve endings start repairing,’ and ‘After a month, your lung capacity is 5% better,’ or something like that….these things are happening in your body [when you quit smoking]. I think those would be good for the visualizations.” Another member noted, “We don’t want to feel shameful and ashamed, so knowing that not everything that we’ve done [to our bodies] is permanent is kind of a good thing.” Participants suggested the imagery files help the listener visualize the effects of good nutrition, with one person explaining, “The whole reason I’m eating colorful foods is because you get a lot of nutrients. You don’t mention the nutrients flowing through your body, you know, nourishing your body.”

Participants wanted the recordings to be a “reward” for listeners that would replace the rewarding characteristics of a cigarette. For example, they recommended including enjoyable scenarios, such as walking in fresh air while listening to birds singing, and highlighting increased energy, stamina, and strength. Emphasizing their desire for a reward in exchange for quitting, one participant noted, “Cigarettes are rewarding to us, physically and psychologically, they’re rewarding. …Give me the reward, man!…I quit [smoking]!” This theme assisted the team in creating better integration among the imagery audio files, as discussed below.

Theme #3

Clarify the relationship between smoking cessation, good diet, and improved physical activity. Focus group participants wanted more thematic integration among the audio files. They suggested we emphasize how the physical benefits of individual behavior changes will affect other behaviors. For instance, one participant suggested adding that foods taste and smell better when one is smoke-free: “Everybody I know who has quit smoking talks about how much better food tastes…. They say that smoking’s an oral fixation, so…[another participant interjects: “So address the mouth!] Yeah, address the mouth! And the sensation of that full, ripe peach or a juicy kiwi or something more descriptive, where we really get, ‘Okay, there’s something in my mouth and it tastes good. And it tastes better than a cigarette.’”

They suggested using “cravings” to link the targeted behaviors, so that listeners might consider addressing smoking cravings with physical activity: “I’m thinking of the idea of replacing the “bad” – the cigarette smoking—with working out. So, maybe, if you feel a craving, you should go outside and do a walk around your house or something.” Members noted that many cravings are associated with daily routines (e.g. smoking after dinner), and acknowledged the benefits of multiple behavior changes: “I like [that this audio file] mentioned the replacement of smoking with a new life routine because you’re trying to change a habit that pervades your every day. If you don’t completely substitute new things for this smoking habit—routine–then you’re not gonna be successful.”

In subsequent focus groups, women listened to the extensively revised imagery script audio files. While most audio files’ likeability scores increased to four or more (on a scale of five “like a lot”), “Feel Fantastic” continued to receive less favorable likeability ratings (3 to 3.5). Participants wanted this audio file to: more clearly recognize the struggle involved in quitting; celebrate the accomplishment; and highlight the commitment it takes to quit smoking. Throughout the audio files, participants liked the scenarios described because they could be recalled during the day. In the final group, likeability scores reached at least four for all imagery scripts.

In these final groups, participants offered feedback about the app’s user interface, content, and functionality. Most of the women preferred a pink background design, which they described as “happy,” “light,” and “feminine.” They agreed that a “money-saved” counter would motivate them to continue a smoke-free lifestyle and noted potential pleasure in seeing their cigarette cravings change from high to low on the tracking graph. Participants commented that one motivational message a day for three months is sufficient and does not burden the user with too much app interaction. They also approved of their first names being used in the messages, such as “Tomorrow you will be smoke-free, [Participant’s name]. Remember to do deep belly breaths and practice guided imagery to help you through the tough times.”

Functional Specifications

The 4.0 Android operating system was chosen as the minimum platform version for SMSF in order to capture the maximum number of users and potential research participants, while maintaining functionality that is not available in older operating systems. Interactive app features include the ability to play, pause, stop, and record guided imagery audio files; receive notifications (e.g., reminders, motivational messages, and awards); earn virtual awards and share via social media; access a calendar with event descriptions (e.g., app start date, quit date, and responses to daily questions about cigarette cravings, smoking, diet, and exercise); access links to external web sites with resources for smoking cessation, healthy eating, and physical activity; and a link to the local tobacco quitline. The app home screen also displays time quit, amount of money saved by not smoking, and a graph of cravings over time (Figure 3).

Figure 3.

SMSF Home Screen

The project infrastructure consisted of the SMSF app, a centralized server and a database. SMSF performed data synchronization with the remote server when a new user created an app profile. If the user consented to research participation, data synchronization occurred daily in order to collect responses to surveys (daily questions, baseline, 30 day, 90 day, and consumer satisfaction surveys) and track resources the user accessed during that day (e.g. audio files, awards). The server runs Java Tomcat 7.0 and only accepts secure connections with the protocol HTTPS. The database server runs MySql and was set up to be accessible only via secure connections from the monitoring server.

Internal Quality Control Testing, User Testing

Internal quality control (QC) testing

In internal QC testing, staff created accounts and tested all app components and functionality. Team members were able to identify and correct obvious functionality problems (e.g. addressing logic problems in the app set-up) and research data collection issues (e.g. correcting skip patterns in the surveys).

User testing

In user testing with six participants, all women (100%) rated the app highly (4 or 5 on a five point scale) for its format and design, its ability to address issues about possible weight gain when quitting smoking, and its helpfulness in supporting people in smoking cessation. They reported that they were likely (33.3%) or very likely (66.7%) to recommend this app to other women trying to quit smoking. Participants agreed that the app was easy to use (agree, 66.7%; strongly agree, 33.3%) and that most people would learn to use the app very quickly (agree, 66.7%; strongly agree, 33.3%). However, some users reported difficulty with aspects of the app based on their phone model. Some phone models did not display all features of SMSF; for instance, the “dismiss” button on the motivational messages did not appear on some phones. Users identified app flow, function, design, and language issues that were confusing and which were fixed and user tested in subsequent versions of the app. For instance, some testers were unable to find profile/settings in the app; as a result, the development team created a special profile icon to increase the ease with which users could change their settings. Based on user testing, daily reminders were modified to direct users to the particular imagery audio file scheduled for the day, rather than sending users to the main audio file menu.

Discussion

Creating an effective and engaging mHealth app required a transdisciplinary team and user input throughout development. Changes to SMSF app content and function were made iteratively by the development team as a result of focus group and user testing. Each round of interaction with study participants produced new knowledge, which was used to better target behaviors and tailor the app to individual needs. Moreover, observations during participant activities—such as research staff noting the great variety of phone hardware used by members of focus groups and user testers—helped the team plan for technology contingencies and identify appropriate hardware for internal testing.

Despite extensive testing, unanticipated issues emerged during deployment to the Google Play Store. Although the team tested survey data collection on several phone models, problems with other phone models emerged during deployment, such as survey reminders failing to appear or data failing to transmit, that required periodic app version releases to correct problems. Overall, 84 different phone models (e.g. Galaxy S5) were supported by 16 different carriers and ran a variety of operating systems (i.e., Android 4.4, 5.0). Nineteen updated versions of the app were deployed. Frequently, users did not enable automatic app updates, so study staff contacted participants to request they update their software to the newest version. Older versions of the app could malfunction, leading to user frustration and drop-out or incomplete data collection. Certain phone models did not transmit use data, and bugs associated with some hardware/software combinations limited participants’ ability to complete in-app surveys. When this occurred, study staff emailed participants a link to a web-based survey. Data collected with these two methods were then merged into one database prior to analysis, which resulted in complete data sets but increased the complexity and burden of data management.

Limitations

The lack of racial/ethnic diversity and the relatively high education levels of participants in the focus groups and user tests may be a limitation in the development of the SMSF app. We recruited participants for these activities through online and print media coverage and study flyers posted throughout the community, but we did not target ethnic/racial minorities. Participants in our developmental phase reported having some college education (53%), which is higher than the population of women smokers in the U.S. (19.5%). Our total sample size for focus group testing was small due to our desires to keep group size small and to conduct member checks with previous participants. By collecting data from a small number of participants until saturation, our findings reflect their perspectives on supporting women in quitting smoking, with limited generalizability to the broader population.

Conclusion

More than simply “testing” content and app functionality, focus groups and member checks helped to thematically frame the mHealth intervention and maximize user engagement. Through internal testing, the team was able to identify and resolve functionality problems and data collection issues, but testing missed some phone models’ data transmission problems. User testing was essential in maintaining the app’s usability across phone models and preventing model-specific functionality issues.

Future versions of the app will be expanded to additional software platforms (i.e., iOS). A cross-platform app may have better flexibility to prevent critical data problems through optimization. It will also expand the pool of potential research participants by enabling iPhone, Android, and Windows phone users to participate. In addition, we plan to separate survey data collection from app functionality to streamline data management and improve the research team’s ability to collect, report, and analyze data.