Validating the Information Technology (IT) Implementation Framework to Implement mHealth Technology for Consumers: A case study of the Sense2Quit App for Smoking Cessation

Abstract

Objective:

The goal of this paper was to understand the applicability of the Information Technology (IT) Implementation Framework, a multi-level approach to identify factors that impede or promote IT usage, for incorporating a mHealth technology for consumers in the community setting.

Methods:

A case study of the implementation of the Sense2Quit App for smoking cessation among people living with HIV was examined to parse out the factors within the framework that are applicable to mHealth technology and the factors that may need modification for use of this framework within this context.

Results:

Findings suggest that phases two through five of the framework were applicable to our study and phase one was not.

Conclusion:

Findings support the use of the theory for implementation of mHealth technology for promoting consumer health at the community level. This use case may be useful for developers and researchers evaluating implementation of mHealth for patients with chronic conditions as it highlights the need to identify preferences of app specifications, personal habits, and various factors such as confidentiality and digital literacy which may challenge sustained usage.

Graphical Abstract

1.0. Introduction

The Information Technology (IT) Implementation framework was developed to provide a multi-level approach to identify factors that impede IT usage [1]. The integrative framework was adapted from PRECEDE/PROCEDE, [2] a conceptual framework used by health planners in scores of published public health program planning studies [3–7]. It has been widely used to guide the implementation of IT systems at the clinic and organizational level [7]. Nonetheless, a gap in understanding the multi-level factors that contribute to the implementation of mobile health (mHealth) technologies at the individual consumer level persists. Several technology acceptance models have been applied to examine consumer uptake of mHealth technologies but most of these studies have focused on usability and acceptability of mHealth technology [8–11], rather than implementation and uptake. Therefore, understanding the relevance of an extant IT implementation framework in the context of mHealth technology use at the consumer level is timely and potentially very impactful given the ubiquitous use of mHealth technology and the growing focus on implementation science [12, 13].

To illustrate, mHealth technology has transformed remote access to healthcare. It presents a more equitable and affordable option as approximately 98% of people living in the United States own a smartphone [14]. mHealth has become a pervasive tool for use by patients and consumers in the US and globally [15]. Nonetheless, there have been limited studies on the implementation of mHealth tools for consumer use, [16] and user acceptance of mHealth technology depends on a variety of factors and behaviors.

The IT implementation framework proposes that technology use is multi-dimensional, and that there is a need for active participation of the end users to determine the technology’s success. The authors consider several levels of influence which may maximize or minimize usage behavior. More specifically, this theory considers five phases of influence at the individual and organizational levels. Phase one is an investigation of the organizational needs and goals; phase two identifies components of the expressed needs that can be managed by technology; phase three identifies individual and collective behaviors and environmental factors associated with technology use; phase four identifies predisposing, reinforcing, and enabling factors that influence behaviors linked to technology use; and phase five focuses on developing and implementing factors identified in phase four [1]. We used this framework to guide our analysis on an mHealth tool to identify multi-level factors influencing technology use at the consumer level; rather than assessing needs at an organizational level, however, we focused on individual-level needs, habits, and behaviors among a group of participants living with a chronic condition.

Study Context: To conduct a preliminary validation of this framework, we sought to analyze its relevance to consumer uptake of the Sense2Quit app, an mHealth intervention connected via Bluetooth to a smartwatch, which was created through an iterative process to help people with HIV (PWH) quit smoking cigarettes [17]. The app and smartwatch were tested by participants in the active intervention condition during a pilot randomized controlled trial (RCT). Similar smartwatch technology, which tracks and logs smoking behavior, has been tested in smoking cessation studies yet none of these enrolled PWH specifically [18–21]. While there is no shortage of smoking research among PWH [18, 22, 23], a key population since an estimated 34–47% of PWH smoke cigarettes and PWH who smoke are more likely to die from lung cancer than an AIDS-related cause [24, 25], there is limited research on the implementation of mHealth as a smoking intervention for PWH. The Sense2Quit app was developed and refined through rigorous formative work [26, 27] among PWH and ultimately consisted of the following features: a smoking trends page which displayed smoking data and money spent on cigarettes over the week prior, quit smoking tips and videos, audiovisual distractions including concert and meditation videos, and Pac-Man and Tetris games, a reminders page to add daily reminders, and a chat to communicate with the study team. The Sense2Quit RCT was feasible and acceptable with high usability scores (a mean score of 4.37[SD 0.62] on the Health Information Usability Evaluation which is scored 1 [low] to 5 [high]), a high overall retention rate (91.7%), a continuous increase in app usage throughout the study (with a total data usage of nearly 25GB by the end of the trial), and greater self-reported quit attempts among the intervention arm [28].

This purpose of this paper is to present a well-described theoretical framework and discuss its adaptation for understanding implementation of a novel mHealth tool for use by consumers in the community.

2.0. Methods

2.1. The Sense2Quit Pilot RCT

The Sense2Quit RCT pilot study took place between March 2023 and January 2024. All participants came to the study office for a baseline, 4-week follow-up, and 12-week follow-up visit to complete a survey and to measure their exhaled carbon monoxide using a breathalyzer. Intervention arm participants utilized the mHealth and smartwatch intervention for the 12-week trial period. Study findings can be found elsewhere [28]; smoking history data including baseline Fagerstrom Test for Nicotine Dependence (FTND) scores which range from 0–2 (very low dependence) to 8–10 (very high dependence) are described to contextualize smoking behaviors of interview participants [29].

2.2. Sample Size

We invited all intervention participants who showed up for their 12-week follow-up visit to participate in an interview. The total sample size of the Sense2Quit pilot RCT was 60 since there is a general flat rule to include at least 30 subjects to estimate a parameter to conduct a t-test [28]. A sample size of approximately 60 participants has similarly been used in other mHealth studies [30–32]. Thirty of the 60 participants were randomized to the intervention arm; we conducted interviews with participants until saturation was reached which we estimated would be up to 30 participants. One participant withdrew from the study and three were lost to follow-up thus our final sample was 26. This sample size is consistent with other studies which determined 20 to 30 participants are sufficient to reach saturation of themes in qualitative research [33–35].

2.3. Qualitative Interviews

An interview guide was created by the Principal Investigator (PI) and guided by the Reach, Efficacy, Adoption, Implementation, and Maintenance (RE-AIM) framework to assess the acceptability and perceived usefulness of the Sense2Quit app and smartwatch intervention [36]. Research staff with educational backgrounds in social sciences were trained on how to conduct qualitative interviews by the PI who has extensive experience in qualitative research [4, 7, 27, 37, 38].

A total 26 qualitative interviews were conducted in-person with intervention condition participants from the Sense2Quit pilot RCT at their 12-week follow-up appointment at Columbia University School of Nursing to assess the usefulness of the mHealth app for smoking cessation. Interviews took place between June 2023 and January 2024. All participants signed informed consent prior to interview participation. Interview procedures were approved by the Columbia University Institutional Review Board under protocol number AAAT7031. During the interviews, participants were asked open-ended questions about their experience using the app during the 12-week intervention period (see Table 1). All interviews were transcribed verbatim.

Table 1.

Sense2Quit App Interview Questions

Interview Questions
1	Describe your general perceptions of the Sense2Quit app and its usefulness for helping with tobacco cessation
2	How helpful was the Sense2Quit app for improving your tobacco cessation?
3	What would you change or improve about the Sense2Quit app?
4	How did the Sense2Quit app help you gain information about your tobacco cessation?
5	How comfortable were you in using the Sense2Quit app in social settings? Probe: where and when did you use it mostly?
6	How often did you use the Sense2Quit app in a typical week? Would you recommend it to a friend?
7	Did you stop using the app altogether at some point? (If yes) Why?
8	Describe the usefulness of the reminders to overcome your triggers.

2.4. Qualitative Analysis

An initial codebook was developed using the IT implementation framework; the codes consisted of phase one, phase two, phase three, phase four, and phase five. Phases two through four were each divided into three sub-codes: system objectives, system specifications, and system functionality (phase two); individual behaviors, collective behaviors, and environmental factors (phase three); predisposing factors; and enabling factors and reinforcing factors (phase four). Definitions of each phase and sub-construct were adapted based on Kukafka’s theory to align with interview questions and app functionality [1]. Figure 1 illustrates the five phases.

Figure 1.

Five Phases of the IT Implementation Framework

Following development of an initial codebook, three coders (SF, MB, DH) coded the first five transcripts and met to discuss and refine the codebook. After independently coding the first five transcripts, the coding team reviewed inconsistencies of code applications between coders and updated codes and revised definitions. Once a final codebook was created, the three coders recoded the first five transcripts and independently coded the remaining 21 transcripts, meeting weekly to check intercoder agreement. All coders left memos in the Dedoose software to keep track of questions that arose when coding. In each weekly meeting, coders discussed memos and passages in depth to address any discrepancies in the codes. This iterative coding process which has been used in other team-based qualitative analyses [39] continued until all transcripts had been sufficiently coded. The results section details the adaptation of the definition of each construct of the IT implementation framework to create the final codebook for the analysis of Sense2Quit qualitative interviews.

3.0. Results

3.1. Participant Demographics and Smoking Behaviors

A total of 26 PWH who smoke participated in interviews. The average age of participants was 55.6 years (SD± 10.4). Twenty participants (76.9%) identified as Black or African American, three (11.5%) as White and non-Hispanic, two (7.7%) as multiracial, and one (3.8%) as White and Hispanic. Fourteen participants (53.8%) were cisgender females, ten (38.5%) were cisgender males, and two (7.7%) were transgender females.

Participants estimated the age they started smoking which ranged from 10 to 20 years old with an average of 13.8 years old (SD± 3.3). Most participants (N=21; 81%) had made at least one quitting attempt in the past and 5 (19%) reported that they had never tried to quit smoking previously. FTND scores ranged from 2 (very low dependence) to 9 (very high dependence) with a mean score of 5.2 (SD± 1.8), or moderate dependence [29].

3.2. Qualitative Analysis

3.2.1. Phase One

The IT implementation framework posits the first phase as an identification of organizational needs and goals. The Sense2Quit app was developed at an earlier stage of the study, guided by the Information Systems Framework [40]. Thus, individual needs for PWH who wanted to quit smoking were previously identified through focus groups, design sessions, and usability testing [26, 27]. As a result, we determined this phase did not apply to the Sense2Quit interviews, which consisted of feedback on the app’s usefulness and functionality during the 12-week trial. Phase one was excluded from the codebook.

3.2.2. Phase Two

The second phase identifies which components of the end-users’ needs can be managed by technology. Specifically, the framework emphasizes the importance of assessing the success of system objectives, system specifications, and system functionality. In the case of the Sense2Quit app, the system objective was to aide participants in their smoking cessation journey. As we analyzed the qualitative data, we discovered that although the objective of the intervention was quitting smoking completely, several participants (N=9; 35%) found that the app was more useful for helping them reduce the number of cigarettes smoked daily rather than achieving abstinence. Nearly all participants (N=25; 96%) found the app and smartwatch to be helpful in their quitting process, yet opinions regarding specific features of the app varied.

Specifications of the app included the smoking trends page for smoking and financial tracking, the tips and videos section, reminders page, chat feature, and the games. Generally, participants found these features useful; of note, the reminders function and alerts from the smartwatch reinforced that smoking habits were harmful to overall health. One participant also appreciated the tips page for this reason, saying that they used the tips section to do their own “research”. On the other hand, some participants (N=4; 15%) did not find it useful. One participant said the messages on the watch were a “constant nagging reminder” to quit smoking, while other participants said that the app and watch combo “kept [them] on their toes,” and that “[they] wouldn’t have gotten as far as [they] did without [the app].” The tracking features which logged number of cigarettes smoked, and amount of money spent on cigarettes each week were favorable components of the app. Some people (N=3; 12%) said they retrospectively edited their smoking log at the end of the day to fix any imperfections in the watch’s tracking or to compare to the number of cigarettes they smoked the week prior. One person mentioned the money spent tracking feature as a useful way to figure out “how much money [they’d] wasted.” Some people (N=2; 8%), however, found the money-tracking feature difficult to use because they had to update the price of a pack of cigarettes in the settings whenever they bought a new pack with a different price.

System functionality was modified to specifically address the capabilities of the app and the watch; it was defined in our codebook as the usefulness and accuracy of the combined smartwatch and mHealth app. Importantly, participants’ thoughts about the watch’s ability to track their smoking gestures were coded as system functionality. Feedback on the functionality of the smartwatch tracking varied among participants. Most participants (N=20; 77%) noted that the watch would sometimes inaccurately capture the number of cigarettes they had smoked. One person described the inaccuracy as “frustrating,” and that to use it more in the future, there would need to be improvements to the watch’s detection capabilities. For some participants (N=3; 12%), this was not an issue since they were able to recall the number of cigarettes they smoked and would update the log feature in the Smoking Trends page. Two participants (8%), however, noted that they stopped using the smartwatch during some of the intervention period because they were bothered by the watch’s inability to distinguish smoking from other everyday gestures.

3.2.3. Phase Three

The third phase of the IT implementation framework focuses on the behaviors linked with system use. Individual behaviors were defined in the codebook as the way the mHealth app was integrated into the users’ daily life. Each participant described their use of the app throughout the week, including how many times a week they used the watch and app; responses ranged from three to seven days a week. Participants shared the ways that the intervention suited or did not suit their habits and interests; in particular, some participants (N=4; 15%) shared that they liked playing games on their phones, so this feature was a good distraction when they had an urge to smoke. One participant said they “used the app to distract [them] and fall asleep.” On the other hand, two people said they never played games, so they were not useful for smoking cessation. In terms of the watch usage, responses varied greatly with one person (4%) not liking to wear the watch at all, five (19%) who said they wore it on and off throughout the trial, and seven (27%) who said they wore the watch throughout the day every day.

Collective behaviors, in the context of the intervention, were defined as the way that participants worked with staff to resolve issues within the app. Participants stated they were unable to download the app on their phone if it was inadvertently deleted, or re-connect the smartwatch if Bluetooth was disconnected. Others (N=3; 12%) noted issues with the battery of the watch in that the watch would stop working altogether. These issues required the study staff to step in and assist the participant in using the technology. Within the app, participants also said they found it difficult to use both the financial tracking features and games on their own.

Environmental factors were defined as the way that the participant’s environment influenced or controlled their use of the app. Participants said they were generally comfortable with using the watch in any social setting. Two people (8%) said they did not smoke in public, so they only used the app and watch at home; one also said they “didn’t want people in [their] business,” and preferred that others not see them using the app. Most participants (N=17; 65%), however, said they were not worried about or embarrassed by some seeing them use their watch or app in public.

3.2.4. Phase Four

The fourth phase identifies the factors and behaviors associated with use of the system. The authors separate these factors into three categories: predisposing, enabling, and reinforcing. Predisposing factors translated to knowledge and aptness to use technology in the context of the Sense2Quit study. Several participants (N=4; 15%) stated that they struggled with using the app and watch and one said they could benefit from a tutorial on how to use the intervention properly. Some participants (N=5; 19%) admitted they had little knowledge of how to use their phones or technology in general, which presented a barrier to enrolling in the study. Two participants (8%) also mentioned they had issues with using Bluetooth or did not understand that Bluetooth needed to be always enabled for the watch to maintain communication and tracking between the two devices.

Enabling factors are conditions in the environment that create facilitators or barriers to the system use [1]. In the case of the Sense2Quit app, we used the code enabling factors to identify informational support provided by the app which facilitated access to information and subsequently, smoking cessation. Several participants (N=8; 31%) stated that the app was an educational source for them throughout their quit process; they noted that the app provided easy access to YouTube videos for “inspiration as far as quitting or to distract your mind from instantly going to a cigarette,” as well as easy access to information about tobacco use. On the other hand, one participant said they had been trying to quit for “over forty years,” so they did not learn much from the app.

Reinforcing factors are rewards or incentives to encourage use of the technology; there were not any perceived or financial rewards noted by participants associated with the Sense2Quit app thus reinforcing factors were not included as part of our analysis.

3.2.5. Phase Five

The final phase of the framework focuses on developing and implementing system-use inducing strategies identified in prior phases. We modified the definition of phase five in our codebook to include recommendations for improvement in future development and refinement of the app. Most of the suggestions (N=20; 77%) for the watch were to improve the smoking detection algorithm. One participant suggested creating a feature in the app where users could add a goal of how many cigarettes they wanted to cut down per week, and a summary of how successful they were in reaching that goal. Other suggestions included making the app customizable so that participants could change the color of the interface, incorporate a community aspect where smokers could chat about their quitting process, or adding historical facts into the tips section that did not relate to tobacco cessation. One participant suggested adding some fun facts, for example, about the history of Columbia University.

Illustrative quotes for each construct of the final codebook are listed in Table 2. Quotes are followed by gender, race and age of the cited participant.

Table 2.

Illustrative Quotes for Each Construct

Construct	Illustrative Quote
Phase 2
System objectives	“It was like my little reminder; my little angel on my shoulders telling me, you know, this is not good for you; this is bad, you should stop smoking; the health benefits when you stop smoking; things like that. It was very, very beneficial.” (Female, Black, 34)
System specifications	“I mean the tip page was fine. The smoking thing was ok. But you know, it was pretty much the same information you can get off of like any type of smoking website. Or looking at any other commercial that says smoking is bad for you. Or vaping is bad for you.” (Male, Black, 35)
System functionality	“I liked the whole graphs where you could see the difference. It definitely lets me reflect and see how much I am smoking. I noticed I went from four to five to two and three. To me, that’s great. It’s awesome. So, I am happy about that.” (Female, Black, 34)
Phase 3
Individual behaviors	“I use it mostly at home or out in the park, like outside if I’m sitting. Stuff like that, when I’m bored and got the time. Maybe on the bus, I’d look at the app, and the train. I like that. I put my headphones in and put the phone on. Or waiting for a doctor’s appointment.” (Female, Black, 61)
Collective behaviors	“I came in and the Bluetooth wasn’t connected. And I wasn’t getting a response [from the smartwatch] for a couple of weeks…And I was so confused. I said, ‘Let me just wait till I come in. I got to call.’ I came in. The problem was fixed. I haven’t had a problem with it since then.” (Male, Black, 65)
Environmental factors	“No, in the social setting, it was good. It didn’t bother me opening it up in the social setting. It was good for a distraction, and stuff like that.” (Female, Black, 64)
Phase 4
Predisposing Factors	“I think it was like the second or third week, it was just getting in the way. I was like, ‘Oh my god. What is going on?’ And I’m not a tech geek. And I’m like, ‘Did I press something wrong?’ And so I was like, ‘Okay. Just breathe. Call them and let them know.’ But yeah, it was at one point, I was like, ‘I’m not putting this watch on anymore.’” (Male, White, 51)
Enabling factors	“But honestly, for me, I didn’t know how addicted I was to smoking, I mean, I knew I was a smoker. But I was able to see how bad I was on it, which kind of made me lose confidence in myself.” (Male, Black, 65)
Phase 5 System Use Inducing Factors	“If there was a way for me, like, to customize the secret tips or whatever, then that’d be kind of cool. Because if you can maybe change it up, switch it up to maybe like, well, me in New York. You know what I’m saying? You could probably do like maybe secret tips or like some historical facts. Just to change it up and give it some type of mix-up. So, it’s not always about like smoking and whatever… You know something like, something you really don’t think of and might learn something new that day.” (Male, Black, 35)

4.0. Discussion

The IT implementation framework was selected to guide the coding of the qualitative data due to the wide range of factors and behaviors that are known to influence the uptake and use of technology among PWH, such as stigma and psychosocial barriers [41, 42]. While many other widely cited theoretical frameworks such as Consolidated Framework for Implementation Research (CFIR) [43–47] and RE-AIM [48–50] may be suitable options for this study, we decided to choose a technology-specific theory for our qualitative analysis on a smoking cessation mHealth tool. The IT implementation framework is a cross-theoretical integration of models as it considers disparate theories such as the Technology Acceptance Model (TAM), Behavioral Intention Theory, Diffusion Theory, Social-Cognitive Theory, and Task-Technology Fit Model and posits that they are complementary rather than competing [1]. Since the publication of this framework in 2003, health technology has made significant advancements with the rise of smartphones and proliferation of mHealth apps with more than 350,000 mHealth apps now available globally [51]. Thus, application of this framework to understand uptake of a novel technology by consumers presents a timely analysis for the advancement of implementation science. The goal of this analysis was to provide a use case for the adaptation of the IT implementation framework for understanding mHealth adoption at the consumer level rather than in a healthcare setting. The Sense2Quit study presents an interesting use case for this framework given that it was found to be feasible and acceptable with high rates of app usage and reported usability [28].

Overall, the IT implementation framework was useful in guiding us through the qualitative analysis of the Sense2Quit app as consumer mHealth technology. Components and needs defined in phase two were easily adapted to the context of Sense2Quit. The code of system objectives in phase two allowed us to better understand if the app was well-suited to the needs of PWH who wanted to quit smoking. Most participants said that they were not able to quit completely, but the app was generally helpful in reducing the total amount of cigarettes they smoked. Smoking reduction is often used as a strategy to quit smoking, and studies have shown the efficacy of setting long-term ambitious goals (i.e. quitting completely by a certain date in the future) rather than setting ambiguous, immediate goals [52, 53]. A pilot RCT which enrolled smokers ambivalent to quitting found an enhanced care intervention with exercises to clarify goals, strengthen quitting motivation and learn behavioral skills without making a commitment to quitting had high satisfaction and increased motivation to quit [54]. Another RCT found a mobile game for people not yet ready to quit alongside provision of nicotine replacement therapy can enhance long-term cessation among those who are not yet ready to quit smoking [55]. Thus, use of the app to effectively cut back on smoking may be a promising solution for abstinence in the long-term. Further, it is not surprising that participants struggled to quit given that research shows it takes several attempts before people can stop smoking, with one study showing that it may take thirty or more tries to quit altogether [56]. Additionally, PWH smoke at disproportionately high rates and may experience increased barriers to smoking cessation, such as mental health conditions and substance use [57]. Personalized goal-setting should be a considered feature in future iterations of the app.

System specifications allowed us to understand the usefulness of each app feature and how preferences varied among users. Of note, participants appreciated the app’s notifications on the watch throughout the study period, which served as a reminder to refrain from smoking. Overall, mixed reviews of each feature support the inclusion of several cessation strategies within the app. The sub-code of system functionality revealed a need to make improvements to the smartwatch smoking tracking feature. We took feedback on the app’s functionality into consideration and performed additional testing with confounding gestures to improve the watch’s ability to distinguish smoking from everyday hand gestures [58].

Individual and collective behaviors and environmental factors within phase three also aligned with our study data. The sub-code of collective behaviors allowed us to identify potential barriers to using the app and smartwatch intervention outside of a trial setting. Many participants stated they needed additional support to figure out how to use and navigate the app. The frequently asked questions (FAQ) page was not mentioned as helpful by participants who needed support from staff, suggesting that users may need a more easily accessible or understandable help tool to understand how to use the app. Additionally, environmental factors clarified discreetness of the app. Given stigmatization of people with chronic conditions such as HIV, [59] confidentiality is often a barrier to mHealth adoption and must be taken into consideration when developing mHealth tools that could potentially be accessed by someone other the intended user [60]. Feedback within the code of environmental factors suggested that users felt comfortable using the app in public spaces where others may see them.

Predisposing factors within phase four allowed us to identify digital literacy as a barrier, which is a common challenge in technology adoption, especially among older populations [61]. Lack of understanding of concepts such as the Bluetooth connection created barriers to using the app throughout the study period. Digital literacy is an important consideration when developing mHealth since studies have found positive associations with health literacy, digital literacy, and health outcomes [62]. Given that the mean age of our study participants was 55.6, earlier stages of app development should be appropriately designed for older adults to identify how to overcome barriers associated with knowledge gaps (i.e. Bluetooth) for sustained usage of the mHealth-smartwatch intervention. Additionally, the sub-construct of enabling factors was slightly modified in our codebook because participants did not discuss the individual skills required to use the system or the available resources within the organizational infrastructure, as defined by the authors. Adaptation of this sub-code, however, allowed us to consolidate feedback on the app as an educational tool.

We adapted phase five to include recommendations for future versions of the app since these recommendations would, according to participants, increase use of system. In summary, four of the five phases within the framework aligned with our study data, suggesting transferability of the applied framework to future mHealth evaluations.

4.1. Limitations

Phase one and the sub-construct of phase four, reinforcing factors, were not included in this analysis which is a limitation. Phase one did not apply to interview data since participants were not involved in the development of the app, only in retrospective feedback of the final version of the Sense2Quit app for the current pilot study. Thus, we are not able to make any conclusions about the applicability of this phase in the context of this case study. To incorporate this phase in our analysis, we could have included a question about perceived needs and goals in our interview guide, and how those needs and goals were met, or were not met, with the intervention. Inclusion of this phase would have likely strengthened our understanding of app uptake. As for reinforcing factors, although this theme was not included in our codebook, we had hoped to include some form of reward system in the design of the app, but it was ultimately not included as a feature by developers. Future iterations of the app will likely include a reward system and, as a result, this code would help guide analysis.

As 76.9% of the sample identified as Black or African American, and participants were primarily older adults with a mean age of 55.6 years, the homogeneity of the sample limits applicability of this analysis to a broader population. The small sample size (N=26) also limits generalizability. While we can conclude that participants generally thought the Sense2Quit intervention was helpful in the short-term, we were not able to assess consumer uptake or intentions to use the app in the long-term.

Further, the interview questions primarily focus on usefulness and user experience of the app rather than implementation challenges which limits our ability to understand adoption barriers and sustained use of the intervention. Because the IT Implementation Framework was not used to create the interview guide, questions do not sufficiently reflect the multi-level nature of the framework such as environmental barriers, social influences, and health system integration which limits our interpretation of findings. Given these limitations, we suggest using this framework to develop interview questions in future studies to consider all five phases of influence when analyzing perceived usefulness and adoption of mHealth technology. In particular, applying phase five of this framework to interview questions would allow researchers to inquire about long-term usage of the app.

5.0. Conclusion

This study highlights the effectiveness of the IT Implementation Framework in evaluating mHealth technologies at the consumer level. By applying the framework to the Sense2Quit app, we identified key features contributing to user engagement, as well as areas for improvement in future iterations. The findings reinforce the framework’s adaptability for guiding the use of mHealth interventions, supporting their role in promoting health behavior change within community settings. This use case may be useful for developers and researchers evaluating implementation of mHealth for patients with chronic conditions as it highlights the need to identify preferences of app specifications, personal habits, and various factors such as confidentiality and digital literacy which may challenge sustained usage. Importantly, future research will focus on refining implementation strategies for enhanced usability and long-term adoption.

Highlights.

• A validation of the IT Implementation framework using an mHealth study was conducted

• The Sense2Quit RCT tested a smoking cessation mHealth app for people with HIV

• This use case is useful for evaluating mHealth for patients with chronic conditions

• Findings suggest four phases of this framework were applicable and one was not

Statement of Significance:

Problem	There is a gap in understanding multi-level factors that contribute to the implementation of mobile health (mHealth) technologies at the consumer level.
What is Already Known	The information technology (IT) implementation framework was developed to provide a multi-level approach to identify factors that impede IT usage. It has been widely used to guide the implementation of IT systems at the clinic and organizational level.
What This Paper Adds	We found that the IT implementation framework was useful for examining consumer uptake of our mHealth intervention from a pilot randomized controlled trial which was found to be feasible and acceptable. In particular, phases two through five were applicable to our study and guided our analysis of the Sense2Quit app for people living with HIV who want to quit smoking.
Who would benefit from the knowledge in this paper	Researchers who would like to understand the relevance of an extant IT implementation framework in the context of consumer-level mHealth use.

Data Statement:

The data sets generated during this study are available from the corresponding author on reasonable request.