Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Nov 15.
Published in final edited form as: Curr Dir Psychol Sci. 2024 Jun 3;33(4):239–246. doi: 10.1177/09637214241254328

Towards the Science of Engagement with Digital Interventions

Inbal Nahum-Shani 1, Carolyn Yoon 2
PMCID: PMC11567151  NIHMSID: NIHMS1989610  PMID: 39552747

Abstract

Digital technologies, such as mobile devices and wearable sensors, are ingrained in daily life, making them a promising vehicle for delivering health behavior interventions. However, a critical challenge that undermines the utility of digital interventions is the suboptimal engagement of participants, where participant engagement is defined as the investment of physical, cognitive, and affective energies in a focal stimulus or task. Recent years have seen substantial growth in research aiming to understand how to increase engagement with digital interventions. This paper highlights several limitations of the existing evidence that restrict its scientific and practical utility and discusses opportunities for advancing the science of engagement with digital interventions. Synthesizing the current body of evidence, we call for conceptualizing digital interventions as a collection of stimuli (e.g., notifications, reminders) and tasks (e.g., open the mobile app, practice a relaxation technique) and considering engagement with digital interventions as a process rather than a state (i.e., momentary conditions/experiences) or trait (i.e., a relatively stable disposition). This approach has the potential to enhance scientific rigor and transparency in measuring, reporting, and interpreting engagement with digital interventions that would ultimately serve to bolster progress towards developing strategies for optimizing engagement.

Introduction

Digital technologies like mobile devices and wearable sensors have become integral to our daily lives, making them useful for delivering health-related behavior change interventions. Digital interventions are defined as interventions that leverage technology to facilitate or support positive behavior change, namely the alteration of existing unhealthy/maladaptive behavior, or the uptake and maintenance of new healthy/adaptive behavior (West & Michie, 2016). However, a critical impediment to the effectiveness of digital interventions is the suboptimal engagement by participants. Participant engagement here refers to the extent to which an individual invests physical, cognitive and affective energies in a focal stimulus or task (Nahum-Shani, Shaw, et al., 2022).

In recent years, there has been substantial growth in research focused on understanding and enhancing engagement with digital interventions. Synthesizing the current body of evidence, we suggest an organizing framework to guide the measurement, reporting, and interpretation of engagement with digital interventions. Our proposed framework emphasizes the necessity of conceptualizing digital interventions as a collection of interrelated stimuli (e.g., notifications, reminders) and tasks (e.g., open the mobile app, practice a relaxation technique), while also considering engagement as a process rather than a state (i.e., momentary conditions/experiences) or trait (i.e., a relatively stable disposition). We discuss ways in which this framework can be used to advance the science of engagement with digital interventions.

Suboptimal Engagement with Digital Interventions

Empirical evidence indicates that participant engagement with digital interventions declines over time (e.g., Baumel et al., 2019; Romeo et al., 2019). However, several limitations hinder both the scientific validity and practical utility of the existing evidence. First, engagement with digital interventions, despite being a multi-faceted construct, is primarily measured by how much physical energy an individual invests in the intervention. Typical measures of engagement rely heavily on objective assessments of usage that capture the extent to which individuals interact with the digital tool. Examples include the frequency of visits to the intervention app/website, the number of page views, session completion rates, and time spent interacting with the digital tool (Alkhaldi et al., 2016; Molloy & Anderson, 2021). This narrow focus does not adequately assess the investment of emotional and cognitive energies in digital interventions, and thereby provides an incomplete picture of engagement.

Second, studies typically focus on measuring only one or two indicators of engagement (e.g., self-monitoring completion, module completion), leading to results that likely obscure more complex patterns. For example, Sobolev et al. (2022) found relatively stable response rates to self-monitoring prompts over time (i.e., messages from a mobile device delivered several times throughout the day to encourage the participant to complete a survey about their momentary experiences), suggesting consistent engagement. However, they also found a decline in the number of self-monitoring prompts delivered per day that was influenced by participants’ interactions with their mobile devices. In this study, the number of prompts delivered per day was a function of whether the participants kept their mobile device charged and turned on; this meant that only those who were engaged with the digital tool (here, the mobile device) had the opportunity to engage with the self-monitoring prompt. The results imply that the apparent stability of engagement with the self-monitoring prompt may reflect the behavior of a subset of the participants engaged with the digital tool, thus skewing overall interpretations.

Third, there is a notable gap in testing the associations between engagement and the health behavior outcomes of interest (Gan et al., 2021; Molloy & Anderson, 2021). This gap can be attributed to multiple reasons, including the implicit assumption in digital health research that engagement is inherently beneficial (Nahum-Shani, Shaw, et al., 2022; Yardley et al., 2016) and the absence of standards for conceptualizing and accurately measuring different patterns of engagement (Borghouts et al., 2021; Saleem et al., 2021). This hampers the field’s understanding of the extent of engagement with digital interventions needed by individuals to achieve positive behavior change. Understanding what constitutes ‘optimal engagement’— the level, duration and nature of engagement required to achieve a strategic balance between effectiveness (i.e., doing more good than harm in relation to a pre-specified behavior change outcome) and resource efficiency (i.e., minimizing participant’s time and effort)—is crucial yet underexplored (Collins et al., 2024; Yardley et al., 2016). Notably, what constitutes optimal engagement likely depends on various aspects, including the nature of the target population, the digital intervention in question, and the desired behavior change outcome.

Finally, the heterogeneity of engagement metrics used across studies of digital interventions poses a significant challenge to the generalizability of the findings (Alkhaldi et al., 2016). For example, some studies adopt objective measures (e.g., completion of a digital activity; Carpenter et al., 2023), while others assess perceived engagement via self-report measures (Nahum-Shani et al., 2021). The scope of engagement also varies, with some research focusing on relatively minimal engagement, such as mere interaction with the app or a message (Psihogios et al., 2021), while others measure more effortful types of engagement, like completing lessons or reading articles (Michaelides et al., 2016). Furthermore, the focus of these studies ranges from engagement with purely digital components (e.g., a mobile app; Militello et al., 2022), to those involving both digital and non-digital components (e.g., remote coaching; Suzuki et al., 2023). This varied landscape of engagement research underscores a pressing need for clearly defined and accepted guidelines that would streamline the processes of conceptualizing, measuring, and reporting engagement, especially in relation to its impact on achieving intended behavior change outcomes (Borghouts et al., 2021; Saleem et al., 2021).

An Organizing Framework for Advancing the Science of Engagement with Digital Interventions

Building on the existing literature (e.g., Brouwer et al., 2008; Perski et al., 2017), we propose an organizing framework to guide the measurement, reporting and interpretation of engagement in digital intervention research. This framework highlights the importance of (a) conceptualizing digital interventions as a collection of stimuli and tasks; (b) clarifying whether each stimulus and task is digital and/or non-digital, and (c) considering engagement as a process rather than a state or a trait. Below we discuss each of these aspects.

Digital Interventions as a Collection of Stimuli and Tasks

Digital interventions typically include two basic ingredients: (a) Prompts, which may be in the form of text messages, push notifications, or emails, delivering recommendations or reminders; and (b) On-demand content, which individuals can access at their convenience. Within each element, there are various stimuli and tasks that require participant engagement. In this context, ‘stimulus’ refers to an external cue intended to produce behavior change, and ‘task’ indicates a prespecified performance requirement (Nahum-Shani, Shaw, et al., 2022). Depending on the workflow of the digital intervention, these two basic elements – stimuli and tasks – can be mapped out in many ways. This has implications for how participants are expected to engage with the intervention and how their engagement can be measured.

For instance, consider the Mobile Assistance for Regulating Smoking (MARS) digital intervention, which was designed to support tobacco users attempting to quit (Nahum-Shani et al., 2021). This intervention includes on-demand content in the form of guided mindfulness-based activities in text or audio format, accessible via a mobile app. The app also delivers two types of prompts via push notifications: (1) Low-effort prompts, which recommend brief strategies for regulating and managing craving and negative affect (like suggesting a short walk or a deep breathing exercise); and (2) High-effort prompts, which recommend the use of one of the guided mindfulness-based activities available in the app. Engaging with a low-effort prompt necessitates engagement with a stimulus (the notification) and a task (implementing the suggested self-regulatory strategy). Similarly, engaging with a high-effort prompt requires engagement with a stimulus (the notification) and multiple tasks (including opening the mobile app, selecting the desired content, interacting with the text and audio, and practicing the guided mindfulness-based activity).

Digital and Non-Digital Stimuli and Tasks

The stimuli and tasks in a digital intervention can be digital and non-digital. The distinction between digital and non-digital has implications for the measurement and interpretation of engagement. For example, in MARS, engaging with a high-effort prompt requires engagement with a digital stimulus (the notification), digital tasks (opening the mobile app, selecting the desired content, interacting with the text and audio), and non-digital tasks (practicing the guided mindfulness-based activities). Here, an individual may engage with a digital task (e.g., play an audio with a guided mindfulness-based activity) without engaging with the non-digital task it is intended to promote (e.g., they might be thinking about their plans for dinner instead of practicing the guided activity). In this case, measuring only the individual’s mobile app usage would lead investigators to the wrong conclusion that they were engaged with the prompt.

Engagement with Digital Interventions as a Process

The collection of stimuli and tasks in digital interventions are often sequentially interrelated. This emphasizes the importance of conceptualizing engagement in digital interventions as a process. In the MARS example, the delivery of a prompt depends on the participant keeping their mobile device charged and ensuring it is turned on and not in sleep mode. Since engagement with the device is a prerequisite for engagement with the prompt, it represents a key consideration in interpreting patterns of engagement with the prompt. Therefore, it is essential to measure engagement with both the device and the prompt. This enables the identification of potential breaks in the engagement process that would be informative for the development of effective targeted engagement strategies.

Recommendations and Directions for Future Research

Overall, conceptualizing digital interventions as a collection of digital and non-digital stimuli and tasks and considering engagement as a process calls for researchers to articulate a pragmatic model mapping out how participants are anticipated to engage with a digital intervention. This pragmatic model (see template in Figure 1) can guide the efforts to measure engagement and design strategies for promoting engagement. It can also stimulate scientific inquiries to shed new light on trends of declining engagement and the mechanisms through which engagement can facilitate health behavior change.

Figure 1:

Figure 1:

Open in a new tab

Template of a pragmatic model that maps out the process through which participants are expected to engage with a digital intervention

Measuring Engagement

Our organizing framework emphasizes the importance of measuring engagement as a process of evolving reactions to multiple stimuli and tasks rather than as a state or a trait. Measuring engagement merely as a state may be insufficient for capturing and improving engagement in digital interventions since momentary reactions to one stimulus/task may differ from reactions to another stimulus/task. For example, a person may engage with a stimulus (e.g., read the message in the notification, appreciate the content and consider the recommendation), without engaging with the recommended task (e.g., they may not play the audio with the recommended mindfulness-based activity because they don’t have enough time to do so). In this case, measuring only the person’s engagement with the task would lead investigators to draw the wrong conclusion that the person was not at all engaged.

Measuring engagement merely as a trait-like construct may also be insufficient because this approach implies stability within individuals across time and context. However, since digital interventions require engagement with multiple stimuli and tasks in a context that is highly dynamic (i.e., daily life), rapidly changing conditions may facilitate or undermine engagement, thus resulting in substantial within-person variability. For example, consider a person who has an innate disposition to be engaged with digital health tools. Thus, whenever they receive a high-effort prompt, they read it carefully and appreciate the content (i.e., they engage with the stimulus). However, suppose that in many cases the prompt is sent when this person is performing another task (e.g., childcare or a work-related task) and hence they are unable to engage with the recommended mindfulness-based activity (i.e., the task). Here, measuring engagement as a trait will fail to capture situations in which individuals who are predisposed to engage do not benefit from specific stimuli or tasks due to circumstances that unexpectedly arise in daily life.

Distinguishing between digital and non-digital stimuli and tasks when measuring engagement is also critical. Objective measures of participants’ interactions with digital tools can be used mainly to measure the investment of physical energy in digital stimuli or tasks. While non-intrusive, these objective assessments are limited in their ability to measure the investment of cognitive and affective energies. Subjective, self-reported assessments (e.g., via Ecological Momentary Assessments; EMAs) can offer measures that capture the investment of physical, affective and cognitive energies in digital and non-digital tasks, but they are potentially more burdensome and often not feasible to implement in real-world settings. Thus, investigators must weigh the advantages (e.g., comprehensive engagement measurement) against the disadvantages (e.g., increased burden) when selecting among different approaches for measuring engagement with each stimulus and task in the digital intervention.

Trends of Declining Engagement

Mapping out the process through which an individual is expected to engage with the stimuli and tasks comprising a digital intervention can elucidate the mechanisms underlying the observed decline in participant engagement. The concept of receptivity in the area of digital interventions (Nahum-Shani et al., 2015; Nahum-Shani et al., 2018) refers to the likelihood that an individual will engage with a specific intervention effectively— yielding more benefits than drawbacks (Nahum-Shani et al., 2023). Given that each stimulus and task in a digital intervention likely requires different levels of investment of physical, affective and cognitive energies, receptivity should be considered with respect to each stimulus and task.

For example, consider the high-effort prompts in the MARS intervention. These prompts require engagement with a digital stimulus (the notification), a digital task (play the audio with the guided activity), and a non-digital task (practice the guided mindfulness-based activity). During a meeting, while a person may be able to invest physical, affective and cognitive energies in the stimulus (i.e., attend to the notification, appreciate the message and consider the suggested activity), practical constraints will likely pose a barrier to their ability to invest the required energy in the recommended tasks (i.e., play the audio and follow the guided activity).

As individuals experience the digital intervention over time, contextual factors related to accumulated exposure to intervention-related stimuli and tasks may impact their receptivity. Decline in receptivity over time may signify either beneficial or harmful processes, depending on the specific stimuli or tasks involved.

Processes that are beneficial for behavior change.

Repeated engagement with a digital stimulus over time may facilitate habit formation— a process that plays a vital role in health behavior change and maintenance (Lally & Gardner, 2013). Habits are established through repetitive responses linked to specific stimuli like sensory cues or environmental contexts, resulting in automatic behaviors, irrespective of the expected outcomes (Lerner, 2020). To study habit formation within digital interventions, researchers must clearly specify the stimulus and the corresponding behavior. In the MARS example, if someone forms a habit of performing the digital task (i.e., accessing the app and playing the audio with the mindfulness-based activity) in response to the digital stimulus (i.e., the notification), they are more likely to do so consistently (compared to someone who did not form a similar habit) regardless of external motivations or rewards (Wood & Rünger, 2016). Habit formation enables fast and efficient responses that require minimal cognitive effort to process stimuli (Lerner, 2020). Consequently, habitual responses to notifications could result in lower engagement requirements, potentially reducing the need to read and comprehend messages to access the app and initiate the mindfulness-based activity. Habit formation in mindfulness-based exercises in response to notifications requires repeated exposure and consistent engagement with both stimulus and task. Once formed, receptivity to the notification may decline while it remains consistent for the mindfulness-based activity.

Similarly, repeatedly engaging with a digital task over time can facilitate skill learning, defined as “learning which results following repeated practice” (Schwizer Ashkenazi et al., 2021; P.173). For instance, regardless of how often one engages with the notifications, consistent engagement with the mindfulness-based activity in the app can lead to mastery without digital assistance. This learning process requires repeated engagement with both digital and non-digital tasks. Once the skill is acquired, receptivity to the digital task will likely decline, while receptivity to the non-digital task could remain stable or even grow.

Processes that hinder behavior change.

Decline in receptivity over time may also indicate harmful processes such as habituation and intervention burden. The intensity (i.e., frequency over a specific time period) with which a specific stimulus is delivered during the course of a digital intervention can lead to habituation— the diminishing response to repeated stimulation not attributed to physiological effects like fatigue or adaptation (Moskowitz & Sussman, 2023). As a person habituates to a stimulus (e.g., the notification), their receptivity to it decreases, likely resulting in decreased receptivity to both digital and non-digital tasks recommended by the stimulus, unless other stimuli (e.g., advice from a health professional, different types of digital prompts) boost receptivity. Additionally, prolonged engagement with intervention-related stimuli and tasks can heighten intervention burden—the person’s perception of the overall effort and resources allocated to their healthcare (Boyd et al., 2014). As the burden accumulates throughout the digital intervention, receptivity to all stimuli and tasks comprising the intervention would likely decline.

Summary.

The aforementioned processes suggest that declining engagement with a digital intervention may not necessarily signify its ineffectiveness. Rather, they may indicate that certain digital stimuli or tasks are not needed for downstream non-digital tasks. To investigate these nuanced processes, it is essential to map out the various stimuli and tasks comprising the digital intervention, differentiating between digital and non-digital components. Furthermore, it is necessary to identify stable factors (such as demographics, digital literacy, and symptom severity) and dynamic factors (like prior exposure to and engagement with the specific elements, as well as location and social context) that may shape receptivity to each element.

Engagement in Digital Interventions and Health Behavior Change

Conceptualizing engagement with digital interventions as a process encompassing a collection of stimuli and tasks can elucidate the different pathways (mechanisms) through which the engagement can facilitate positive outcomes. Engagement enhances the likelihood that participants derive benefits from the intervention. For example, practices like deep breathing and meditation foster training in self-regulation, thereby improving mental control and overall well-being (Lekagul et al., 2022). Engaging in these tasks is likely to promote health and well-being through the specific mechanisms targeted by each task.

Moreover, engagement itself can augment health and well-being by fostering development of biopsychosocial resources essential for coping and mental health thriving. Positive engagement entails investing positively valenced physical, affective, and cognitive energies directed toward a focal stimulus or task (Nahum-Shani, Shaw, et al., 2022). This type of engagement is crucial to realizing beneficial behavior changes, as both theories and a growing body of empirical evidence suggest that positive affect and emotions can promote behavior change, whereas negative affect and emotions (e.g., fear and guilt) typically have little effect, and may even backfire (Kok et al., 2018; Shiota et al., 2021). Positively valenced affective energies have been shown to broaden cognition and behavioral repertoires, thereby building personal resources such as mindfulness, resilience, social connectedness, and physical health (Fredrickson, 2001). Furthermore, positive engagement can transform appraisals of future well-being. Appraisals, cognitive evaluations of the personal meaning of situations, are influenced by individuals’ positive engagement with specific stimuli and tasks (Ashar et al., 2017 p.73). Consequently, their perceptions of future well-being and the personal significance of their symptoms may evolve in ways that directly improve their physical and psychological conditions. Thus, mapping out the process of engagement with both digital and non-digital stimuli and tasks can support data collection aimed at addressing scientific questions regarding the mechanisms through which engagement in digital interventions contributes to health behavior change.

Conclusion

Advancing the science of engagement with digital interventions necessitates meticulously focusing on the various stimuli and tasks that comprise these interventions, as well as articulating the process through which participants are expected to engage with these elements to reap benefits. The proposed approach has the potential to improve scientific rigor and transparency in measuring, reporting, and interpreting engagement with digital interventions. Further, it can leverage advancements in experimental designs, such as micro-randomized trials (Qian et al., 2022) and hybrid experimental designs (Nahum-Shani, Dziak, et al., 2022), to investigate the specific conditions that foster engagement with different stimuli and tasks and to test the utility of various engagement strategies. This approach offers an opportunity to accelerate the pace of progress toward optimizing engagement with digital intervention.

Funding:

Inbal Nahum-Shani was funded by National Institutes of Health grants: P50 DA054039; U01 CA229437; and R01 DA039901

Contributor Information

Inbal Nahum-Shani, Institute for Social Research, University of Michigan.

Carolyn Yoon, Ross School of Business, University of Michigan.

References

  1. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, & Murray E (2016). The effectiveness of prompts to promote engagement with digital interventions: a systematic review. Journal of medical Internet research, 18(1), e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ashar YK, Chang LJ, & Wager TD (2017). Brain mechanisms of the placebo effect: an affective appraisal account. Annual review of clinical psychology, 13, 73–98. [DOI] [PubMed] [Google Scholar]
  3. Baumel A, Muench F, Edan S, & Kane JM (2019). Objective user engagement with mental health apps: systematic search and panel-based usage analysis. Journal of medical Internet research, 21(9), e14567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Borghouts J, Eikey E, Mark G, De Leon C, Schueller SM, Schneider M, Stadnick N, Zheng K, Mukamel D, & Sorkin DH (2021). Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. Journal of medical Internet research, 23(3), e24387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Boyd CM, Wolff JL, Giovannetti E, Reider L, Weiss C, Xue QL, Leff B, Boult C, Hughes T, & Rand C (2014). Healthcare task difficulty among older adults with multimorbidity [Article]. Medical Care, 52(3 SUPPL. 2), S118–S125. 10.1097/MLR.0b013e3182a977da [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brouwer W, Oenema A, Crutzen R, De Nooijer J, de Vries NK, & Brug J (2008). An exploration of factors related to dissemination of and exposure to internet-delivered behavior change interventions aimed at adults: a Delphi study approach. Journal of medical Internet research, 10(2), e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Carpenter SM, Yap J, Patrick ME, Morrell N, Dziak JJ, Almirall D, Yoon C, & Nahum-Shani I (2023). Self-relevant appeals to engage in self-monitoring of alcohol use: A microrandomized trial. Psychology of Addictive Behaviors, 37(3), 434. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Collins LM, Nahum-Shani I, Guastaferro K, Strayhorn JC, Vanness DJ, & Murphy SA (2024). Intervention Optimization: A Paradigm Shift and Its Potential Implications for Clinical Psychology. Annual Review of Clinical Psychology, 20, 10.11–10.27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fredrickson BL (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American psychologist, 56(3), 218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gan DZ, McGillivray L, Han J, Christensen H, & Torok M (2021). Effect of engagement with digital interventions on mental health outcomes: a systematic review and meta-analysis. Frontiers in Digital Health, 3, 764079. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Kok G, Peters G-JY, Kessels LT, Ten Hoor GA, & Ruiter RA (2018). Ignoring theory and misinterpreting evidence: the false belief in fear appeals. Health psychology review, 12(2), 111–125. [DOI] [PubMed] [Google Scholar]
  12. Lally P, & Gardner B (2013). Promoting habit formation. Health psychology review, 7(sup 1), S137–S158. [Google Scholar]
  13. Lekagul A, Piancharoen P, Chattong A, Suradom C, & Tangcharoensathien V (2022). Living through the psychological consequences of COVID-19 pandemic: A systematic review of effective mitigating interventions. BMJ open, 12(7), e060804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Lerner TN (2020). Interfacing behavioral and neural circuit models for habit formation. Journal of neuroscience research, 98(6), 1031–1045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Michaelides A, Raby C, Wood M, Farr K, & Toro-Ramos T (2016). Weight loss efficacy of a novel mobile Diabetes Prevention Program delivery platform with human coaching. BMJ Open Diabetes Research and Care, 4(1), e000264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Militello L, Sobolev M, Okeke F, Adler DA, & Nahum-Shani I (2022). Digital Prompts to Increase Engagement With the Headspace App and for Stress Regulation Among Parents: Feasibility Study. JMIR formative research, 6(3), e30606. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Molloy A, & Anderson PL (2021). Engagement with mobile health interventions for depression: A systematic review. Internet interventions, 26, 100454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Moskowitz HS, & Sussman ES (2023). Sound category habituation requires task-relevant attention. Frontiers in Neuroscience, 17, 1228506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Nahum-Shani I, Dziak JJ, Walton MA, & Dempsey W (2022). Hybrid experimental designs for intervention development: what, why, and how. Advances in methods and practices in psychological science, 5(3), 25152459221114279. Provides an introduction to hybrid experimental design – a new experimental approach that can be used to investigate how to best integrate digital and human-delivered strategies for promoting engagement.
  20. Nahum-Shani I, Hekler EB, & Spruijt-Metz D (2015). Building health behavior models to guide the development of just-in-time adaptive interventions: A pragmatic framework. Health Psychology, 34(S), 1209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Nahum-Shani I, Potter LN, Lam CY, Yap J, Moreno A, Stoffel R, Wu Z, Wan N, Dempsey W, Kumar S, Ertin E, Murphy SA, Rehg JM, & Wetter DW (2021). The Mobile Assistance for Regulating Smoking (MARS) Micro-Randomized Trial Design Protocol. Contemporary Clinical Trials, 110, 106513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Nahum-Shani I, Shaw SD, Carpenter SM, Murphy SA, & Yoon C (2022). Engagement in Digital Interventions. The American psychologist, 77(7), 836. Provides a theoretical framework based on a neurophysiological account of engagement, to explain how in-the-moment engagement unfolds in response to a digital stimulus.
  23. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, & Murphy SA (2018). Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine, 1–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Nahum-Shani I, Wetter DW, & Murphy SA (2023). Adapting just-in-time interventions to vulnerability and receptivity: Conceptual and methodological considerations. In Digital Therapeutics for Mental Health and Addiction (pp. 77–87). Elsevier. [Google Scholar]
  25. Perski O, Blandford A, West R, & Michie S (2017). Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Translational behavioral medicine, 7(2), 254–267. Reviews definitions and measurements of engagement across digital intervention studies.
  26. Psihogios AM, King-Dowling S, O’Hagan B, Darabos K, Maurer L, Young J, Fleisher L, Barakat LP, Szalda D, & Hill-Kayser CE (2021). Contextual predictors of engagement in a tailored mhealth intervention for adolescent and young adult cancer survivors. Annals of Behavioral Medicine, 55(12), 1220–1230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Qian T, Walton AE, Collins LM, Klasnja P, Lanza ST, Nahum-Shani I, Rabbi M, Russell MA, Walton MA, & Yoo H (2022). The microrandomized trial for developing digital interventions: Experimental design and data analysis considerations. Psychological Methods, 27(5), 874–894. https://doi.org/ 10.1037/met0000283. Provides an introduction to micro-randomized trials – a new experimental approach that can be used to test the conditions that foster engagement with different stimuli and tasks and to test the utility of various engagement strategies.
  28. Romeo A, Edney S, Plotnikoff R, Curtis R, Ryan J, Sanders I, Crozier A, & Maher C (2019). Can smartphone apps increase physical activity? Systematic review and meta-analysis. Journal of medical Internet research, 21(3), e12053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Saleem M, Kühne L, De Santis KK, Christianson L, Brand T, & Busse H (2021). Understanding engagement strategies in digital interventions for mental health promotion: scoping review. JMIR mental health, 8(12), e30000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Schwizer Ashkenazi S, Sacher Y, & Vakil E (2021). New insights in implicit sequence learning of adults with traumatic brain injury: As measured by an ocular serial reaction time (O-SRT) task. Neuropsychology, 35(2), 172. [DOI] [PubMed] [Google Scholar]
  31. Shiota MN, Papies EK, Preston SD, & Sauter DA (2021). Positive affect and behavior change. Current Opinion in Behavioral Sciences, 39, 222–228. [Google Scholar]
  32. Suzuki J, Loguidice F, Prostko S, Szpak V, Sharma S, Vercollone L, Garner C, & Ahern D (2023). Digitally Assisted Peer Recovery Coach to Facilitate Linkage to Outpatient Treatment Following Inpatient Alcohol Withdrawal Treatment: Proof-of-Concept Pilot Study. JMIR formative research, 7(1), e43304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. West R, & Michie S (2016). A guide to development and evaluation of digital behaviour interventions in healthcare. Silverback Publishing. [Google Scholar]
  34. Wood W, & Rünger D (2016). Psychology of habit. Annual review of psychology, 67, 289–314. [DOI] [PubMed] [Google Scholar]
  35. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, Merchant GC, Naughton F, & Blandford A (2016). Understanding and promoting effective engagement with digital behavior change interventions. American journal of preventive medicine, 51(5), 833–842. Highlights the importance of focusing on optimal engagement and reviews approaches for promoting engagement in digital interventions.

RESOURCES