Microrandomized Trials: Developing Just-in-Time Adaptive Interventions for Better Public Health

Abstract

Just-in-time adaptive interventions (JITAIs) represent an intervention design that adapts the provision and type of support over time to an individual’s changing status and contexts, intending to deliver the right support on the right occasion. As a novel strategy for delivering mobile health interventions, JITAIs have the potential to improve access to quality care in underserved communities and, thus, alleviate health disparities, a significant public health concern.

Valid experimental designs and analysis methods are required to inform the development of JITAIs. Here, we briefly review the cutting-edge design of microrandomized trials (MRTs), covering both the classical MRT design and its outcome-adaptive counterpart.

Associated statistical challenges related to the design and analysis of MRTs are also discussed. Two case studies are provided to illustrate the aforementioned concepts and designs throughout the article. We hope our work leads to better design and application of JITAIs, advancing public health research and practice. (Am J Public Health. 2023;113(1):60–69. https://doi.org/10.2105/AJPH.2022.307150)

Just-in-time adaptive interventions (JITAIs), also known as dynamic tailoring,¹ ecological momentary interventions,² and intelligent real-time therapy,³ represent an intervention design that adjusts the provision and type of support over time to deal with an individual’s changing status and contexts, which intend to deliver the most appropriate support on the right occasion.^4,5 The microrandomized trial (MRT) design has been proposed in recent years as a novel experimental design to construct evidence-based JITAIs. According to this design, participants are sequentially randomized to different intervention options (e.g., whether to send a text message).^6,7 Briefly speaking, JITAIs involve strategies that determine when to intervene and which intervention to provide, while MRTs focus on the optimization of such strategies by selecting and optimizing intervention components for use in a JITAI.

There are 2 key concepts that distinguish JITAIs from standard interventions: just-in-time and adaptive.⁵ By just-in-time, JITAIs intend to intervene only when needed to alleviate the intervention fatigue and low engagement problems. On the other hand, adaptive refers to the strategy employed by the intervention design to determine which intervention to provide and when to intervene according to the user’s ongoing information. To capture the right timings, JITAIs require continuous monitoring of the user’s internal state and contexts, typically via sensors in mobile phones or wearable devices. As a result, delivering just-in-time interventions face to face is not feasible in practice; JITAIs heavily rely on the use of mobile health (mHealth) technologies.⁵

The JITAI in mHealth has the potential to enhance health care and reduce health disparities,^8,9 benefiting various domains of public health research and practice. The MRT has been deliberately introduced to assist with developing these interventions, which can provide information about the dynamics of the best intervention beyond theories and directly inform the construction of JITAIs.⁶ Nevertheless, JITAIs and MRTs have not yet been widely adopted in public health, partly because of the unfamiliarity with the concepts, designs, and analysis methods.

With this article, we aim to introduce key ideas in JITAIs, especially focusing on the novel experimental design of MRTs, covering both classical MRT design and its outcome-adaptive counterpart. We discuss the main characteristics of JITAIs along with their potential in public health by relating to 2 mHealth studies with embedded MRT design. We also highlight statistical considerations when designing and analyzing MRTs.

OVERVIEW OF INTERVENTION AND TRIAL DESIGN

The JITAI is an intervention design aiming to deliver adaptive and personalized support only at the time when needed.^4,5 A JITAI consists of 6 key components: decision points, tailoring variables, intervention options, decision rules, proximal outcomes, and distal outcomes.⁵ The specific definitions are summarized in Box 1. An intervention option is selected at each decision point based on the values of tailoring variables via a predefined decision rule. The intervention is expected to achieve the distal outcomes by directly impacting the proximal outcomes.

Box 1—

Definitions and Examples of Just-in-Time Adaptive Intervention (JITAI) Components

Components	Definitions	Examples (DIAMANTE)
Decision points	Time points or steps at which an intervention decision is made	Once per day (selected within 4 timeframes by an RL algorithm)
Tailoring variables	Individual information and real-world external context	Demographics, health status, and other baseline information; day of study; number of steps walked yesterday; days since each message type was sent
Intervention options	Various types or amounts of support	Motivational messages (4 categories), feedback messages (5 categories)
Decision rules	The strategy that specifies which intervention option to provide at each decision point according to the tailoring information	Optimized by an RL algorithm (linear Thompson sampling)
Proximal outcomes	The short-term goals the intervention options are intended to achieve, which can be mediators or intermediate measures of the distal outcome	Change in daily step counts
Distal outcomes	The ultimate goals of the JITAI, usually a primary clinical outcome	Diabetes and depression

Note. DIAMANTE = Diabetes and Mental Health Adaptive Notification Tracking and Evaluation; RL = reinforcement learning.

On the other hand, the MRT is an experimental design that provides evidence for building JITAIs. It involves the serial randomization of individuals to different intervention options at each decision point.⁵ In general, they can answer several essential research questions arising from the construction of JITAIs:^6,7

• 1.Which intervention will have an impact on the proximal outcomes (proximal effects)?
• 2.What baseline or time-varying covariates will moderate the proximal effects (moderating effects)?
• 3.How will the proximal effects change over time?
• 4.When and how frequently should the intervention be delivered?

In what follows, we will illustrate the MRT design and how it connects to the construction of JITAIs by describing 2 mHealth studies: StayWell at Home and Diabetes and Mental Health Adaptive Notification Tracking and Evaluation (DIAMANTE).

Case Studies

StayWell at Home

The StayWell at Home trial examined the effect of a 60-day text messaging intervention that intended to help individuals manage their depression and anxiety during the COVID-19 pandemic.¹⁰ This MRT consisted of adults aged 18 years or older who had a functioning mobile phone and spoke English or Spanish. Figure 1 shows the study design of StayWell at Home.

FIGURE 1—

Schematic of the Microrandomized Trial (MRT) Design of the StayWell at Home Study

Note. GAD-7 = 7-item Generalized Anxiety Disorder Questionnaire; PHQ-8 = 8-item Patient Health Questionnaire.

The distal outcome was the management of depression and anxiety during COVID-19 social distancing. The proximal outcome was a daily mood rating in the following 3 hours after receiving the message, an intermediate measure of the distal outcome that captures short-term progress toward better management of depression and anxiety. The intervention was supportive text messages, half related to behavioral activation and half about coping skills. Many contextual variables were also collected, including time-independent variables such as demographics and questionnaire data and time-varying variables such as study day and yesterday’s mood rating. The decision point was chosen among 3 timeframes, including 9 am to 12 pm, 12 pm to 3 pm, and 3 pm to 6 pm.

In this design, all participants received uniform random messages. The randomization probability for each message category (behavioral activation vs coping skill) was 0.5, and the probability for each of the 3 time-frames was 0.33. The collected data allowed for pre‒post analysis (i.e., the difference in depression and anxiety levels before and after receiving StayWell messages and the differential effects on mood ratings for the 2 categories of messages and different timings). The pre‒post analysis has already been published,¹¹ while the analysis of MRT data is still underway.

Diabetes and Mental Health Adaptive Notification Tracking and Evaluation

The DIAMANTE trial aimed to evaluate the effect of a text-messaging smartphone application targeting diabetes and depression management through an intermediate outcome representing physical activity.¹² This 6-month MRT study consisted of patients aged 18 to 75 years being treated at the Zuckerberg San Francisco General Hospital who had been diagnosed with diabetes and documented depressive symptoms. Figure 2 presents an overview of the design of this study.

FIGURE 2—

Schematic of the Microrandomized Trial (MRT) Design of the DIAMANTE Study

Note. DIAMANTE = Diabetes and Mental Health Adaptive Notification Tracking and Evaluation.

The distal outcome was the improvement of clinical outcomes for comorbid diabetes and depression among low-income, low‒health literacy, and ethnic minority individuals. Because lack of physical activity is an overlapping risk factor for these diseases, the embedded intervention was focused on improving an easy-to-measure proximal outcome (i.e., changes in daily step count). A multicomponent intervention consisting of motivational messages (4 categories) and feedback messages (5 categories) were adopted in DIAMANTE. Many contextual variables were collected during the study, including time-independent variables such as demographics, mobile technology familiarity, and other engagement measures, and time-varying variables such as study day and day of the week. The decision point was determined either by uniform randomization or some algorithm among 4 time-frames—that is, 9 am to 11:30 am, 11:30 am to 2 pm, 2 pm to 4:30 pm, and 4:30 pm to 7 pm.

At the macro level, this study is a randomized controlled trial with 3 groups, including a uniform random messaging group, an outcome-adaptive messaging group operationalized through reinforcement learning (RL), and a control group. Participants in the control group received no intervention during the study. With 3 groups, it enables the comparison of adaptive messaging to uniform messaging and no intervention. In the initial 2 weeks, uniform randomization was employed in both the uniform random messaging group and the adaptive messaging group to speed up algorithm learning. After that, the uniform random messaging group used a classical MRT design. Patients received up to 2 randomly selected messages per day within 4 randomly selected timeframes. For the adaptive messaging group, the message categories and timing were chosen by a reinforcement learning algorithm (i.e., linear Thompson sampling).¹³ During the conduct of the trial, a JITAI is constructed concurrently, and its characteristics are summarized in Box 1. Data collection for this MRT is currently under way.

Key Design Elements

As an experimental design for empirically informing the construction of JITAIs, the design elements of an MRT study should be tightly connected with JITAI components. The intervention and tailoring variables included in a JITAI are usually supported by theoretical and empirical evidence. An intervention may consist of several components, and each of them may have multiple options. MRTs can be used to assess the proximal effects of 1 or more components simultaneously (e.g., motivational and feedback messages in DIAMANTE). During an MRT, it is essential to collect potential tailoring variables, including individual information and external contexts. These variables can serve as an indicator of individual availability.¹⁴ In the HeartSteps I study, if a participant was driving or already walking, it was considered inappropriate to deliver an activity suggestion, and, thus, the participant was considered unavailable.¹⁵ Potential moderation effects can also be investigated by collecting these variables.

The decision points are determined by the frequency of meaningful changes in the tailoring variables (suggested by empirical evidence and theories), as well as the associated assessment burden.⁵ They might occur (1) at a prespecified time interval, (2) at specific times of day or days of week, or (3) following random prompts.⁵ MRTs can also provide useful information regarding the selection of decision points. In both DIAMANTE and StayWell at Home, meaningful changes in an individual’s context were expected to occur daily. As a result, there was 1 decision point per day. Furthermore, timeframes were treated as an experimental factor to examine the differential intervention effects, which can address the question of when to intervene each day.

At each decision point, participants are randomized to various options of an intervention component—for example, different categories of text messages in DIAMANTE and StayWell at Home, based on predetermined probabilities. These interventions are intended to have an impact on a distal outcome (e.g., diabetes or depression) by affecting an easy-to-measure proximal outcome (e.g., daily step count). Proximal outcomes are often specified as mediators of the distal outcome.⁵ For example, ample evidence suggests that lack of physical activity is a risk factor for diabetes and depression. If the target distal outcome is comorbid diabetes and depression, physical activity would be a natural choice for the proximal outcome.¹²

In some cases, the distal outcomes are sustainable behavior change with limited knowledge of corresponding mediators; hence, proximal outcomes can also be short-term measures of the distal outcome.⁵ For example, daily mood rating is an intermediate measure of depression and anxiety and can be an appropriate proximal outcome.¹⁰ Note that the distal outcome can be affected by the intervention through multiple causal pathways, leading to a multivariate proximal outcome at each decision time in practice.⁵ For example, the proximal outcomes in DIAMANTE and StayWell at Home concern the mechanism lying behind the clinical condition. However, the engagement with intervention may also affect the distal outcome and can be targeted by a JITAI.⁵

During an MRT, each participant may be randomized hundreds or even thousands of times (e.g., 180 times in DIAMANTE and 60 times in StayWell at Home). Randomization permits valid estimation of the intervention’s time-varying proximal effects, as it balances unobserved covariates between the intervention options. An important design element of an MRT is randomization probability—that is, the probability of assigning participants to each option of the intervention component. They are motivated by both scientific and practical considerations. For example, assigning higher probabilities to less-demanding options may reduce participant burden. According to whether the randomization probabilities are updated to prioritize the intervention appearing to be optimal, MRTs can be further categorized as classical (e.g., in StayWell at Home) or outcome-adaptive (e.g., in DIAMANTE).¹⁶ We will illustrate these 2 versions of MRTs in the next section.

Classical Vs Outcome-Adaptive Trial Design

Within the classical MRTs, participants are repeatedly randomized to different intervention options according to a fixed time-invariant scheme (i.e., a scheme wherein the probabilities of being allocated to each intervention option remain uniform over time) or a time-varying allocation strategy, (i.e., a strategy wherein the randomization probabilities depend on the individual’s previous observations).^17,18 Specifically, the latter seeks to avoid excess burden by constraining the number of interventions per day and to spread the interventions uniformly across different strata of decision points (e.g., stressed minutes or nonstressed minutes).¹⁹

However, classical MRTs focus on post‒data-collection JITAI optimization (i.e., data analysis is conducted at the end of the trial [known as “offline” learning in the computer science literature]). Such classical MRTs share similarities with the traditional fixed design (with fixed randomization probabilities) or the biased coin design (with time-varying randomization probabilities) of randomized clinical trials,²⁰ where interventions appearing to be desirable for users (i.e., showing better effectiveness) cannot be prioritized during the trial. The goal is to collect high-quality data that may inform practices for future patients, while participants of the current trial cannot benefit from the new findings.¹⁶

For outcome-adaptive MRTs, the randomization probabilities are adaptively changed in favor of intervention options with superior performance or the highest expected proximal outcome.¹⁶ An essential feature of outcome-adaptive MRTs is that the randomization probabilities are continually adjusted so that the user is assigned to an intervention appearing to be optimal with a higher chance. The outcome-adaptive MRT involves a number of interim analyses during the trial: at every decision point, the design algorithm, usually a reinforcement learning algorithm, selects an intervention option based on historical proximal outcomes and current context, enabling the timely delivery of proper support when needed (known as online learning in the computer science literature). This process leads to an online version of JITAI, which is certainly not the case with classical MRTs.

The outcome-adaptive MRT design is more ethical and efficient when compared with its classical counterpart. First, it can benefit current participants as the intervention delivery is continually optimized to their current context,¹⁶ intending to maximize the cumulative proximal outcomes. The design algorithm can also uncover effective predictors of the right timings when users are more likely to benefit from support and can monitor changes in these predictors, triggering appropriate interventions when needed.¹⁶ Furthermore, the outcome-adaptive MRT may save money by avoiding the collection of useless covariates (i.e., covariates not moderating the intervention effects). More importantly, data collected from outcome-adaptive MRTs can also be used for deriving causal effects and informing offline JITAI construction. Both mHealth and reinforcement learning literatures have provided ways for the analysis of MRTs with time-varying randomization probabilities.^21,22

Despite that, some drawbacks, such as the increase in trial complexity and the statistical inefficiency caused by unequal allocation,²³ raise questions regarding the use of outcome-adaptive MRTs. As a result, careful considerations should be given to the adoption of outcome-adaptive MRTs, especially to ethical issues (e.g., whether the current participants are in urgent need of interventions) and budget constraints (e.g., balancing between the increased human resources because of trial complexity and the cost reduction during the data collection procedure).

POTENTIALS IN PUBLIC HEALTH

As a plausible substitute for face-to-face interactions, JITAIs in mHealth might assist with enhancing health care and reducing health disparities caused by inequities in health care systems, especially in underserved communities.^8,9 First, the relative cost and scalability of these interventions allow for faster delivery of quality care, particularly critical in emergencies. During the COVID-19 pandemic, there has been a surge in interest and use of mHealth to meet the increasing demands of health care. With physical distancing restrictions and a lack of in-person care, mHealth has the potential to improve access to mental health care^24,25 and enable self-management.²⁶ For example, participants who received the StayWell text messaging intervention showed improved depression and anxiety symptoms after the study,¹¹ indicating that such mHealth interventions are beneficial.

Second, the high uptake of mobile technologies among minority and low-income patients has the potential to improve the health care of populations with limited access to traditional health care resources.²⁷ For example, the DIAMANTE study focused on low-income, low‒health literacy, and ethnic minority individuals. These populations may experience higher prevalence and worse outcomes for both diabetes and depression, and, at the same time, they may lack access to health care. Because of the demonstrated effectiveness of DIAMANTE messages,²⁸ deploying these mHealth interventions can potentially decrease disparities in health care by reaching vulnerable populations.

More importantly, JITAIs have a distinct advantage in capturing the exact moment of users’ needs and providing just-in-time support only when it is more likely to benefit the user. As a result, JITAIs may alleviate the practical issues of low engagement and declining effectiveness over time faced by many conventional mHealth interventions, which are delivered uniformly to users without taking into account the contexts and individual information. For these reasons, JITAIs are increasingly being used in various public health domains, including physical activity maintenance,²⁹ mental health management,³⁰ weight loss,³¹ and smoking cessation.³² The effectiveness of JITAIs on various health outcomes has been demonstrated in empirical studies.³³ In general, JITAIs hold enormous potential in public health research and practice.

STATISTICAL CHALLENGES FOR MICRORANDOMIZED TRIALS

In this section, we will describe several vital statistical considerations in the design and analysis of MRTs. When designing an MRT, sample size calculation is crucial, while for an outcome-adaptive MRT, the online “adaptive algorithm” for updating the randomization probabilities adds an additional layer of complexity. In what follows, we will briefly review existing methods related to the preceding issues. Other significant challenges are also summarized.

Sample Size Calculations

When calculating the sample size, it is necessary to specify a primary research question out of all questions of interest. In an MRT study, the primary research question is typically the time-varying effects of an intervention component on the proximal outcome, and sample size calculations are performed to ensure an adequate power to detect statistically significant effects. For example, in StayWell at Home, the primary aim was to investigate whether the type of text messages would affect participants’ moods. To facilitate MRT design, Liao et al.³⁴ proposed an approach to determine the sample size for continuous outcomes by modifying sample size formulas originally developed in the context of generalized estimating equations. Seewald et al.³⁵ developed an online sample size calculator to help domain scientists implement this method. The calculator for binary outcomes can be accessed via https://tqian.shinyapps.io/mrt_ss_binary, while the methodology article has not yet been published online. In addition, Dempsey et al.¹⁷ developed a stratified MRT and provided a sample size formula, and Xu et al.³⁶ proposed a flexible MRT design allowing for the addition of intervention options during the study and derived corresponding sample size estimators. Box 2 presents information on the sample size calculation methods and software. Nonetheless, other open questions still warrant further research in the design of MRTs, such as the sample size formula for MRTs with count (e.g., number of cigarettes, number of active minutes) or ordinal (e.g., mood rating on a scale of 1–9) proximal outcomes, which are commonly seen in practice.

Box 2—

Summary of Statistical Methods for Sample Size Calculations and Data Analysis in Microrandomized Trials

Method	Outcome Type	Functionality	Software
Sample size calculations
Liao et al.34	Continuous	Sample size calculators for MRTs detecting the proximal effects	R shiny app (MRT-SS-Continuous; https://statisticalreinforcementlearninglab.shinyapps.io/mrt_ss_continuous)
Qian et al.14	Binary	Sample size calculators for MRTs detecting the proximal effects	R shiny app (MRT-SS-Binary; https://tqian.shinyapps.io/mrt_ss_binary/)
Dempsey et al.17	Continuous	Sample size calculators for stratified MRTs detecting the nested proximal effects	R code (https://github.com/wdempsey/stratified_mrt)
Xu et al.36	Continuous	Sample size calculators for flexible MRTs (which allows for flexible addition of intervention options) detecting the proximal effects	R shiny app (FlexiMRT-SS; https://kennyxu.shinyapps.io/FlexiMRT-SS)
Data analysis
Boruvka et al.21	Continuous	Estimating causal excursion effect (moderation) of a time-varying component on a time-varying outcome	R (geepack; https://cran.r-project.org/web/packages/geepack/index.html/geeM; https://cran.r-project.org/web/packages/geeM/geeM.pdf), SAS (PROC GEE; https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.4/statug/statug_gee_syntax01.htm), Stata (xtgee; https://www.stata.com/features/overview/generalized-estimating-equations/) R code for small sample correction; https://github.com/StatisticalReinforcementLearningLab/HeartstepsV1Code/blob/master/xgeepack.R
Qian et al.37	Binary	Estimating causal excursion effect (moderation) of a time-varying component on a time-varying outcome	R code (https://github.com/tqian/binary-outcome-mrt)
Shi et al.38	Continuous	Estimating causal excursion effect (moderation) of a time-varying component on a time-varying outcome under potential cluster-level treatment effect and interference	R code (https://github.com/Herashi/MRT-mHealthModeration)
Li and Wager39	Binary	Estimating various causal estimands (short-term and long-term direct effect, long-term total effect) under cross-unit interference	NA

Note. MRT = microrandomized trial; NA = not available.

Randomization Probability Updates

In outcome-adaptive MRTs, the randomization probabilities are continuously changed in favor of better-performing interventions (i.e., the intervention option that can lead to a higher proximal outcome), allowing the optimization of online JITAIs. Reinforcement learning provides an ideal framework for solving such sequential decision-making problems:⁴⁰ an agent continuously interacts with a stochastic environment, or the context, and learns how to make better actions or interventions to maximize the cumulative feedback or proximal outcome over time.

Currently, most methods for constructing JITAIs online within the outcome-adaptive MRT fall in a subcategory of RL (i.e., contextual multiarmed bandits).⁴¹ Various algorithms have been proposed for contextual multiarmed bandits, making different assumptions about the data-generating process.^41,42 In particular, Thompson sampling has demonstrated not only valid theoretical performance guarantees but also strong empirical performance.^43,44 Because of these advantages, as well as its randomized exploration nature, Thompson sampling has been adopted in the DIAMANTE¹² study and the HeartSteps II study.⁴⁵ In addition, other reinforcement learning methods have been employed in mHealth studies. We refer readers to Deliu et al.⁴² and Tewari and Murphy⁴¹ for comprehensive reviews of existing RL approaches for developing JITAIs, and to Trella et al.⁴⁶ and Figueroa et al.⁴⁷ for guidelines regarding the design of online RL algorithms for mHealth interventions.

Data Analysis

Analyzing MRT data and deriving causal effects are critical steps for constructing efficacious JITAIs. Because MRT data include time-varying interventions and endogenous covariates (i.e., depends on previous interventions or outcomes), standard methods for longitudinal data, including generalized estimating equations and mixed effect approaches, can lead to inconsistent estimates of causal effects.⁴⁸

A weighted and centered least squares (WCLS) estimation procedure has been proposed to obtain unbiased estimates of the causal excursion effects of time-varying components on a time-varying continuous outcome.²¹ With some modifications, the WCLS estimator and its standard error can be derived using standard software for generalized estimating equations,^6,21 such as geepack⁴⁹ in R (R Foundation for Statistical Computing, Vienna, Austria) and PROC GEE in SAS (SAS Institute, Cary, NC). This approach can also be generalized to the setting where the randomization probabilities may change over time.²¹

Because the WCLS approach is limited to continuous proximal outcomes, Qian et al.³⁷ proposed a semiparametric estimator of the causal excursion effect in MRTs with binary proximal outcomes. Furthermore, Shi et al.³⁸ developed a general inferential approach for the causal excursion effect with continuous outcome under potential cluster-level treatment effect and interference. Li and Wager³⁹ provided estimation strategies for various causal estimands (i.e., the short-term and long-term direct effect) and the long-term total effect, with a binary outcome under cross-unit interference. See Box 2 for a summary of these methods, as well as the available software. Despite that, specific methods for other data types, such as ordinal or count outcomes, are still not well-developed, as is the case with sample size calculation.

Variable Selection

Within JITAIs, the content and delivery of interventions are tailored to an individual’s ongoing information and external context. Hence, it is necessary to collect all relevant contextual variables and assess the moderation effects of each variable. If there is a large number of potential moderators, we need to conduct moderation analyses using the WCLS approach many times, which can be inconvenient and burdensome. For example, the DIAMANTE study has a high number of baseline and time-varying covariates, which may also interact with the interventions. Although Seewald et al.⁵⁰ recommended prespecifying the relative “priority” of each variable, in practice, scientists may not have a priori knowledge about these variables. Therefore, how best to select a subset of variables for subsequent moderation analyses remains unresolved.

Missing Data

In the pilot study of DIAMANTE,²⁸ there were 670 days with missing steps and 3 participants with 2 or fewer days of step data. A critical issue with analyzing MRT data is how to handle these missing data, as missingness may lead to selection bias in subsequent causal inference.⁵¹ However, there is no comprehensive discussion of missing data issues in MRT. In general, 3 methods to handle missing data have been used in MRT studies: complete-case data analysis,^28,52 single imputation,¹⁵ and multiple imputation.⁵³ In practice, the choice of method should depend on the missingness mechanisms, which can be identified by collecting reasons for missing data during the study.⁵⁰ Nevertheless, no matter which method is adopted in the main analysis, sensitivity analysis is recommended to assess the robustness of the findings, especially when data are not missing at random.⁵⁴

CONCLUSION

In this review, we have summarized the general framework of the JITAI emerging in mHealth, as well as the key concepts, design considerations, and statistical challenges of a novel experimental design, MRT, that can empirically inform JITAIs by assessing the proximal effects and moderation effects. In particular, MRTs can be categorized as classical or outcome-adaptive according to whether the randomization probabilities are adaptively updated in favor of the optimal intervention. When designing outcome-adaptive MRTs, researchers need to choose an appropriate reinforcement learning algorithm for updating these probabilities in addition to the sample size considerations. Nonetheless, there are still some challenges concerning designing and analyzing MRT studies—for example, analysis methods for other types of proximal outcomes, variable selection, and missing data.

With this review, we have sought to introduce the intervention design (JITAI), as well as the cutting-edge experimental design (MRT) of mHealth interventions, to a broad range of readers in the field of public health. We hope our work will lead to greater interest among the public health research community in the uptake of JITAIs and MRT methodologies, ultimately leading to the improvement of human lives.