Discussion of ‘Estimating time-varying causal excursion effects in mobile health with binary outcomes’

1. Introduction

We congratulate Qian et al. (2021) on a thought-provoking paper, which highlights the challenges associated with constructing solutions that are statistically elegant and contribute to the advancement of intervention science. In sequential decision problems where the same intervention is being offered repeatedly, a natural question is whether or not the intervention is effective. For example, consider an ongoing school attendance intervention study, conducted by the Institute of Education Services, in which parents or guardians of children in grades K–2 are sent messages on their smart devices encouraging school attendance (Institute of Education Services, 2020). While investigators conducting this trial are interested in identifying when, in what contexts and for whom these messages are likely to be most effective, their initial, primary, question was whether using text messages is a promising strategy for increasing regular attendance. This point is underscored by a recent report on findings from the study, titled Can Texting Parents Improve Attendance in Elementary School?, issued by the American Institutes for Research (2020), which was contracted to run the study. As the title suggests, investigators want to know whether text-based messaging is worthwhile before investing in subsequent studies aimed at refining the content, timing and personalization of the messages.

Of course, whether a text-based intervention is worthwhile depends on how we operationalize its effect. In working on the attendance messaging study referenced above, as well as many other studies of sequential treatment packages, we have found that intervention scientists often want to address marginal effects first, i.e., does the intervention work, before exploring more nuanced questions such as when does it work and for whom. The causal excursion effect defined by Qian et al. (2021) is a causal estimand that aligns with the goals of intervention scientists. By marginalizing over all but a small subset of patient history, the causal excursion could heuristically be viewed as a kind of main effect in the spirit of classic factorial designs, where levels of other factors have been averaged out. Furthermore, the causal excursion effect has the benefit of obviating the need to model the potentially complex relationship between the interim outcome and patient history.

While the motivation for the causal excursion effect resonates with us, we have heretofore had reservations about defining effects that marginalize over only part of the history, as the estimand then depends on the study design. Thus, two studies that run micro-randomized trials with patients drawn from the same population, and which evaluate the same interventions, may come to substantively different conclusions if they use different randomization probabilities, even if the sample sizes in the trials were infinite. It is possible that one of the two hypothetical studies will decide, based on the causal excursion effect, that an intervention is worth additional investigation, while the other finds otherwise. We illustrate this point through two examples in the next section. Based on these examples, we discuss how exploratory analyses may be necessary to identify severe sensitivity of the causal excursion effect to the trial design.

2. Illustrative examples

We present two simple examples that illustrate the sensitivity of the causal excursion effect to study design. We use the same notation as Qian et al. (2021), and for simplicity we assume that patients are always available and hence omit the availability indicator from .

Example 1.

We first consider a multi-stage randomized trial with binary treatments. Suppose that at each stage , treatment is assigned independently of patient history so that for some fixed constant . Further, suppose that and for any . For , assume that the potential outcome follows a Bernoulli distribution with

for any treatment sequence . Thus, is affected by only the two most recent treatments. We shall consider a situation where and , so that alternating treatments is better than always treating or never treating, which is the case when treatment is effective at the right intensity, for example. For , we choose the proximal outcome to be so that .

Consider the fully marginal causal excursion effect

which under the current generative model simplifies to

It can be seen that is a decreasing function in . Define

as and , it follows that . Moreover, when and when . Therefore, if one were to take the causal excursion effect as a measure of the intervention’s effectiveness, the conclusions would depend critically on the randomization probability: if , the causal excursion effect is positive, suggesting that the treatment is beneficial; if , the causal excursion effect is negative, suggesting that the treatment is harmful; and if , the causal excursion effect is zero, suggesting that the treatment has no effect.

In this example, the causal effect as defined in the marginal structural model is

which is independent of the randomization probability .

Example 2.

We now consider a slightly more complicated setting in which one wants to condition on a time-varying covariate. As in the previous example, the randomization probability remains the same across all the stages so that for all . For each , let be a two-dimensional vector , where is a time-varying covariate and is the proximal outcome. Suppose that , and for any . For , suppose that the conditional distribution of , given all the prior potential outcomes, is Bernoulli with

and that the conditional distribution of , given all the prior potential outcomes, is Bernoulli with

where and so that the right-hand side is between and . Using induction, it is straightforward to verify that the marginal distribution of satisfies for any and any . Furthermore, the marginal distribution of satisfies for any and any .

For , we choose the proximal outcome to be with . Let . We consider the marginal causal excursion effect of on , defined as

By the definition of and the property of conditional probability, we have

The denominator satisfies

for any . To simplify the numerator, we first calculate the probability mass function of the joint distribution of , which is given in Table 1. Therefore

The marginal causal excursion effect is thus

As an illustration, the relationship between and when is shown in Fig. 1. It can be seen that the causal excursion effect depends on the randomization probability . Furthermore, the sign of may change as varies. Let . Because , it follows that . In a trial with , the causal excursion effect given will be negative if and positive otherwise. Thus, treatment appears to be harmful to users with . However, in a trial with , the causal excursion effect given will be positive if and negative otherwise, so that the treatment will appear to be beneficial to users with . Hence, in this contrived example, the interpretation of depends critically on the design.

For comparison, under the marginal structural model it follows that the marginal effect of is

for any treatment sequence and any , whereas under a structural nested mean model the conditional effect of given the full history is

which is independent of and clearly indicates that the treatment is beneficial to users with .

Table 1.

The joint distribution of , where the rows with are omitted because they are not needed for calculation of the causal effect

				Probability mass

Fig. 1.

The causal excursion effect plotted against the randomization probability when , conditional on (solid) or (dashed). The horizontal axis represents the randomization probability and the vertical axis the value of . Thus, the sign and magnitude of the conditional causal excursion effect given depend on the trial protocol through the randomization probability .

3. Discussion

Qian et al. (2021) acknowledge the dependence of the causal excursion effect on the trial design, and argue that if the protocol of the micro-randomized trial is similar to how the intervention might be deployed in practice, then this dependence may be unimportant. It seems that such a condition might be limiting in terms of what kinds of conclusions can be drawn, or actions taken, from the estimated causal excursion effect. As we have seen, a large negative causal excursion effect does not mean that a treatment is harmful; rather, it may mean that the protocol drove patients into states for which treatment appeared harmful. It is therefore not clear how an intervention scientist should proceed if confronted with a negative causal excursion effect. Dismissing the intervention as not being worthy of follow-up could be a serious mistake in that an effective intervention might be overlooked, whereas proceeding with further investigation would seem to ignore the causal excursion effect altogether. At the very least, it appears as though the causal excursion effect would need to be supported by a series of exploratory analyses to examine its sensitivity to the trial protocol. One way of conducting such exploratory analyses would be to use a partially observable Markov decision process where the latent, i.e., unobserved, structure is used to capture the non-Markov dynamics of patient histories. With such a model, one could evaluate different tailored intervention strategies as well as the causal excursion effect under different trial designs. Another approach might be to vary the randomization probabilities or other features of the design across patients so that one could use kernel weighting or regression to estimate how the causal excursion effect varies across these features.

The work of Qian et al. (2021) raises a number of interesting and challenging questions associated with micro-randomized trials and, more generally, the design and evaluation of mobile health interventions. We are inspired by their goal of closely connecting statistical analyses to the primary objectives of intervention scientists and look forward to hearing more of their thoughts and experiences regarding how the causal excursion effect should be used by intervention scientists to guide their research.