Identification and Estimation of Causal Mechanisms in Clustered Encouragement Designs: Disentangling Bed Nets using Bayesian Principal Stratification

Abstract

Exploration of causal mechanisms is often important for researchers and policymakers to understand how an intervention works and how it can be improved. This task can be crucial in clustered encouragement designs (CED). Encouragement design studies arise frequently when the treatment cannot be enforced because of ethical or practical constrains and an encouragement intervention (information campaigns, incentives, etc) is conceived with the purpose of increasing the uptake of the treatment of interest. By design, encouragements always entail the complication of non-compliance. Encouragements can also give rise to a variety of mechanisms, particularly when encouragement is assigned at cluster level. Social interactions among units within the same cluster can result in spillover effects. Disentangling the effect of encouragement through spillover effects from that through the enhancement of the treatment would give better insight into the intervention and it could be compelling for planning the scaling-up phase of the program. Building on previous works on CEDs and non-compliance, we use the principal stratification framework to define stratum-specific causal effects, that is, effects for specific latent subpopulations, defined by the joint potential compliance statuses under both encouragement conditions. We show how the latter stratum-specific causal effects are related to the decomposition commonly used in the literature and provide flexible homogeneity assumptions under which an extrapolation across principal strata allows one to disentangle the effects. Estimation of causal estimands can be performed with Bayesian inferential methods using hierarchical models to account for clustering. We illustrate the proposed methodology by analyzing a cluster randomized experiment implemented in Zambia and designed to evaluate the impact on malaria prevalence of an agricultural loan program intended to increase the bed net coverage. Farmer households assigned to the program could take advantage of a deferred payment and a discount in the purchase of new bed nets. Our analysis shows a lack of evidence of an effect of the offering of the program to a cluster of households through spillover effects, that is through a greater bed net coverage in the neighborhood.

1. Introduction

The main purpose of clinical trials and impact evaluations is to provide evidence to guide the development of policies or programs as well as preventive or control measures. Evidence-based practice has gained considerable interest over the last decades in the fields of economics, psychology, political, social and health sciences. With mediation analysis, research has gone far beyond providing evidence of overall effects. A deep understanding of underlying mechanisms could be used by psychologists, social workers, health services managers or policymakers to better design their interventions. Improvements could include tailoring and focusing on particular successful components of the interventions. Modern causal inference approaches to mediation analysis, grounded in the potential outcomes framework (Rubin, 1974, 1978), have garnered tremendous support among both researchers and practitioners. Indeed, since the first attempts to exploration of causal mechanisms (Baron & Kenny, 1986; MacKinnon et al., 2002), researchers have provided a more formal framework, based on causal effects whose definitions depend on hypothetical interventions on the intermediate variables. These estimands are known in the literature as ’"direct" (or net) and "indirect" (or mediational) effects, which essentially refer to the effect of the exposure or intervention on the outcome respectively not through or through a change in the intermediate variable (Robins & Greenland, 1992; Pearl, 2001). Because of their definitions, they involve quantities, sometimes named a priori counterfactuals (Rubin, 2004) or cross-world counterfactuals, that cannot be estimated from the observed data without strong and assumptions. Most of the approaches to mediation analysis, from the parametric to the latest semi-parametric estimators, hinge on sequential ignorability assumption (Imai et al., 2010a,b; Hafeman & VanderWeele, 2011) that requires un-confoundedness of both the assignment and the intermediate variable, given the baseline covariates. The key assumption of unconfoundedness of the intermediate variable is strong and will often not hold; several authors have tried to address the problem through different techniques such as instrumental variables (Ten Have et al., 2007; Dunn & Bentall, 2007; Albert, 2008; Small, 2012), sensitivity analysis (Imai et al., 2010b; VanderWeele, 2010a), or Principal Stratification (PS) (Frangakis & Rubin, 2002).

Principal Stratification has been introduced in the context of mediation analysis primarily as a way to highlight the limitations of standard approaches, in terms of questionable assumptions and conceptual issues (Mealli & Rubin, 2003; Rubin, 2004; Mealli & Mattei, 2012). Its use to address these limitations has been developed by Hill and colleagues (2002) and subsequently applied with Bayesian estimation techniques by Gallop and colleagues (2009), Elliott and colleagues 2010, and Page (2012). A specific augmented desgin was proposed by Mattei & Mealli (2011); further comparisons can be found in the literature (e.g. Jo, 2008; Lynch et al., 2008; Ten Have & Joffe, 2012). The precise scope of the use of PS is still subject to an ongoing debate (Pearl, 2011; Mealli & Mattei, 2012; VanderWeele et al., 2012).

Oftentimes, the so called Principal Strata Direct Effects (PSDE), that is the effect of the assignment for those whose treatment uptake does not depend on the assignment, are "wrongly" interpreted as "direct" effects for the whole population, implicitly making some kind of homogeneity assumptions across all principal strata (Jo, 2008; Elliott et al., 2010; Flores & Flores-Lagunes, 2009a; Page, 2012).

Interesting questions concerning mechanisms can be raised in a typical non-compliance setting: encouragement designs. Encouragements, such as incentives, different strategies for treatment supply or public policies in general, are used when a treatment cannot be enforced for ethical or practical reasons. Treatments can be therapeutic drugs or programs, preventive measures (e.g., vaccines, condoms, bed nets), protective or risky behaviors (e.g., drug or alcohol abuse). We may think of the treatment as an exposure or intervention that has already been evaluated in previous experimental or observational studies, providing evidence of its beneficial or detrimental effect on the outcome of interest, but its use or disuse cannot be imposed in the population. In these circumstances encouragement interventions can be conceived to foster a behavioral change of the target population. Hirano et al. (2000) were the first to apply the Principal Stratification approach to encouragement designs to estimate intention-to-treat effects within principal strata, i.e. PCE, with and without exclusion restriction assumptions (Imbens & Angrist, 1994; Angrist et al., 1996; Imbens & Rubin, 1997). Oftentimes, in fact, the encouragement is itself a cause of alternative behaviors that would affect the outcome even without inducing a change in the treatment received. Even when the major interest relies on the effect of the encouragement on the treatment uptake and in turn on the outcome, investigating the underlying mechanisms through which the encouragement program achieves its goal is important for both descriptive and prescriptive reasons.

Here we consider Cluster Randomized Encouragement Designs (CED), where encouragement is randomized at the level of a cluster of subjects (e.g villages or communities) because of the specific structure of a community-based intervention (e.g. information campaigns, immunization camps, public policies, etc.) or because of particular constrains, but compliance is at the individual level. CEDs with individual non-compliance can be found relatively frequently in many field experiments (Sommer & Zeger, 1991; McDonald et al., 1992; Hirano et al., 2000;Morris et al., 2004; among others). Frangakis, Rubin & Zhou (2002) extended previous work with PS to account for clustering using Bayesian hierarchical models for inference. To the best of our knowledge no previous work has attempted to apply concepts of mediation analysis to general non-compliance in cluster encouragement settings, with the treatment being the intermediate variable. CEDs are intriguing because they can give rise to several mechanisms that are worthwhile to investigate. In fact, not only does their relationship with the outcome depend on a change in the treatment uptake but often encouragements lead to an overall behavioral change and other actions that can substantially affect the outcome. Furthermore, since the encouragement is randomized at the cluster level, social interactions occurring among people living or working in the same environment give rise to what in the literature is referred to as interference or spillover effects (Sobel, 2006; Hudgens & Halloran, 2008; Tchetgen Tchetgen & VanderWeele, 2012).

VanderWeele et al. (2013) have already attempted to disentangle spillover effects in cluster randomized trials, using sequential ignorability assumptions that accommodate cluster-level assignment and spillovers, but again the assumption made in this setting are very strong.

This paper makes three contributions to the literature. First, we conceptualize the mediating role of the treatment variable in clustered encouragement designs using definitions of effects based both on hypothetical interventions on the treatment uptake and on principal strata. Second, we provide two alternative sets of homogeneity assumptions that enable one to extrapolate information across principal strata and use the estimated PCE to estimate the effects involving a priori counterfactuals. We discuss the flexibility of these assumptions and make clear what specific causal effects can be identified by each one of them. Third, building on previous work (specifically Frangakis, Rubin & Zhou, 2002), we incorporate an imputation-based procedure for the estimation of these intervention-based causal effects under the required assumptions.

Our paper is organized as follows. Section 2 describes the motivating study that we will use to illustrate the methodology. Section 3 provides notation and setup. In Section 4 we introduce the Principal Stratification approach and define a new class of causal estimands that adapt to the context of CED the notion of mechanisms based on a priori counterfactuals. Section 5 presents our innovative structural assumptions deriving the identification results. Section 6 concerns the models for Bayesian inference. In Section 7 we study the KAHS data and Section 8 concludes.

2. Motivational Study: Katete Agriculture and Health Study (KAHS)

The proposed methodology is motivated by the Katete Agriculture and Health Study (KAHS) implemented in Katete District, a rural area with highly endemic malaria in Zambia’s Eastern Province (Fink & Masiye, 2012). From a list of 256 clusters, corresponding to small rural settlements of about 250 households each, the study was restricted to 49 non-contiguous clusters, with a minimum distance of 3 km between each other. The 49 clusters were randomly assigned to one of three arms: 15 were assigned to the control group, 15 to a free net distribution and the other 19 to a subsidized bed net loan program. The purpose of the two ‘encouragement’ interventions was to increase bed nets coverage and ultimately reduce malaria prevalence. Here, we use a subset of the original data from the first and the third arms.

The target population of the study comprised rural farmers, known to be a population group at high risk of malaria. In each cluster, 11 farmer households were randomly selected from a complete listing of all farmers working with Dunavant Cotton, the partner organization of the program. All the households enrolled in the study, in all the three arms, were surveyed twice, once prior to the rainy season and a second time five months later. All the 11 households selected in the clusters assigned to the third arm, after the baseline interview, were allowed to obtain insecticide-treated bed nets (ITNs) at a subsidized price, with repayments due at the end of the harvesting season with a crop sale deduction system. However, not all the households that were offered the subsidies took advantage of them ordering new bed nets, whereas in the control clusters families could also buy new ITNs from local markets. Fink & Masiye (2012) evaluated the average effect of offering the agricultural loan program on the household prevalence of malaria with an Intent-to treat analysis.

There has been an extensive effort over the past decade to show the effectiveness of bed nets uptake in reducing malaria morbidity (Alonso et al., 1993;D’Alessandro, 1995;Nevill et al., 1996). Relying on these results, past studies in this field usually focus on the evaluation of strategies to improve coverage. However few studies attempt to understand how these strategies work and whether their merit goes beyond the increase in bed net uptake. One of the underlying mechanisms that can occur in large-scale interventions is interference. Given the minimum distance of 3 km between clusters, any concern of interference across clusters can be reasonably ruled out. In contrast interference within clusters is likely to take place. Bed net usage yields protection from malaria infection not only for subjects sleeping under them but also for individuals living in the same area. In the literature this effect is referred to as mass community effect. First and foremost, bed nets reduce the reservoir of infection by preventing the physically protected individuals from being infected. This effect is analogous to the contagion effect in vaccine trials (VanderWeele et al., 2012). In addition bed nets commonly used in the last two decades are insecticide-treated bed nets (ITNs). ITNs yield an additional mass effect by affecting the vector of transmissions in three ways. First, insecticides kill adult mosquitos infected with malaria parasites reducing the probability of a person in the community being bitten by an infected mosquito. Second, mass coverage shortens the lifespan of the mosquitos and lowers the possibility for maturation of the parasites, resulting in a reduction of the proportion of mosquitos that become infective. Third insecticides repel mosquitos. It has been argued that the repellant effect of the insecticides can be either harmful or beneficial for those who do not sleep under the nets. In fact, mosquitoes could be diverted to neighboring houses lacking nets. However this fear, plausible at low coverage, has been largely allayed especially if the coverage is high. On the contrary, a massive presence of bed nets might divert certain species of mosquitoes from human to animal biting, thereby reducing human-to-human transmission. These three components are analogues to the infectiousness effect (VanderWeele et al., 2012).

Some of these components of the mass community effect of bed nets have been assessed by researchers in randomized trials (Binka et al., 1998; Howard et al., 2000; Hawley et al., 2003). Nevertheless, none of the encouragement studies have tried to investigate the extent to which interference of the actual bed nets uptake or behavioral changes in the neighborhood plays a role for those who are assigned to receive new bed nets. The purpose of the our analysis of the KAHS study is to investigate the different mechanisms through which the offer of agricultural loans had an effect.

3. Notation and Definitions

In this section we will give formal definitions of the aforementioned effects in the potential outcomes framework (Rubin, 1974, 2005). The setting consists of j = 1, …, J clusters and i = 1, …, N_j units in each cluster with a total of N units uniquely denoted by the pair of indices ij. Let A_j denote a binary cluster encouragement assignment, so that A_j = 1 if cluster j is assigned to the encouragement program and A_j = 0 otherwise. Let M_ij ∈ {0, 1} and Y_ij ∈ 𝒴 denote the treatment received and outcome variables for unit i in cluster j. We also introduce a vector of covariates, C_ij = (X_ij, V_j, h_i (X_−ij)) ∈ 𝒞, where X_ij is a vector of covariates of unit i in cluster j, V_j is a vector of cluster-specific characteristics and h_i (X_−ij) is a function of the vector of covariates of all the units living next to unit i. Finally let A, M and Y be the (J × 1)-dimensional vector of encouragement assignments and the (N × 1)-dimensional vectors of treatment received and outcomes, respectively.

In the KAHS study farmer households are the units of analysis and the clusters of settlements are the units of assignment to either the agricultural loan program (A_j =1) or control (A_j =0). The treatment concerns the purchase of new bed nets between the baseline and the follow-up survey. To simplify the methodology, the analysis is based on a binary treatment variable, being M_ij = 1 if household i in cluster j bought at least one more bed net and M_ij = 0 if no purchase was carried out. In terms of the outcome, let Y_ij be the proportion of reported cases of malaria during the month prior to the follow-up interview in each household i belonging to cluster j. Note that throughout the paper we will use the term "individual" to refer to households.

We now introduce notation for the primitive potential outcomes. Let M_ij (A) denote the potential purchase of at least one bed net a household i would have decided to carry out under assignment vector A. Similarly let Y_ij(A, M) denote the potential outcome that household i in cluster j would have experienced if A and M were the vectors of assignments and treatments received in the whole population.

Assumption 1

Cluster-level SUTVA for the encouragement assignment

Cluster-level Stable Unit Treatment Value Assumption (SUTVA) for the encouragement assignment consists of two parts:

1. An individual’s potential outcomes and potential values of the intermediate variable do not vary with encouragements assigned to clusters other than the individual own cluster, i.e. M_ij (A) ≡ M_ij (A_j) and Y_ij(A, M) ≡ Y_ij(A_j, M_j), where M_j is the vector of dimensions N_J × 1 of treatment received by individuals of cluster j.

2. For each cluster there are no different versions of each encouragement level. Formally:

   $$\text{if }{A}_j=A_j^{\prime }\text{ then }{M}_{ij}(A_j)=M_{ij}(A_j^{\prime })\text{ and if }{A}_j=A_j^{\prime }\text{ and }\mathbf{M}j={\mathbf{M}}_j^{\prime }\text{ then }{Y}_{ij}(A_j,{\mathbf{M}}_j)=Y_{ij}(A_j^{\prime },{\mathbf{M}}_j^{\prime })$$

Cluster-level SUTVA is an extension of the individual-level SUTVA introduced by Rubin (1978, 1980, 1990) to settings with cluster-level assignments and individual-level intermediate variable. Yet it is worth noting that part (i) does not rule out the possibility of spillover effects of the intermediate variable within clusters, that is Y_ij can be affected by the treatment received by other units of the same cluster j. Under cluster-level SUTVA we can use the notation M_ij (A_j) and Y_ij(A_j, M_j).

Note that the only observable potential outcome is the one where, if A_j were set to a, the treatment received by all the units in cluster j were left to the value it would take under encouragement condition a, that is Y_ij (a, M_j (a)). Throughout we will use the notation Y_ij(a) for potential outcomes of this type.

Based on these potential outcomes, the overall average effect of the cluster encouragement intervention on the individual outcome, referred to as Intent-to-Treat Effect (ITT), within each level c of baseline covariates, is defined as the following contrast:

$$ITT(\mathbf{c})≔\mathrm{E}[Y_{ij}(1)|{\mathbf{C}}_{ij}=\mathbf{c}]-\mathrm{E}[Y_{ij}(0)|{\mathbf{C}}_{ij}=\mathbf{c}]$$ (3.1)

To shed light on the heterogeneity of the effects, we will define all causal estimands as average effects within levels of the baseline covariates C_ij.

4. Principal Stratification Approach

Principal stratification has been first introduced by Frangakis and Rubin (2002), in order to address post-treatment complications. Its use in mediation analysis has been proposed as a way to relax the sequential ignorability assumption and focus on so called principal strata direct effects (VanderWeele, 2008; Gallop et al., 2009; Elliott et al., 2010; Page, 2012; Mattei & Mealli, 2011). The units under study can be stratified in subpopulations, the so-called Principal Strata, defined according to the potential values of the actual treatment received:

$$S^{m_0{m}_1}≔\{i:M_{ij}(0)=m_0,M_{ij}(1)=m_1\}$$ (4.1)

Since only one of the two potential values is observed, these four subpopulations are latent, in the sense that in general it is not possible to identify the specific subpopulation a unit i belongs to. Let S_ij be the indicator of the latent group to which subject i belongs. When both A_j and M_ij are binary there are 4 strata S_ij ∈ {S⁰⁰, S¹¹, S⁰¹, S¹⁰}, often referred in the literature on compliance as never-takers, always-takers, compliers, and defiers. Strata membership can also be referred to as compliance status.

In the bed nets application household can be divided in principal strata based on the behavior in terms of bed nets uptake under both encouragement conditions. Never-takers are the families who would not buy a new bed net neither if assigned nor if not assigned to receive subsidies, always takers are those who would buy new bed nets anyway, compliers those families who would buy new bed nets only if they were offered subsidies and defiers would be those who would not buy new bed nets with subsidies but would carry out the purchase at full price. We can argue that this last category is not plausible in this setting and thus make the following assumption.

Assumption 2

Monotonicity of Compliance

Monotonicity of encouragement assignment on treatment receipt requires

$$M_{ij}(0)\le M_{ij}(1)\hspace{1em}\forall i,j$$

This assumption conveys the idea that there is no unit who would take the treatment if not encouraged to do so but would not if encouraged. This restricted pattern of compliance behavior enables the conditional distribution of compliance status to be consistently estimated. In fact, if we let π_m0m1 (c) ≔ P(S_ij = S^m₀m₁ | C_ij = c) denote the probability of belonging to stratum S^m₀m₁ conditional on baseline covariates, the monotonicity assumption implies the following result ∀c ∈ 𝒞:

$${\pi }_{10}(\mathbf{c})=0;\hspace{1em}{\pi }_{11}(\mathbf{c})=P(M_{ij}(0)=1|{\mathbf{C}}_{ij}=\mathbf{c});\hspace{1em}{\pi }_{00}(\mathbf{c})=P(M_{ij}(1)=0|{\mathbf{C}}_{ij}=\mathbf{c});\hspace{1em}{\pi }_{01}(\mathbf{c})=1-{\pi }_{11}(\mathbf{c})-{\pi }_{00}(\mathbf{c})$$ (4.2)

As mentioned previously, in the KAHS study this assumption is plausible because there should not plausibly be any reason to buy a bed net at a full price but not with subsidies.

4.1 Principal Causal Effects

The overall effect of the cluster encouragement within each principal stratum and within levels of baseline covariates is named principal causal effect (PCE) and is defined as:

$$PCE(m_{\mathit{0}},m_{\mathit{1}},\mathbf{c})≔\mathrm{E}[Y_{ij}(1)|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]-\mathrm{E}[Y_{ij}(0)|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]$$ (4.3)

ITT is then a weighted average of PCEs, with weights given by the conditional probability of belonging to each principal stratum:

$$ITT(\mathbf{c})=\sum _{m_0{m}_1}PCE(m_{\mathit{0}},m_{\mathit{1}},\mathbf{c})·{\pi }_{m_0{m}_1}(\mathbf{c})$$ (4.4)

In principal strata where the treatment receipt is unaffected by the encouragement, i.e., never-takers and always-takers, principal causal effect, PCE(m, m, c) with m ∈ {0, 1}, are called dissociative causal effect (DCE(m, c)).

DCEs include all the mechanisms that do not involve a change in the treatment received. In particular they are a combination of two different types of effects: pure encouragement effects, that is, effects of the cluster encouragement through modification in the environment or in individual behaviors other than in the treatment receipt (Frangakis, Rubin & Zhou, 2002), and effects due to mechanisms of interference, by behavioral changes in other inhabitants of the same cluster, both in terms of treatment receipt or in terms of other actions. Several behavioral changes often occur when the encouragement intervention is provided by information campaigns that will increase the awareness of the problem and also encourage the use of other measures together with the treatment of interest. Interventions designed to boost the use of bed nets often comprise different components that are responsible for different mechanisms. First, encouragements, such as subsidies, could influence the usage as well as the quantity of new bed nets; second, an awareness-raising component could lead to a better usage of old bed nets as well as the uptake of other preventive measures such as repellents or mosquito screens for windows and doors; third, another component could be a village cleaning or disinfestation.

The difference between the two dissociative effects for never-takers and always-takers can be substantial. On the one hand, this can be due to the possible encouragement-treatment interaction, that is, a change on the effect of A_j on Y_ij depending on the treatment uptake M_ij ; on the other hand, the different inherent characteristics of the two strata can influence the way the encouragement has an effect on their outcome.

Estimation of the latter effect within levels of covariates C_ij would allow one to identify the individual as well as cluster characteristics of the units who do not get any benefit from the cluster intervention when they do not take the treatment, neither through interference nor through other mechanisms. In the phase of scaling up the intervention to other communities, alternative targeted strategies can be applied to people with these characteristics, e.g., free distribution of bed nets. Moreover, estimation of the effect for always-takers will provide us with a better understanding of the relevance of the encouragement and also whether the encouragement itself has a beneficial effect even for this sub-population. As far as compliers are concerned, PCE(0, 1, c) is a combination of all the aforementioned mechanisms as well as the effect of the encouragement involving a change in the individual treatment uptake.

4.2 Individual Treatment Mediated Effect and Net Encouragement Effect

In order to disentangle for the whole population the two different types of causal mechanisms, through or not through a change in the individual treatment uptake, it is necessary to introduce quantities based on hypothetical interventions on the intermediate variable. Let us decompose M_j into M_j = [M_ij, M_−ij], where M_−ij denotes the vector of treatment taken by all the individuals in cluster j, except for unit i, and let M_−ij (a) be its potential value under A_j = a. We can then rewrite the potential outcomes Y_ij(A_j, M_j), already defined in section 3, as Y_ij(A_j, M_ij, M_−ij). Let us now consider a particular intervention on the intermediate variables that would set M_j = [M_ij, M_−ij] = [m, M_−ij (a)]. Among the 2^N_j+1 potential outcomes that can be conceived for each unit, based on a joint intervention on the encouragement and on the treatment receipt, we will focus solely on 4 of them, precisely the ones of the form Y_ij(a, m, M_−ij (a)), denoting the outcome that unit i in cluster j would have experienced if cluster j were assigned to the encouragement status A_j = a, the treatment received by unit ij were set to M_ij = m and all the other individuals in the cluster could take the treatment they would have taken under the encouragement status that has been set to a. Since the third term in the potential outcome is a function of the encouragement condition, we will use the simplified notation: Y_ij(a, m) ≡ Y_ij(a, m, M_−ij (a)). A peculiar case occurs when M_ij is set to the value it would take under encouragement ã, i.e. Y_ij (a, M_ij (ã)). As mentioned, potential outcomes of this form require that we conceive, together with the clustered encouragement intervention, an additional intervention that is able to set the treatment received by each subject to a specific value, without having any effect on the outcome. For instance, the joint intervention underlying the potential outcome Y_ij(1, 0) is conceivable if there were a rationing, that is the number of bed nets available in the program were less than the number that households belonging to the villages where the program was implemented could potentially request. We then can think of an intervention that offers subsidized bed nets to household ij, but at the same time it creates the condition for which that household finds nets out of stock, assuming no secondary consequences.

Potential outcomes of the type Y_ij(a, M_ij (ã)), whenever they can be deemed well-defined, allow the definition of causal estimands that decompose the overall encouragement effect into causal mechanisms, through or not through a change in the individual treatment uptake: Individual Treatment Mediated Effect (iTME) and Net Encouragement Effect (NEE). Note that in this article no attempt will be made to disentangle spillover effects from pure encouragement effects. We now give formal definition of the two main casual mechanisms of interest, within principal strata.

We define Net Encouragement Effect (NEE) within principal stratum S^m₀m₁ as the following contrast:

$$NE{E}^{ã}(m_0,m_1,\mathbf{c})≔\mathrm{E}[Y_{ij}(1,M_{ij}(ã))|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]-\mathrm{E}[Y_{ij}(0,M_{ij}(ã))|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]$$ (4.5)

In words, it is the difference between potential outcomes under the two encouragement conditions intervening to keep the individual treatment received by unit ij, M_ij, fixed at the value it would take under A_j = ã, averaged over all units belonging to the principal stratum S^m₀m₁ and with values of covariates C_ij = c. This quantity represents the effect of the encouragement on the outcome net of the effect of the treatment uptake. By definition, NEEs are a combination of spillover effects by intermediate variables of other subjects belonging to the same cluster and other mechanisms that do not involve a change in the individual treatment uptake. In the KAHS study, NEE^ã(m₀, m₁, c) indicates the average, over all units with C_ij = c and belonging to principal stratum S^m₀m₁, of the effect of offering subsidies to the 11 farmer households enrolled in the study and belonging to the same cluster on the risk of malaria for one of these units, not through the change in the number of bed nets owned by the household itself and, specifically, if we intervened to keep the binary indicator of bed nets purchase of this household to what it would have been under the clustered encouragement status A_j = ã.

Likewise, the Individual Treatment Mediated Effect (iTME), for each encouragement condition A_j = a, is given by the following expression:

$$iTM{E}^a(m_0,m_1,\mathbf{c})≔\mathrm{E}[Y_{ij}(a,M_{ij}(1))|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]-\mathrm{E}[Y_{ij}(a,M_{ij}(0))|S_{ij}=S^{m_0{m}_1},{\mathbf{C}}_{ij}=\mathbf{c}]$$ (4.6)

In words, it is the average difference of the potential outcomes, within each principal stratum and within each level of the covariates, resulting from an intervention that varies the actual treatment for each unit i in cluster j, M_ij, from the one that this unit would have received having assigned cluster j to the active encouragement condition, A_j = 1, to the one that it would have received under the control encouragement condition, A_j = 0, keeping the encouragement status fixed at a. Precisely this quantity captures to what extent the encouragement achieves its aim through its main characteristics, i.e., an increase or reduction of the treatment uptake in the population. ¹

Let us focus now on the strata S^mm = {i : M_ij (0) = M_ij (1) = m}, with m = {0, 1}, where the individual treatment received, M_ij, does not depend on the encouragement intervention A_j, namely never-taker (m=0) and always-takers (m=1). Within these two strata the individual treatment mediated effect is canceled out and the dissociative causal effect equals the both net encouragement effects:

$$DCE(m,\mathbf{c})\equiv NE{E}^0(m,m,\mathbf{c})=NE{E}^1(m,m,\mathbf{c})$$ (4.7)

In contrast, the overall effect of the clustered encouragement for compliers decomposes into the net encouragement effect and the individual treatment mediated effect:

$$PCE(\mathit{0},\mathit{1},\mathbf{c})=NE{E}^{1-a}(0,1,\mathbf{c})+iTM{E}^a(0,1,\mathbf{c})$$ (4.8)

In our example, iTME^a(0, 1, c) represents the average effect of the agricultural loan program on the proportion of malaria cases experienced by each complier household with C_ij = c through an increase in the number of bed nets owned by the household itself, under the clustered encouragement status A_j = a.

Equation (4.8) is the typical decomposition found in mediation analysis. The intuition behind this decomposition, with different indexes for the two complementary effects, arises from the possible presence of an interaction between the treatment variable and the encouragement intervention. In fact, translating the work by VanderWeele (2014) into this context, the total effect of the intervention for compliers is given by the sum of the effect of the intervention when they do not buy new bed nets, the effect of buying new bed nets when the program has not been offered to the whole cluster plus the differential effect of new bed nets in the presence or absence of the program in the cluster. This last interaction effect could be included in the individual treatment mediated effect, resulting in the decomposition given by NEE⁰(0, 1, c) + iTME¹(0, 1, c), or in the net encouragement effect, resulting in NEE¹(0, 1, c) + iTME⁰(0, 1, c). The choice of one or the other decomposition, indexed by a, depends on the specific application. As we can see compliers are the only units to actually exhibit a non zero iTME besides a possible difference between the two NEEs.

In any case, a conceptual point has to be made. In this application the effects NEE¹(0, 1, c) iTME⁰(0, 1, c) are problematic because they involve the potential outcome Y_ij(0, M_ij (1)), which for compliers is equal to Y_ij(0, 1). This quantity is not well-defined because it would require an intervention that sets M_ij to 1, namely that makes a complier household ij buy at least one new bed net, while each household in cluster j, including ij, is not assigned to the loan program. Since the purchase of bed nets is a treatment that cannot be enforced, such intervention is hard to conceive and it would rather be another kind of encouragement that would affect the actual number of bed nets bought as well as lead to other mechanisms. On the contrary, NEE⁰(0, 1, c) and iTME¹(0, 1, c) involve the potential outcome Y_ij(1, M_ij (0)), which is equal to Y_ij(1, 0) for compliers. This quantity hinges on an intervention that sets M_ij to 0, namely that precludes the purchase of any new bed net for a complier household ij, while each household in cluster j, including ij, is assigned to the loan program. This might be easier to conceptualize if we think on the rationing intervention described earlier.

In light of these considerations, the scope of our analysis will be to disentangle NEE⁰(0, 1, c) and iTME¹(0, 1, c) for compliers and estimate dissociative causal effects for always-takers and never-takers.

We can now derive population effects. The population net encouragement effect, averaged over subgroups of the population with the same level of covariates, is given by the weighted sum of the net encouragement effect of all the strata:

$$NE{E}^0(\mathbf{c})=\sum _{(m_0,m_1)}NE{E}^0(m_0,m_1,\mathbf{c}){\pi }_{m_0{m}_1}(\mathbf{c})=\sum _{m}DCE(m,\mathbf{c}){\pi }_{mm}(\mathbf{c})+NE{E}^0(0,1,\mathbf{c}){\pi }_{m_0{m}_1}(\mathbf{c})$$ (4.9)

Conversely, the population intermediate treatment mediated effect, averaged over subgroups of the population with the same level of covariates, results from intermediate treatment mediated effect for compliers, scaled by the conditional probability of belonging to this principal stratum:

$$iTM{E}^1(\mathbf{c})=iTM{E}^1(0,1,\mathbf{c}){\pi }_{01}(\mathbf{c})$$ (4.10)

By virtue of the particular behavior of compliers, i.e. M_ij (0) = 0 and M_ij (1) = 1, we can interpret their individual treatment mediated effect for compliers as the average causal effect of the receipt of treatment within this subpopulation (see supplemental materials). This makes clear, then, how the individual treatment mediated effect, being a product of two quantities, represents both the impact of the encouragement on the treatment take-up (π₀₁(c)) and the treatment effect on the outcome (iTME).

5. Assumptions for Causal Mechanisms

Throughout we will make the following assumption:

Assumption 3

Unconfoundedness of the cluster encouragement assignment

Conditional on a set of covariates C_ij, the encouragement status of each cluster, Aj, is independent of all the potential outcomes and the potential values of the treatment received:

$$\{Y_{ij}(a),M_{ij}(ã)\}⫫A_j|{\mathbf{C}}_{ij}=\mathbf{c}\hspace{1em}\hspace{1em}\forall \mathbf{c}\in 𝒞,a=ã=\{0,1\}\text{ and }\forall i,j$$

When the encouragement is randomized, unconfoundedness of the encouragement assignment holds without conditioning on covariates. This is actually the case in the KAHS study. It is worth remarking that assumption (3) implies that the encouragement is also unconfounded within principal strata and levels of baseline covariates, that is Y_ij (a) ⫫ A_j | S_ij = S^m₀m₁, C_ij = c.

If strata memberships were known this unconfoundedness assumption would allow to identify principal causal effects comparing the outcome under the two encouragement conditions of individuals with the same values of compliance status and covariates. Unfortunately we do not in general know which individuals are in which principal stratum. Observed value of the intermediate variables are in general mixtures of different principal strata. The monotonicity assumption (2) allows one to identify as always-takers the units in the control group who take the treatment and, similarly, as never-takers those who do not take the treatment under encouragement. However, in all other observed groups strata membership is not known. Without additional assumptions - such as exclusion restrictions, which substantially rule out the presence of net effects and hence cannot be invoked in these settings where these are effects of interest - this missing information results in models and causal parameters that are not fully identified, i.e., for which a consistent estimator does not exist. Nevertheless, the use of Bayesian inference circumvents this identifiability problem because, even when causal estimands are intrinsically not fully identified, posterior distributions are always proper when proper priors are assumed. Once principal causal effects have been estimated, a full assessment of causal mechanisms, as defined in the previous section, solely requires a last step, which is the decomposition of PCE(0, 1, c) for compliers into NEE⁰(0, 1, c) and iTME¹(0, 1, c).

NEEs and iTMEs involve potential outcomes of the form Y_ij(a, m) and, in particular, causal estimands of interest in the application of this paper are defined based on comparisons between Y_ij (1, M_ij (0)), Y_ij(0, M_ij (0)) and Y_ij(1, M_ij (1)). Information about potential outcomes of this form for each unit is not in general in the data. In one specific experiment, where only the encouragement assignment is randomized, only one of all these possible potential outcome is ultimately observed, namely Y_ij(A_j, M_ij (A_j)), where A_j is the encouragement status assigned to cluster j. Potential outcomes of the type Y_ij(a, m), with m set to a particular value for all units or to M_ij (ã), are observable only if m ≡ M_ij (a), in which case the potential outcome collapses in Y_ij(a). This occurs if the treatment receipt for unit ij is actually set to M_ij (a) or if it is set to M_ij (ã), with ã ≠ a, but for this unit M_ij (0) ≡ M_ij (1), i.e., the unit is a never-taker or an always-taker. On the contrary, potential outcomes can never be (not even potentially) observed for units with M_ij (a) ≠ m, hence they are called a priori counterfactuals (Rubin, 2004). Here the problematic counterfactuals are Y_ij(1, M_ij (0)) for compliers. Estimation of these missing potential outcomes would require an extrapolation from other individuals in the data, based on specific assumptions.

Identification of effects involving a priori counterfactuals typically relies on sequential ignorability assumptions (see Ten Have & Joffe (2012) for a review of the different specifications). The critical feature of evaluating causal mechanisms in cluster randomized encouragement designs (CED) is that even if the experiment randomizes the encouragement, the intermediate variable, i.e., the actual treatment received, is instead self-selected by individuals. Consequently, unconfoundedness of the intermediate variable required by the sequential ignorability assumption is unlikely to hold, even conditioning on observed covariates. In fact, in our empirical study, households’ decision of carrying out the purchase of new bed nets depends on observed but, presumably, also on unobserved characteristics. Here we propose the use of Principal Stratification approach to, primarily, estimate the overall effect of the clustered encouragement for each principal stratum and, subsequently, recover individual treatment mediated effects and net encouragement effects for all principal strata.

5.1 Homogeneity Assumptions

When the sequential ignorability assumption does not hold, information about Y_ij(1, M_ij (0)) for compliers cannot be extrapolated across strata and thus the principal causal effect for this sub-population cannot be decomposed into the two causal estimands of interest, NEE⁰(0, 1, c) and iTME¹(0, 1, c). Here we provide two alternative homogeneity assumptions that enable to make use of the information available in the strata S^mm, with m = 0, 1, where all potential outcomes are observable, to estimate a priori counterfactuals in other strata. Essentially, these assumptions solely concern the missing information and only allow the extrapolation that is strictly needed across strata with a similar compliance behavior at least under one encouragement condition. Here we focus on identification of Y_ij(1, M_ij (0)) for compliers and present two alternative assumptions that allow to identify the same causal estimands, namely NEE⁰(0, 1, c) and hence iTME¹(0, 1, c), in two different ways.

Assumption 4

Stochastic Homogeneity of the Counterfactual across Never-Takers and Compliers

Stochastic homogeneity of the counterfactual Y_ij(1, M_ij (0)) across never-takers and compliers holds if:

$$Y_{ij}(1,0)⫫M_{ij}(1)|M_{ij}(0)=0,{\mathbf{C}}_{ij}=\mathbf{c}\hspace{1em}\hspace{1em}\forall \mathbf{c}\in 𝒞\text{ and }\forall i,j$$

Assumption (4) coveys the idea that the distribution of the counterfactual Y_ij(1, M_ij (0)), which corresponds to Y_ij(1, 0) for never-takers and compliers, is the same for these two principal strata, conditioning on baseline covariates. This allows one to estimate the a priori counterfactual Y_ij (1, M_ij (0)) for compliers using the information on Y_ij(1, 0) provided by never-takers assigned to A_j = 1, for whom we observe Y_ij(1). This assumption is neither testable nor can find support in the data. If never-takers and compliers share the same conditional distribution of the potential outcome Y_ij (0, M_ij (0)), we could assume that it is also true when encouragement is set to the opposite condition.

Theorem 1

If assumption 4 holds, for compliers the net encouragement effect NEE⁰(0, 1, c), within levels of covariates, is given by:

$$NE{E}^0(0,1,\mathbf{c})=E[Y_{ij}(1)|S_{ij}=S^{00},{\mathbf{C}}_{ij}=\mathbf{c}]-E[Y_{ij}(0)|S_{ij}=S^{01},{\mathbf{C}}_{ij}=\mathbf{c}]$$

Proof

see supplemental materials.

Often assumption (4) cannot be supported, especially when the data do not provide evidence on the equality of the distribution of Y_ij(0) for never-takers and compliers, even within the same levels of covariates. For example in KAHS study never-takers and always-takers can be substantially different households. Therefore, we will provide an alternative assumption that might be more reasonable in some applications.

Assumption 5

Homogeneity of Mean Difference between Counterfactuals for Never-takers and Compliers

Homogeneity of the mean difference between counterfactuals across never-takers and compliers holds if:

$$\mathrm{E}[Y_{ij}(1,0)-Y_{ij}(0,0)|M_{ij}(0)=0,M_{ij}(1),{\mathbf{C}}_{ij}=\mathbf{c}]=\mathrm{E}[Y_{ij}(1,0)-Y_{ij}(0,0)|M_{ij}(0)=0,{\mathbf{C}}_{ij}=\mathbf{c}]\hspace{1em}\forall \mathbf{c}\in 𝒞$$

In words, it states that the average difference of potential outcomes under the two encouragement conditions and intervening to set the treatment receipt of each unit to 0, is the same for all those with M_ij (0) = 0, that is those who would not take the treatment if A_j were set to 0, i.e., never-takers and compliers, and is independent of the potential treatment receipt under the opposite encouragement status, M_ij (1).

In KAHS study this means that households that would not buy any new bed net without loans would have the same average effect of the offer of the program to their cluster on the reduction of risk of infection, if we intervened to keep their number of bed nets bought at follow-up fixed at 0, regardless of their behavior under the control condition. Given this assumption we are able to give the following theorem:

Theorem 2

If assumption 5 is satisfied, for compliers the net encouragement effect NEE⁰(0, 1, c), within levels of covariates, can be extrapolated from the dissociative causal effect for never-takers:

$$NE{E}^0(0,1,\mathbf{c})\equiv DCE(0,\mathbf{c})$$

Proof

see supplemental materials.

The effect of the encouragement is the same for never-takers and compliers, intervening to set M_ij to 0 or in other words to prevent any purchase of new bed nets. Assumption (5) allows then to estimate NEE⁰(0, 1, c) for compliers and hence NEE⁰(c) in the entire population.

Assumptions (4) and (5) provide the possibility of a generalization of the potential outcome Y_ij (1, M_ij (0)) or the net encouragement effect NEE⁰ from never-takers to compliers, as stated by the theorems 1 and 2. As a consequence, these assumptions also yield identification of the individual treatment mediated effect in the latter principal stratum, iTME¹(0, 1, c).

Corollary 1

If assumption (4) holds the individual treatment mediated effect for compliers iTME¹(0, 1, c), within levels of covariates, is given by:

$$iTM{E}^1(0,1,\mathbf{c})=PCE(0,1,\mathbf{c})-\left(\mathrm{E}\right[Y_{ij}(1)|S_{ij}=S^{00},{\mathbf{C}}_{ij}=\mathbf{c}]-\mathrm{E}[Y_{ij}(0)|S_{ij}=S^{01},{\mathbf{C}}_{ij}=\mathbf{c}\left]\right)$$

If assumption (5) holds the individual treatment mediated effect for compliers iTME¹(0, 1, c), within levels of covariates, is given by:

$$iTM{E}^1(0,1,\mathbf{c})=PCE(0,\mathit{1},\mathbf{c})-DCE(0,\mathbf{c})$$

6. Hierarchical Models for Cluster Interventions

In this section we describe the models used for our analysis: a model for the outcome and a model for the principal strata membership. Because of the cluster-level randomization the use of the hierarchical framework is needed as correlation among individuals arising from common environmental factors and even reciprocal influence cannot be ignored. In our setting, individuals living in the same community are likely to show resemblance not only in terms of outcomes, but also in terms of individual treatment uptake. Further, the level of resemblance in outcomes may vary across different individual strata. Correlation in cluster randomized trials with individual non-compliance has been intensively studied by Jo (2008), after Frangakis, Rubin & Zhou (2002), who were the first authors to accommodate in their analysis correlation in both outcome and non-compliance status. Here we extend the model framework used by Frangakis, Rubin & Zhou (2002) for a different outcome distribution.

Potential Outcome Model

Although the general framework applies to any outcome models, here we present the model used to analyze KAHS study. In our malaria example, the outcome of interest, Y_ij, is the proportion of malaria cases that household i in cluster j has experienced in the month prior to the follow-up interview. Therefore, we assume a relative binomial distribution for the potential outcomes of the form Y_ij(a)

$$Y_{ij}(a)|S_{ij},{\mathbf{C}}_{ij}~\frac{\mathit{\text{Bin}}(n_{ij},p_{ij})}{n_{ij}}$$ (6.1)

and we provide a hierarchical generalized linear model for the probability p_ij = p_ij (a, S_ij, C_ij), as a function of the encouragement A_j = a, the principal stratum S_ij and the vector of covariates C_ij :

$$g\left(p_{ij}\right(a,S_{ij},{\mathbf{C}}_{ij}\left)\right)={\mathit{\beta }}^{S_{ij}T}{\mathbf{Z}}_{ij}^{Yf}+{\mathit{b}}_j^T{\mathbf{Z}}_{ij}^{Yr}={\mathit{\beta }}_0^{S_{ij}T}{\mathbf{C}}_{ij}^{\prime }+{\mathit{\beta }}_1^{S_{ij}T}{\mathbf{C}}_{ij}^{\prime }a+b_{0j}+{\mathbf{b}}_{1j}^T{\mathbf{X}}_{ij}$$ (6.2)

$${\mathbf{b}}_j~N(\mathbf{0},{\mathrm{\Sigma }}_b)$$

g (·) is a link function, β^S_ij are the fixed effects for each principal stratum and b_j are the random effects, with variable vectors ${\mathbf{Z}}_{ij}^{Yf}=[1,{\mathbf{C}}_{ij},a,{\mathbf{C}}_{ij}a]$ and ${\mathbf{Z}}_{ij}^{Yr}=[1,{\mathbf{X}}_{ij}]$ respectively, allowing for random intercepts and random individual covariates slopes. We also assume that the two potential outcomes Y_ij(0) and Y_ij(1) are independent, given the covariates and strata membership.

Principal Strata Model

Principal strata membership can also be modeled by a hierarchical generalized linear model to take into account cluster correlation in individual treatment:

$$g\left(P\right(S_{ij}=S^{m_0{m}_1}|{\mathbf{C}}_{ij}))={\mathit{\alpha }}^T{\mathbf{Z}}_{ij}^{Sf}+{\mathbf{a}}_j^T{\mathbf{Z}}_{ij}^{Sr}={\mathit{\alpha }}^T{\mathbf{C}}_{ij}^{\prime }+a_{0j}+{\mathbf{a}}_{1j}^T{\mathbf{X}}_{ij}$$ (6.3)

$${\mathbf{a}}_j~N(\mathbf{0},{\mathrm{\Sigma }}_a)$$

where g (·) is the link function, α^S_ij are the fixed effects and a_j are the random effects, with variable vectors ${\mathbf{Z}}_{ij}^{Sf}=[1,{\mathbf{C}}_{ij}]$ and ${\mathbf{Z}}_{ij}^{Sr}=[1,{\mathbf{X}}_{ij}]$ respectively, assuming covariate C_ij to be predictors of strata membership.

Here we follow the approach used in Frangakis, Rubin & Zhou (2002) and Barnard et al. (2003), who modeled the strata membership using an Ordinal Probit model. In general in an Ordinal Probit model for an ordinal outcome with L categories the probability of belonging to a category lower than l is modeled as P(Y_i ≤ l)|C_ij) = Φ (α_l C_ij)), with l = 1, …, L − 1, so that the probability of belonging to the category l ends up being P(Y_i = l | C_ij) = (P(Y_i ≤ l + 1)| C_ij)) (1 − Φ (α_l C_ij))). The function Φ(·) is the standard normal cumulative distribution function.

According to this parametrization here we illustrate the Ordinal Probit Model for S_ij when monotonicity is assumed, so that we end up with three strata with two linked probit models, the first modeling membership in the never-taker stratum and the second modeling membership in the complier stratum conditional on exclusion from the never-taker stratum. In our setting of cluster-based interventions, we extend the above model to an Ordinal Mixed Probit Model, parameterized as:

$${\mathrm{\Psi }}_n({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a})=P(S_{ij}=S^{00})|{\mathbf{C}}_{ij})=1-\mathrm{\Phi }({\mathit{\alpha }}_n^T{Z}_{ij}^{Sf}+{\mathbf{a}}_{nj}^T{Z}_{ij}^{Sr})$$

$${\mathrm{\Psi }}_c({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a})=P(S_{ij}=S^{01})|{\mathbf{C}}_{ij})=(1-{\mathrm{\Psi }}_n({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a}))(1-\mathrm{\Phi }({\mathit{\alpha }}_c^T{Z}_{ij}^{Sf}+{\mathbf{a}}_{cj}^T{Z}_{ij}^{Sr}\left)\right)$$ (6.4)

$${\mathrm{\Psi }}_a({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a})=P(S_{ij}=S^{11})|{\mathbf{C}}_{ij})=1-{\mathrm{\Psi }}_n({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a})-{\mathrm{\Psi }}_c({\mathbf{C}}_{ij},\mathit{\alpha },\mathbf{a})$$

with α = (α_n, α_c) and a = (a_n = (a_n1, …, a_nJ), a_c = (a_c1, …, a_{c J})) and

$${\mathbf{a}}_{nj}~N(\mathbf{0},{\mathrm{\Sigma }}_{a_n})\hspace{1em}{\mathbf{a}}_{c_j}~N(\mathbf{0},{\mathrm{\Sigma }}_{a_c})$$

The above model has an equivalent formulation as a latent-variable model. In this formulation the two probit models are represented as arising from two underlying continuous random variables $S_{ij}^n$ and $S_{ij}^c$:

$$S_{ij}=\{\begin{array}{ccc}{S}^{00}\hfill & \text{if}\hfill & S_{ij}^n\equiv {\mathit{\alpha }}_n^T{Z}_{ij}^{Sf}+{\mathbf{a}}_{nj}^T{Z}_{ij}^{Sr}+V_{ij}\le 0\hfill \\ S^{01}\hfill & \text{if}\hfill & S_{ij}^n\ge 0\text{ and }{S}_{ij}^c\equiv {\mathit{\alpha }}_c^T{Z}_{ij}^{Sf}+{\mathbf{a}}_{cj}^T{Z}_{ij}^{Sr}+U_{ij}\le 0\hfill \\ S^{11}\hfill & \text{if}\hfill & S_{ij}^n\ge 0\text{ and }{S}_{ij}^c\ge 0\hfill \end{array}$$ (6.5)

where U_ij and V_ij are independently distributed as N(0, 1). The latter formulation is going to facilitate computation later.

7. Application to KAHS Study

The baseline covariates that a preliminary analysis has shown to be useful for predicting strata membership include: the number of household members (C_1ij), the maximum grade reached by any member of the households (C_2ij), the number of bed nets per sleeping space (C_3ij, labeled household baseline coverage), the number of sleeping spaces per household member (C_4ij), and finally the proportion of members that have been sick with malaria during the year prior to the baseline survey (C_5ij). We also included a neighbors’ characteristic, being the average number of bed nets per sleeping space owned at baseline by all the remaining households of the cluster (C_6ij, labeled neighborhood baseline coverage). Cluster covariates (V_j) are not considered.

As far as priors specification is concerned, priors’ hyper-parameters for the fixed effects of the principal strata model are set as follows: ${\mathit{\mu }}_{\alpha 0}^n={\mathit{\mu }}_{\alpha 0}^c=\mathbf{0},{\mathrm{\Lambda }}_{\alpha 0}^n={\mathrm{\Lambda }}_{\alpha 0}^c=10\mathrm{I}$. Furthermore, we argue that random intercepts suffice to explain compliance within cluster correlation, meaning that, while the overall principal strata distribution might vary across clusters, the extent to which compliance status for each household depend on baseline covariates is sufficiently constant. Accordingly, we set to zero every random slope, i.e. a_n1j = a_c1j = 0. As a consequence, random effects are assumed to follow a uni-dimensional normal distribution, $a_{n0j}|{\sigma }_{a_n}^2~N(0,{\sigma }_{a_n}^2)$ and $a_{c0j}|{\sigma }_{a_c}^2~N(0,{\sigma }_{a_c}^2)$, and the conjugate prior distribution of their variances reduces from inverse-Wishart to inverse-gamma, ${\sigma }_{a_n}^2~IG({\eta }_0^n,s_0^n)$ and ${\sigma }_{a_c}^2~IG({\eta }_0^c,s_0^c)$, where we set ${\eta }_0^n={\eta }_0^c=0.01$ and $s_0^n=s_0^c=0.01$.

As already said, we posit a binomial distribution for the potential outcomes, with probability p_ij(a, S_ij, C_ij) being a function of the principal stratum, the encouragement condition and baseline covariates, as modeled in (6.2). We adopt as the logit link g (·). In the outcome model we consider a subset of C_ij given by all the covariates used in the strata model excluding the number of household members. Moreover, we are particularly interested in probing the heterogeneity of the effect of the encouragement on malaria risk between different levels of household bed net coverage at baseline. Thus, we consider only the interaction term corresponding to the variable of interest, namely C_3ij a, while all the other interaction coefficients are set to zero: ${\beta }_{11}^{S_{ij}}={\beta }_{12}^{S_{ij}}={\beta }_{14}^{S_{ij}}={\beta }_{15}^{S_{ij}}={\beta }_{16}^{S_{ij}}=0$, ∀ S_ij ∈ {S⁰⁰, S⁰¹, S¹¹}. In addition, we let the coefficients for baseline covariates be the same across strata, with the exception for the covariate that is also present in the interaction term: ${\beta }_{0k}^{S^{00}}={\beta }_{0k}^{S^{01}}={\beta }_{0k}^{S^{11}}={\beta }_{0k}$, with k = 1, 2, 4, 5.

As with the principal strata model, between-cluster variation is taken into account by the inclusion of random intercepts, with the argument that the dependance of the outcome from covariates should not vary consistently across clusters and also that the small sample size does not enable to explore the variation of the effects between clusters. Random intercepts are also deemed constant for all principal strata: b_1j = 0, ∀ j. Finally, the model in (6.2) for the probability of the binomial potential outcome, can be rewritten as follows:

$$\mathit{\text{logit}}\left(p_{ij}\right(a,S_{ij},{\mathbf{C}}_{ij}\left)\right)={\beta }_{00}^{S_{ij}}+{\beta }_{02}{C}_{2_{ij}}+{\beta }_{03}^{S_{ij}}{C}_{3_{ij}}+{\beta }_{04}{C}_{4_{ij}}+{\beta }_{05}{C}_{5_{ij}}+{\beta }_{06}{C}_{6_{ij}}+{\beta }_{10}^{S_{ij}}a+{\beta }_{11}^{S_{ij}}{C}_{3_{ij}}a+b_{oj}$$ (7.1)

$$b_{0j}~N(0,{\sigma }_b^2)$$

We postulate a multivariate normal prior for β, β ~ N (μ_β0, Λ_β0), with hyper-parameters μ_β0 = 0 and Λ_β0 = 10 I, and an inverse-gamma prior distribution for the variance of the random intercept, ${\sigma }_b^2~IG({\eta }_0^b,s_0^b)$, setting ${\eta }_0^b=0.01$ and $s_0^b=0.01$.

Table 1 gives some basic information on our data and summary statistics of the baseline covariates, the intermediate variable and the outcome. As we can see, the randomization of the assignment leads to the baseline covariates being closely balanced in the two subgroups defined by the assignment. The lack of perfect balance for some of them is handled by covariate adjustment. In the intervention arm, 44% of the households did not buy new bed nets; these must be never-takers and the remaining buyers households must be either always-takers or compliers. Similarly in the control arm, 41% of the households did buy new bed nets after the baseline survey; these are always-takers and the remaining non-buyers households are either never-takers or compliers. Probabilities of belonging to each principal stratum are estimated to be 0.15, 0.44 and 0.41 for compliers, never-takers and always-takers respectively. These estimates are a result of the monotonicity assumption, which is arguably satisfied given that the presence of households, labeled as defiers, that would buy new bed nets at full prize and would not if offered loans, is implausible. The last row in table 1 provides an ITT analysis, indicating that the encouragement intervention - being the offer of loans to buy bed nets randomly assigned at cluster level to the 11 farmers participating in the study and belonging to the same settlement of villages - results in a 44.9%(= 0.0476/0.1060 × 100) reduction of the risk of contracting malaria. The between arms difference of −0.0479 in the mean proportion of malaria cases, among the households who do not buy new bed nets, suggests that the encouragement itself has a beneficial effect, regardless of the effect through the purchase of new nets. However this observed difference cannot be interpreted causally because of the different compliance types involved in such contrast, due to the intermediate variable not being randomized.

Table 1.

Summary statistics of the baseline covariates, the intermediate variable and the outcome.

		Control Assignment A = 0		Encouragement Assignment A = 1		Difference between assignments
Clusters		15		19		-	-
Households		161		195		-	-
Household Members C1		5.4660	(0.1640)	6.2205	(0.2103)	0.7547	(0.2631)
Education C2		5.7826	(0.3505)	6.4410	(0.3286)	0.6584	(0.4734)
Household Baseline Coverage C3		0.4569	(0.0914)	0.5646	(0.0425)	0.1076	(0.0992)
Sleeping Spaces per Member C4		0.4532	(0.1612)	0.4913	(0.0192)	0.0380	(0.0248)
Malaria Risk (Baseline) C5		0.3533	(0.0295)	0.3195	(0.0338)	−0.0338	(0.0442)
Neighborhood Baseline Coverage C6		0.4569	(0.0914)	0.5646	(0.0425)	0.1076	(0.0992)
Bed Net Purchase, (P (M | A)),
M = 0		0.5901	(0.0497)	0.4410	(0.0355)	−0.1490	(0.0600)
M = 1		0.4099	(0.0497)	0.5590	(0.0355)	0.1490	(0.0600)
Malaria Risk (Follow-up) (E[Y | A, M])
among bed nets non-buyers (M = 0)		0.1213	(0.0295)	0.0734	(0.0171)	−0.0479	(0.0336)
among bed nets buyers (M = 1)		0.0840	(0.0199)	0.0466	(0.0181)	−0.0374	(0.0228)
All		0.1060	(0.0215)	0.0584	(0.0111)	−0.0476	(0.0241)

Estimates of population means with their standard errors (in parenthesis), based on the method of moments, are reported. The second and third blocks of rows concern the intermediate variable and the outcome. Due to their bernoulli and binomial distributions, estimated means are also estimates of the probability of buying new bed nets and the probability of infection, respectively.

Furthermore, we can make speculations on the mechanisms that in KAHS might give rise to the effect of the agricultural loan program for different types of households. Since no additional awareness campaign and no community-based interventions were provided to treated clusters, the effect for never-takers, i.e., DCE(0, c), can be hypothesized to be mostly due to spillovers of the purchase of new bed nets by other households belonging to the same cluster. Conversely, we can assume that for always-takers spillover effects, if any, can explain only a small part of the effect of the encouragement intervention, whereas most of it likely follows from a greater number of bed nets they would buy under the loan program, thanks to the subsidized price and the deferred payment. This non-negative dissociative effect DCE(1, c) is due to the particular choice of the binary treatment variable that only distinguishes the purchase of zero versus at least one new new bed net between baseline and follow-up (M_ij = 1 if household i in cluster j has bought at least one more bed net and M_ij = 0 if no purchase has been made). As opposed to always-takers, complier households would not buy any new bed net if not encouraged by the loan program. Therefore, for compliers the intervention is presumed to reduce malaria by prompting them to get new bed nets they would not buy otherwise. This effect would be the individual treatment mediated effect, i.e. iTME¹(0, 1, c). Moreover, the purchase of bed nets in neighboring households can, in principle, also benefit complier households. This effect of the clustered intervention through spillovers would explain part, if not all, the net encouragement effects for compliers, i.e. NEE⁰(0, 1, c). These speculations only affect the interpretation of the estimated effects but do not alter the analysis.

In order to disentangle net encouragement effects and individual treatment mediated effects for compliers, we can argue that in the KAHS study we cannot rely on assumption (4) of stochastic homogeneity of counterfactuals. Indeed, in such a study concerning malaria, prevention behavior is difficult to predict by observed characteristics and the risk of infection from malaria depends on many different observed and unobserved factors. Therefore, we believe that, for each household, the distribution of the potential proportion of malaria cases under the loan program assigned to the whole cluster and intervening to set M_ij to 0 is arguably not shared by never-takers and compliers. On the contrary, it can be more reasonable to assume homogeneity of the mean difference between counterfactuals, as stated by assumption (5), which translates into homogeneity between never-takers and compliers of the effect of the clustered encouragement when the household could not make any new purchase of bed nets (theorem 2). On the basis of our previous hypotheses, this means that never-takers and compliers would benefit from the increased number of bed nets in the surroundings in the same way, that is, they would share the same effect of the program assigned in their cluster mainly through spillovers, if their bed net coverage were kept unchanged.

In the supplemental material we provide details on how to exploit the homogeneity assumptions to make inference on the two causal mechanisms making use of a Bayesian imputation approach. In this approach, imputation of the required potential outcomes follows from these assumptions in a straightforward way. The adoption of one of the two homogeneity assumptions implies different steps in the final Bayesian imputation procedure.

7.1 Results

We will first focus on the characterization of principal strata. In table 2 we report posterior means and 95% intervals for the coefficients of the two latent variable models in (6.5). We can see that the only covariates that really matter in the prediction of compliance status are those related to the household baseline coverage and household living space, C_3ij and C_4ij. In particular, there is evidence that the probability of being a never-taker increases with the number of bed nets per sleeping space (mean and 95% interval for α_n4: −2.227 [−2.775, −2.038]) and so does the probability of being a complier (mean and 95% interval for α_c4: −2.289 [−4.540,−1.634]), as compared with being an always-taker. The number of sleeping spaces per household member has a similar pattern since this covariate gives information on room sharing in the house and thus the need of bed nets per sleeping space: households with higher coverage are less likely to buy new bed nets. Another expected result is that neighborhood baseline coverage reduces the probability of being a never-taker, probably because of a peer influence.

Table 2.

Estimated Parameters for Principal Strata Model (6.5)

	Sn Model		Sc Model
	Mean	95% Interval	Mean	95% Interval
α coefficients
Household Members C1	0.061	[−0.014, 0.087]	0.016	[−0.161, 0.073]
Education C2	0.053	[−0.008, 0.074]	−0.090	[−0.410, 0.013]
Household Baseline Coverage C3	−2.227	[−2.775,−2.038]	−2.289	[−4.540,−1.634]
Sleeping Spaces per Member C4	−0.856	[−1.709,−0.568]	−0.691	[−3.270, 0.184]
Malaria Risk (Baseline) C5	0.080	[−0.520, 0.280]	1.260	[−0.350, 1.827]
Neighborhood Baseline Coverage C6	0.938	[0.149, 1.210]	1.032	[−1.347, 1.805]
Random Intercept Variance, ${\sigma }_a^2$	0.096	[0.017, 0.125]	1.588	[0.035, 1.597]

We have also derived estimates of the sample mean of the covariates within each principal stratum, i.e. C̅_hm0m1 = ∑_{i, j: Sij=S^m0m1} C_hij /|S^m₀m₁ |, ∀k ∈ {1, …, 5} and ∀m₀, m₁ ∈ {0, 1}. Posterior distributions of the sample means are averaged over all possible vectors of S and θ from their joint posterior distribution. Means and 95% intervals of these distributions are shown in table 3. Results confirm previous findings, that is, never-takers have on average higher coverage and, on the contrary, always-takers are those with a smaller number of bed nets per sleeping spaces and also a greater number of members per room. In addition, there is evidence that compliers in our study have a greater highest grade in the family, whereas always-takers have on average a greater proportion of malaria cases in the year prior to the baseline survey. The latter result, together with the low household coverage, can explain most of the compliance behavior of the always-takers. Finally, the mean of neighborhood baseline coverage within principal strata, averaged over the remaining covariates, seems to be lower for always-takers with no evidence of a difference between never-takers and compliers.

Table 3.

Distribution of Covariates within Principal Strata

	Compliers		Never-Takers		Always-Takers
	Mean	95% Interval	Mean	95% Interval	Mean	95% Interval
Household Members C1	6.409	[5.483,7.296]	5.443	[5.376,5.527]	6.192	[5.956,6.438]
Education C2	7.212	[6.247,8.228]	5.969	[5.873,6.110]	5.972	[5.678,6.227]
Household Baseline Coverage C3	0.483	[0.338,0.623]	0.786	[0.760,0.827]	0.218	[0.169,0.270]
Sleeping Spaces per Member C4	0.494	[0.422,0.562]	0.502	[0.491,0.509]	0.434	[0.415,0.455]
Malaria Risk (Baseline) C5	0.267	[0.179,0.358]	0.297	[0.279,0.307]	0.400	[0.379,0.425]
Neighborhood Baseline Coverage C6	0.529	[0.446,0.612]	0.545	[0.525,0.564]	0.476	[0.450,0.496]

The estimated variance of the random intercept a_n0j included in the model for the conditional probability of being a never-taker versus being an always-taker or a complier (model for $S_{ij}^n$) is estimated to be 0.096 (95% quintiles: [0.017, 0.125]) reflecting in an intra-class correlation of 0.088 (Table 2). Similarly the estimated variance of the random intercept a_c0j of the model for $S_{ij}^c$, conditional on not being never-takers, is estimated to be 1.588 (95% quintiles: [0.035, 1.597]), reflecting in an intra-class correlation of 0.614: the proportion of never-takers does not seem to differ substantially across clusters conditional on covariates, whereas the proportion of always-takers and compliers does.

The left column of Table 4 shows posterior principal strata rates, in the overall population and within three coverage categories defined by household baseline coverage:

$${\tilde{C}}_{4_{ij}}=\left\{\mathit{\text{Low Coverage }}\right(\text{if }{C}_{4_{ij}}\le 0.4),\mathit{\text{Medium Coverage }}(\text{if }0.4<C_{4_{ij}}\le 0.8),\mathit{\text{High Coverage }}(\text{if }{C}_{4_{ij}}>0.8\left)\right\}$$

The overall probabilities of compliance status, given by the Bayesian procedure, approximately match the aforementioned method of moments estimates.

Table 4.

Principal Strata Rates and Malaria Rates by principal strata

		Principal Strata Rates P(Sij = Sm0m1)		Malaria Rates Ȳ(0)
Principal Strata		Mean (SD)	95% Interval	Mean (SD)	95% Interval
NEVER-TAKERS
Low Coverage		0.198 (0.023)	[0.147,0.231]	0.042 (0.019)	[0.006,0.082]
Medium Coverage		0.520 (0.024)	[0.459,0.561]	0.051 (0.022)	[0.013,0.103]
High Coverage		0.788 (0.016)	[0.745,0.804]	0.074 (0.036)	[0.025,0.167]
All		0.456 (0.017)	[0.418,0.483]	0.060 (0.026)	[0.020,0.123]
ALWAYS-TAKERS
Low Coverage		0.671 (0.031)	[0.603,0.724]	0.084 (0.011)	[0.065,0.109]
Medium Coverage		0.283 (0.044)	[0.204,0.367]	0.092 (0.053)	[0.029,0.221]
High Coverage		0.094 (0.031)	[0.039,0.157]	0.063 (0.075)	[0.000,0.275]
All		0.399 (0.027)	[0.345,0.452]	0.084 (0.020)	[0.057,0.135]
COMPLIERS
Low Coverage		0.130 (0.045)	[0.051,0.224]	0.348 (0.110)	[0.177,0.574]
Medium Coverage		0.197 (0.058)	[0.082,0.306]	0.331 (0.117)	[0.165,0.592]
High Coverage		0.118 (0.039)	[0.039,0.186]	0.290 (0.131)	[0.123,0.633]
All		0.145 (0.038)	[0.073,0.219]	0.321 (0.099)	[0.179,0.545]

Reported results are means, standard deviations and 95%intervals of the posterior distribution of strata membership rates, and the posterior predictive distribution of malaria rates by principal strata under encouragement status A_j = 0, i.e. Ȳ (0). Both distributions are averaged over C_ij (or just over C_1ij, C_2ij, C_4ij and C_5ij when results are presented within household baseline coverage categories C̃_4ij), the clusters and θ,

A deeper characterization of principal strata is provided by the distribution of potential outcomes. The right column of Table 4 summarizes the predictive posterior distribution of malaria rates without encouragement, by principal strata and by coverage categories C̃_4ij. Several important results merit attention here. First, we can see that, among never-takers, there is no evidence of a reduction of risk with an increase of coverage. This unexpected result must be due to other unmeasured factors affecting the relationship between bed nets coverage at baseline and malaria risk without encouragement, as well as compliance status. For example, never-takers with low coverage at baseline are likely to be households at lower risk, because of housing conditions or other protective behaviors. For always-takers 95% intervals get wider as coverage augments due to the small proportion of always-takers in higher levels, hence no definite conclusion can be drawn. For compliers, posterior means seem to decrease with e C̃_4ij, but still intervals cover zero making this pattern consistent to random fluctuation. Secondly, we compare principal strata: at all coverage levels, compliers are those households who would have a considerably higher risk of malaria infection if not encouraged, with an overall mean risk of 32.1% against 6% for never-taker and 8.4% for always-takers. This result may be somewhat surprising, but we can give some intuitive explanations. For never-takers, the low risk of contracting malaria compared to the other principal strata might be due to better housing conditions, as well as a greater use, at least in 2010, of other preventive measures such as windows screens and preventive behaviors such as keeping doors and windows closed at night, being indoors after sunset or removing possible breeding sites in the house. For always-takers, the household coverage at follow-up would increase compared to baseline, even without the encouragement. This can be one of the reasons for their low risk, probably together with the take-up of similar preventive behaviors to the ones used by never-takers. On the contrary, if not encouraged compliers seem to be the sub-population most at risk of malaria. The reason might be, besides the use of less preventive measures and more risky behaviors, the presence of higher risk factors, such as livestock animals, co-morbidities, pregnancies, house damage, as well as possibly, for those with medium high coverage, old bed nets in bad physical integrity. The offer of loans to buy bed nets might make compliers more aware of their risk.

In any case, the different mean potential outcome under control encouragement between principal strata supports our hypothesis of assumption 4 of partial stochastic homogeneity of counterfactuals being implausible.

Table 5 concerns the estimated effects defined in section 4.1, that is principal causal effects PCE, net encouragement effects NEE⁰ and individual treatment effects iTME¹, by principal strata and by coverage levels e C̃_4ij. Estimates are based on imputations from the predictive posterior distributions of potential outcomes (see supplemental materials). Results are based on 45000 iterations, combining three chains, each run for 25000 iterations, with a burn-in of 10000 iterations. To check for convergence, for each effect we computed the potential scale reduction factor (Gelman and Rubin, 1992), giving a maximum value of 1.04, suggesting no evidence against convergence (details available from the authors upon request).

Table 5.

Estimated Effects within Principal Strata and by coverage levels

		NEE0			iTME1			PCE
Principal Strata		Mean	Median (SD)	95% Interval	Mean	Median (SD)	95% Interval	Mean	Median (SD)	95% Interval
NEVER-TAKERS		DCE(0)
Low Coverage		0.025	0.023 (0.039)	[−0.043, 0.114]	–			0.025	0.023 (0.039)	[−0.043, 0.114]
Medium Coverage		0.011	0.014 (0.034)	[−0.058, 0.079]	–			0.011	0.014 (0.034)	[−0.058, 0.079]
High Coverage		0.010	0.017 (0.053)	[−0.106, 0.118]	–			0.010	0.017 (0.053)	[−0.106, 0.118]
All		0.014	0.018 (0.041)	[−0.072, 0.097]	–			0.014	0.018 (0.041)	[−0.072, 0.097]
ALWAYS-TAKERS		DCE(1)
Low Coverage		−0.062	−0.064 (0.017)	[−0.095,−0.026]	–			−0.062	−0.064 (0.017)	[−0.095,−0.026]
Medium Coverage		−0.066	−0.061 (0.048)	[−0.186,−0.002]	–			−0.066	−0.061 (0.048)	[−0.186,−0.002]
High Coverage		−0.050	−0.034 (0.073)	[−0.258, 0.033]	–			−0.050	−0.034 (0.073)	[−0.258, 0.033]
All		−0.062	−0.062 (0.023)	[−0.118,−0.023]	–			−0.062	−0.062 (0.023)	[−0.118,−0.023]
COMPLIERS					CACE1
Low Coverage		0.014	0.014 (0.044)	[−0.073, 0.104]	−0.208	−0.200 (0.132)	[−0.470,0.027]	−0.194	−0.183 (0.128)	[−0.448, 0.029]
Medium Coverage		0.015	0.019 (0.050)	[−0.091, 0.110]	−0.170	−0.157 (0.144)	[−0.473, 0.078]	−0.155	−0.138 (0.141)	[−0.456, 0.079]
High Coverage		0.015	0.022 (0.064)	[−0.125, 0.138]	−0.191	−0.175 (0.157)	[−0.553, 0.071]	−0.176	−0.152 (0.147)	[−0.530, 0.047]
All		0.014	0.018 (0.041)	[−0.072, 0.091]	−0.186	−0.178 (0.125)	[−0.452,0.030]	−0.172	−0.159 (0.123)	[−0.435, 0.032]
ALL								ITT
Low Coverage		−0.028	−0.026 (0.016)	[−0.062,−0.006]	−0.021	−0.019 (0.015)	[−0.058, 0.002]	−0.050	−0.049 (0.023)	[−0.097,−0.014]
Medium Coverage		−0.006	−0.004 (0.024)	[−0.059, 0.043]	−0.023	−0.017 (0.024)	[−0.082, 0.011]	−0.030	−0.022 (0.030)	[−0.100, 0.019]
High Coverage		0.007	−0.005 (0.041)	[−0.078, 0.097]	−0.016	−0.012 (0.016)	[−0.055, 0.005]	−0.009	−0.007 (0.038)	[−0.092, 0.070]
All		−0.016	−0.014 (0.026)	[−0.072, 0.034]	−0.026	−0.024 (0.018)	[−0.068, 0.004]	−0.042	−0.042 (0.027)	[−0.100, 0.008]

Means, medians, standard deviations and 95% intervals of the posterior distribution of net encouragement effects NEE⁰, individual treatment mediated effect iTME¹ and principal causal effects, are presented by principal strata and household baseline coverage categories C̃_4ij. The last block of rows concerns the estimated effect in the whole population.

Consider principal causal effects, presented in the last block of columns. The estimated PCE for compliers is on average a reduction of malaria risk of 17.2% (posterior mean), with similar estimates at every level of household baseline coverage. As expected, this total effect is much larger than PCEs in the other principal strata, being PCE(0, 1, c) the sum of NEE⁰(0, 1, c) and iTME¹(0, 1, c), that is, both the effect of the loan program both through spillovers and through the purchase of bed nets by the household itself. The estimated PCE for always-takers, i.e. DCE(1, c) is on average a reduction of the risk of infection of 6.2% (posterior mean), which is mainly due to the increased number of bed nets bought under the program. 95% credible intervals provide strong evidence of a beneficial effect of the encouragement for both compliers and always-takers.

For never-takers, instead, we find a negligible effect of the encouragement, i.e. DCE(0), for all levels of coverage. The proportion of malaria cases at baseline and potential proportion under control encouragement have not suggested lack of knowledge and awareness of malaria for this subpopulation, but, on the contrary, never-takers are likely the most aware of preventive measures. As said earlier, we can argue that for this principal stratum there is little effect of the encouragement itself, such as an increase in the usage of old bed nets or the undertaking of other measures, thereby suggesting no evidence of spillover effects at any coverage level, at least for never-takers.

The overall ITT, given by the average of the three principal causal effects, is estimated as a decrease in the risk of malaria of 4.2% (95% interval: [−10%, 0.8%]), which approximates the ITT estimated directly from the observed data. Note that 95% posterior intervals at medium and high coverage are wider and include zero making the results consistent with random fluctuation. This is due to the high proportion of never-takers in these categories. Therefore, all the effect of the encouragement for this principal stratum would be through the purchase of new bed nets.

When it comes to disentangling the effects for compliers, iTME¹(0, 1) is estimated by the posterior mean as a reduction of 18.6% (95% interval: [−45.2%, 3.0%]) whereas NEE⁰(0, 1) as a minimal increase with high uncertainty (posterior mean: 1.4%; 95% interval: [−7.2%, 9.1%]). The individual treatment effect for compliers is equivalent to the average effect of the purchase of at least one bed net, i.e. CACE¹.

Average net encouragement effects in the whole population are beneficial with strong evidence only within the low coverage category with a posterior mean of −2.8%. Finally, by multiplying iTME¹(0, 1) by the proportion of compliers, we obtain an estimate of the individual treatment effect in the population given by −2.6% (95% interval: [−6.8%,0.4%]).

All these results rely on both structural and modeling assumptions that were laid out in the previous sections. While some structural assumptions, such as cluster-level SUTVA (assumption 1) and unconfoundedness of the cluster encouragement assignment (assumption 3), plausibly hold by design, monotonicity (assumption 2) and the homogeneity assumptions (assumption 4 or 5) have been invoked based on subject-matter knowledge and are not directly testable from the data. However, monotonicity has a testable implication, which we have verified in our study: monotonicity is not falsified in our data because the number of units who buy new bed nets is significantly greater in the treatment arm than in the control arm. On the other hand both homogeneity assumptions involve a-priori counterfactuals which are never observed in this experiment on any subject and, therefore, do not generate any testable implications. The plausibility can only be judged either by expert knowledge or by some related evidence in the data. For example, the fact that never-takers and compliers are observed to have the same conditional distribution of the potential outcomes Y_ij(0, M_ij (0)) can be viewed as a support of the assumption that they might also share the distribution of Y_ij(1, M_ij (0)) (assumption 4). Also, because homogeneity assumptions are assumptions on a prior-counterfactuals, usual sensitivity analyses cannot be conducted for them, as any deviation from homogeneity would change the estimate of the net encouragement effect for compliers by exactly that same amount.

As far as modeling assumptions are concerned, in order to assess the sensitivity of posterior inference to the specification of the prior distribution, we have derived the distribution of our causal estimands using only the prior distribution over parameters. None of these distributions appear particularly informative for the causal estimands, thereby suggesting that the precision of the posterior distribution is mainly driven by the information in the data (Frangakis, Rubin & Zhou, 2002).

Regarding the models for the outcomes and the principal strata membership, the relative binomial distribution for the proportion of malaria cases and the linked probit model appear to be a reasonably flexible choice. As a model fit diagnostic, we have compared quantities that can be directly estimated from the data with their estimates derived from the estimated model. For example, the global intent-to-treat effect, estimated from the sample statistics, is equivalent to its estimate derived as a weighted average of principal stratum-specific effects (see Tables 1 and 5). Overall model fit could also be assessed using posterior predictive checks, which are however beyond the scope of the paper.

8. Discussion

In this article we provide a framework based on principal stratification approach to investigate the different mechanisms elicited in cluster encouragement designs. The core of this work concerns the proposal of new homogeneity assumptions allowing one to disentangle two different effects for the subpopulation of compliers, under violation of sequential ignorability. Even if we could assume sequential ignorability, which is stronger than our homogeneity assumptions, the characterization of principal strata and the estimation of causal mechanisms for different types of individuals is still an advantage of our proposed methodology over classical mediation analysis.

Principal causal effects themselves provide useful information on how encouragement has an impact on the outcome, within different subpopulations types defined by compliance behavior. Our analysis of the KAHS study, gives evidence that for those households who would buy new bed nets only if agricultural loans were offered, the compliers, the offer of loans to their village reduces the risk of contracting malaria. It also suggests that those who would proceed anyway with the purchase of bed nets, the always-takers, benefit from the loan program, most likely through an increase in the number of bed nets purchased due to the subsidized prize. On the contrary, it shows nonsignificant effect for never-takers, that is for those who would not buy new bed nets regardless of the encouragement. Consequently there is no evidence of spillover effects from the increased number of bed nets in the cluster, due to the encouragement, at least for this subpopulation. It might be the case that the potential beneficial effect of the bed nets in other households killing mosquitos is offset by the effect of diverting mosquitos to households without bed nets. The slightly detrimental effect for this subpopulation, especially with low coverage, even if intervals are too wide to draw definite conclusions, suggests the importance of investigating spillover effects in large scale programs.

Furthermore, the analysis of compliance status provided by the principal stratification framework, compared with simple ITT analysis, gives insight into the extent to which encouragement enhances the treatment uptake, how different types within the population react to the encouragement and what are the characteristics of individuals that encouragement is able to reach. KAHS program evaluation has provided an interesting case study in which principal strata differ substantially by their potential risk under control intervention. Specifically, compliers would have much higher risk of infection. This analysis shows how the loan program was able to reach the subpopulation most at risk and more in need to be prompted to take on better prevention measures. This characterization of principal strata can also help us understand whether and which homogeneity assumption is more plausible to untie the mediated and non-mediated effects among compliers.

A further advantage of our formalization of identifying homogeneity assumptions is the flexibility of specification. In fact, although we have focused on a particular case that is suitable for the application under study, in the supplemental material we provide more general homogeneity assumptions. Each specific assumption enables the identification of a combination of the two effects NEE^ã and iTME^1−ã, with ã = 0 or ã = 1, for always-takers, never-takers and compliers or defiers. The choice about which particular homogeneity assumption holds has to be determined on a case-by-case basis, with the help of subject matter knowledge and comparison of principal strata in terms of covariates and potential outcomes. In our application, we rely on homogeneity of the net encouragement treatment effect between never-takers and compliers, conditional on covariates, and found no evidence of a net encouragement effect among compliers, at any coverage level. Therefore, spillover effects are negligible for both never-takers and compliers and most of the effect of the clustered loan program for compliers is achieved by making these households buy new bed nets. This conclusion is important in that is shows how the sole purchase of few bed nets in a household at high risk can make a real difference. Hopefully, if the loan program were offered to more farmers in each cluster, an increased coverage in the community would exponentially reduce malaria through beneficial spillovers. This study does not allow us to assess this hypothesis, arguably because of the small number of beneficiaries.

Final results suggest that the impact of the encouragement is mostly driven by enhancing the purchase of bed nets in that 15% of population that otherwise would have a high risk of infection and would not prioritize prevention, the compliers, but almost as much is given by the effect due to the subsidized price and deferred payment through the increased number of new bed nets among the alway-takers, who constitute 41% of the population.

Since a negligible effect has been found among never-takers, if resources were limited, baseline information were already available and the offer of the loan program had a cost itself even if subsidies were not used (e.g. mail service, door-to-door visits, etc.), we may want to exclude this subpopulation from the encouragement program. The lack of knowledge of strata membership would force one to exclude those units with higher probability of being never-takers, i.e., those with higher baseline coverage. In addition, the observed heterogeneity in malaria risk highlights the need, in the design phase, of a detailed characterization of behavioral, socio-economic and environmental risk factors of the target population in order to select appropriate suites of interventions.

Our analysis of KAHS has several limitations. First, the choice of a binary intermediate variable, although it sheds light on a well-defined principal stratification of the population, it does not use information on actual number of bed nets. A continuous intermediate variable could also be handled in the principal stratification framework (Jin and Rubin 2008; Bartolucci and Grilli 2011; Schwartz et al. 2011), and homogeneity assumptions could be defined accordingly. Second, homogeneity of spillover effects that these assumptions require can be problematic if principal strata are highly clustered, due to mechanisms such as homophily or peer influence in the compliance behavior. Future works could focus on the estimation of spillover effects accounting for a differential distribution of potential values of the intermediate variable in the neighborhood. In any case, in this article we have emphasized the arguments that can be made in favor or against homogeneity assumptions in a challenging application with possible spillover effects and the presence of important latent features that make the distribution of potential outcomes differ substantially across principal strata. In many applications, the validity of homogeneity assumptions may be less controversial.