Sensitivity analysis for principal ignorability violation in estimating complier and noncomplier average causal effects

Abstract

An important strategy for identifying principal causal effects (popular estimands in settings with noncompliance) is to invoke the principal ignorability (PI) assumption. As PI is untestable, it is important to gauge how sensitive effect estimates are to its violation. We focus on this task for the common one-sided noncompliance setting where there are two principal strata, compliers and noncompliers. Under PI, compliers and noncompliers share the same outcome-mean-given-covariates function under the control condition. For sensitivity analysis, we allow this function to differ between compliers and noncompliers in several ways, indexed by an odds ratio, a generalized odds ratio, a mean ratio, or a standardized mean difference sensitivity parameter. We tailor sensitivity analysis techniques (with any sensitivity parameter choice) to several types of PI-based main analysis methods, including outcome regression, influence function (IF) based and weighting methods. We discuss range selection for the sensitivity parameter. We illustrate the sensitivity analyses with several outcome types from the JOBS II study. This application estimates nuisance functions parametrically – for simplicity and accessibility. In addition, we establish rate conditions on nonparametric nuisance estimation for IF-based estimators to be asymptotically normal – with a view to inform nonparametric inference.

1 |. INTRODUCTION

The study of causal effects of a treatment is often complicated by noncompliance. The principal stratification framework¹ defines types (principal strata) of study participants based on their potential compliance to treatment conditions. In the one-sided noncompliance setting where individuals in the control condition do not have access to the active treatment, there are two principal strata: compliers, who would take the treatment if offered, and noncompliers, who would not. In the two-sided noncompliance setting where all individuals (assigned to either treatment or control) can access the treatment, there are four principal strata, often known as compliers, always-takers, never-takers, and defiers. Principal causal effects are effects of treatment assignment within each stratum, $\mathrm{E}\left[Y_1-Y_0\mid C\right]$, where $Y_1$ and $Y_0$ are potential outcomes² under assignment of active treatment and of control, respectively, and C denotes principal stratum. Of common interest is the complier average causal effect (CACE), but other principal causal effects may also be of interest^3,4.

This paper focuses on one-sided noncompliance, which is common in studies where the treatment is designed and implemented by the study and is not otherwise available, e.g., job search training for unemployed workers⁵, volunteering program for the elderly⁶, or weight management for people with mental illness⁷. We will briefly comment on the two-sided non-compliance case in the Discussion section.

The challenge in identifying principal causal effects is that principal stratum membership $C$ is only partially observed; with one-sided noncompliance $C$ is not observed in the control condition. Effect identification thus requires untestable assumptions. One such assumption is exclusion restriction⁸ (ER), which posits that treatment assignment does not affect the outcome other than through its effect on treatment received. This means there is no effect on noncompliers, and effects on compliers explain the full effect of treatment assignment. ER is not suitable if treatment receipt is not strictly binary, i.e., noncompliers are exposed to some active ingredients in the treatment arm^9,10. This case may arise when an intervention includes several components, and only a major one is used to define compliance. It may also arise due to dichotomization, e.g., only people who attend more than a certain number of treatment sessions are classified as compliers⁶. ER may also not hold if there are compensating behaviors or psychological effects due to being assigned to one condition as opposed to the other¹¹.

Another identification strategy does not restrict the noncomplier effect to zero, but instead invokes the principal ignorability (PI) assumption^12,11,13. This assumption posits that, conditional on a set of pre-treatment-assignment covariates $X$, the potential outcome under control $Y_0$ is independent (or mean-independent) of principal stratum $C$, i.e., compliers and noncompliers share the same conditional $Y_0$ distribution (or mean function). PI may be appealing for studies with rich baseline covariate data. As randomized trials and cohort studies tend to collect a lot of covariate data, one might hope that the covariates account for a substantial part of the dependence between $Y_0$ and $C$. On the other hand, most studies are not designed with noncompliance in mind, and thus not much attention is paid to measuring covariates that predict compliance type to render $C$ and $Y_0$ independent, which means PI may be violated.

1.1 |. Our contribution

In this paper we focus on the PI assumption. Specifically, we develop methods to evaluate the robustness of the estimated principal causal effects to violation of PI, in the one-sided noncompliance setting. We introduce several sensitivity parameterizations representing how (within levels of $X$) the mean of $Y_0$ differs between compliers and noncompliers. These are indexed by an odds ratio, generalized odds ratio, mean ratio, or standardized mean difference, suitable for use with different outcome types. In addition, we tailor sensitivity analysis techniques for pairing with a range of estimation methods that may be used for the PI-based main analysis, including weighting, outcome regression and influence function based estimation.

We illustrate the proposed sensivity analysis methods using the JOBS II Intervention Study⁵, where unemployed workers were randomized to receive either a week-long training program to promote mental health and provide job search skills (treatment) or a booklet with job search tips (control). Just over half of those randomized to treatment actually attended the training, resulting in a setting with compliers and noncompliers. JOBS II has been used by authors investigating different aspects of principal stratification, e.g., identification and estimation under PI^12,11, alternative identification assumptions¹⁴, bias due to failed assumptions¹⁵, and noncompliance combined with outcome missingness¹⁶. For our purpose, JOBS II is an interesting example for two reasons: (i) the study paid attention to the issue of noncompliance and collected baseline data on workers’ motivation to participate in a hypothetical training program on job search skills, making this a prime case for invoking PI; and (ii) the study collected outcomes of several types (binary, continuous and bounded) to which the methods we propose are relevant.

1.2 |. Related work

To our knowledge, two methods have been proposed to assess sensitivity of effect estimates to PI violation. The method used in Ding and Lu (2017)¹³ is the closest to, and inspired, our work. In the one-sided noncompliance context, this method allows the mean of $Y_0$ given $X$ to differ between compliers and noncompliers by a ratio that serves as the sensitivity parameter, and estimates effects under each value of the sensitivity parameter by modifying a PI-based weighting estimator. The application was with a binary outcome, flu-related hospitalization. A drawback is that with a binary outcome this mean ratio parameter may yield predictions greater than 1. This motivated our expansion of the range of sensitivity parameterizations to accommodate different outcome types. Also, we consider sensitivity analysis techniques pairing with different types of PI-based estimators, not just the weighting estimator. The second sensitivity analysis method is that of Wang et al. (2023)¹⁷ for survival outcomes, which imputes unobserved $C$ and $Y_0$ under a parametric model containing a hazard ratio sensitivity parameter. This work differs from our approach in that it relies on this parametric model for identification, whereas we make explicit the assumption required for identification and then use modeling only for estimation. We also avoid refitting models for every value of the sensitivity parameter.

There are methods to assess sensitivity of principal causal effect estimates to violation of assumptions other than PI, such as treatment assignment ignorability^18,19 and ER²⁰. These are not our current focus.

To discuss sensitivity analysis, we will need to start with a description of PI-based estimation. While PI-based methods have been discussed in the literature, it has been in settings that are somewhat different, e.g., randomized treatment assignment^15,11,13 (which we do not require), a qualitatively different assumption¹⁴, or two-sided rather than one-sided noncompliance²¹. The PI-based estimators we list in this paper share certain features (e.g., principal score weighting, multiple robustness) with these earlier works, but are based on results for the current setting.

The paper proceeds as follows. Section 2 presents the setting, the estimands, and identification under PI. Section 3 introduces three types of PI-based estimators to be handled with different sensitivity analysis techniques. Sections 4 and 5 present sensitivity analysis using ratio-type and difference-type sensitivity parameters, respectively, and address each of the three estimator types. Section 6 covers other topics relevant to the sensitivity analyses. Section 7 analyzes JOBS II data. Section 8 closes with a discussion. Proofs are provided in the Web Appendices. Code is provided in the R-package PIsens available at https://github.com/trangnguyen74/PIsens.

2 |. SETTING, ESTIMANDS, AND PI-BASED IDENTIFICATION

2.1 |. Setting, estimands, and standard assumptions

Let $Z$ denote treatment assignment (1 for treatment, 0 for control), $Y$ denote the observed outcome, $Y_z$ the potential outcome had treatment $z$ been assigned $(z=0,1)$, and $X$ denote baseline covariates. Let $S$ be a binary variable indicating whether the person actually receives the treatment $(S=1)$ or not $(S=0)$. (More generally, $S$ can be any post-treatment variable of interest ^6,22,4 The principal stratification framework¹ defines subpopulations (aka principal strata, denoted by $C$) based on $S_1$ and $S_0$, the potential values of $S$ under assignment to treatment and to control. In the one-sided compliance setting, $S_0=0$, so only $S_1$ matters. Hence $C$ coincides with $S_1$ and there are two principal strata: compliers $(C=1)$ who would and noncompliers $(C=0)$ who would not take the treatment, if offered the treatment. The “full” data for an individual are $\left(X,Z,C,Y_1,Y_0\right)$; the observed data are $O:=(X,Z,S,Y)$. Assume that we observe $n$ i.i.d. copies of $O$.

Here we are interested in the complier and noncomplier average causal effects (CACE and NACE). As the PI identification strategy is symmetric with respect to these two effects (and so are the sensitivity assumptions we consider), we focus on the generic estimand

$${\mathrm{\Delta }}_c:=\mathrm{E}\left[Y_1-Y_0\mid C=c\right]=\mathrm{E}\left[Y_1\mid C=c\right]-\mathrm{E}\left[Y_0\mid C=c\right],$$

where $c=1$ gives the CACE and $c=0$ gives the NACE.

Throughout we assume the usual causal inference assumptions:

$$\begin{array}{ll}\text{A}0\left(\text{consistency}\right):& Y=Z{Y}_1+(1-Z)Y_0,S=ZC,\\ \text{A}1\left(\text{treatment}\text{assignment}\text{ignorability}\right):& Z⫫\left(C,Y_1,Y_0\right)\mid X,\\ \text{A}2\left(\text{treatment}\text{assignment}\text{positivity}\right):& 0<\mathrm{P}(Z=1\mid X)<1.\end{array}$$

Under $\text{A}0$, we write $O=(X,Z,ZC,Y)$ to simplify presentation.

As several expressions appear repeatedly in the paper, we will use the shorthand notation

$$\begin{array}{ll}\hfill {\tau }_{zc}& :=\mathrm{E}\left[Y_z\mid C=c\right],\\ \hfill {\mu }_{zc}\left(X\right)& :=\mathrm{E}\left[Y_z\mid X,C=c\right],\\ \hfill {\mu }_0\left(X\right)& :=\mathrm{E}[Y\mid X,Z=0],\\ \hfill e(X,Z)& :=\mathrm{P}(Z\mid X),\\ \hfill {\pi }_c\left(X\right)& :=\mathrm{P}(C=c\mid X),\end{array}$$

for $z=0,1,c=0,1$. Here ${\mathrm{\Delta }}_c={\tau }_{1c}-{\tau }_{0c}$. Note the difference between ${\mu }_{zc}\left(X\right)$ which is the conditional mean of a potential outcome within a principal stratum and ${\mu }_0\left(X\right)$ which concerns the observed outcome in the control condition and does not condition on principal stratum. $e(X,1)$ is the propensity score. ${\pi }_c\left(X\right)$ is the probability of being in stratum $c$ given covariate values, which we also refer to as the principal score, following the literature^12,15,11,13.

Proofs of all results in this section are provided in Web Appendix B.

2.2 |. The identification challenge and the PI assumption

Identification of ${\mathrm{\Delta }}_c={\tau }_{1c}-{\tau }_{0c}$ amounts to identification of ${\tau }_{1c}$ and ${\tau }_{0c}$. The challenge is that while A0-A2 identify ${\tau }_{1c}$, they are not sufficient to identify ${\tau }_{0c}$. To see this, we start with the identity below.

Lemma 1.

$$\stackrel{=:{\tau }_{zc}}{\overbrace{\mathrm{E}\left[Y_z\mid C=c\right]}}=\frac{\mathrm{E}\{\stackrel{=:{\pi }_c\left(X\right)}{\overbrace{\mathrm{P}(C=c\mid X)}}\stackrel{=:{\mu }_{zc}\left(X\right)}{\overbrace{\mathrm{E}\left[Y_z\mid X,C=c\right]}}\}}{\mathrm{E}\left[\mathrm{P}\right(C=c\mid X\left)\right]}.$$ (1)

(To simplify presentation, it is left implicit that ${\mu }_{zc}\left(X\right)$ is only defined where ${\pi }_c\left(X\right)>0$.)

Lemma 1 says that ${\tau }_{zc}$ is equal to the weighted average of the stratum-specific potential outcome mean ${\mu }_{zc}\left(X\right)$ where the weight is proportional to the principal score ${\pi }_c\left(X\right)$. This means ${\tau }_{zc}$ can be identified via identification of ${\pi }_c\left(X\right)$ and ${\mu }_{zc}\left(X\right)$, which we address next.

Proposition 1 (Results without PI).

Under assumptions A0-A2,

$${\pi }_c\left(X\right)=\mathrm{P}(C=c\mid X,Z=1),$$ (2)

$${\mu }_{zc}\left(X\right)=\mathrm{E}[Y\mid X,Z=z,C=c],$$ (3)

$${\tau }_{1c}=\frac{\mathrm{E}\left[{\pi }_c\left(X\right){\mu }_{1c}\left(X\right)\right]}{\mathrm{E}\left[{\pi }_c\left(X\right)\right]}=\frac{\mathrm{E}\left[\frac{Z}{e(X,Z)}\mathrm{I}(C=c)Y\right]}{\mathrm{E}\left[\frac{Z}{e(X,Z)}\mathrm{I}(C=c)\right]},$$ (4)

$${\pi }_1\left(X\right){\mu }_{01}\left(X\right)+{\pi }_0\left(X\right){\mu }_{00}\left(X\right)=\stackrel{=:{\mu }_0\left(X\right)}{\overbrace{\mathrm{E}\left[Y∣X,Z=0\right]}}.$$ (5)

Proposition 1 shows that A0-A2 identify ${\pi }_c\left(X\right)$ and ${\mu }_{1c}\left(X\right)$, but not ${\mu }_{0c}\left(X\right)$. (The RHS of 3) conditions on $C$, which is not observed for $Z=0$.) Hence ${\tau }_{1c}$ is identified, but ${\tau }_{0c}$ is not, so ${\mathrm{\Delta }}_c$ is not.

The problem here is nonidentifiability of the stratum-specific conditional $Y_0$ mean functions ${\mu }_{0c}\left(X\right)$. These two functions, ${\mu }_{01}\left(X\right)$ for compliers (where $c=1$) and ${\mu }_{00}\left(X\right)$ for noncompliers (where $c=0$), are tied together as two unknowns in one equation, 55, which we will call the mixture equation. To identify them, some additional assumption is needed.

One such assumption is PI, which we state here as a conditional mean independence:

$$\mathrm{A}3(\text{principal}\text{ignorability}):\stackrel{=:{\mu }_{01}\left(X\right)}{\overbrace{\mathrm{E}\left[Y_0\mid X,C=1\right]}}=\stackrel{=:{\mu }_{00}\left(X\right)}{\overbrace{\mathrm{E}\left[Y_0\mid X,C=0\right]}}.$$

PI is also sometimes stated as $C⫫Y_0\mid X$ (which implies A3). This version is more intuitive: it is satisfied if $X$ captures all common causes of $C$ and $Y_0$¹³. Like other authors, we assume that A3 and A1 involve the same set of covariates; this can be relaxed.

A3 combined with (5) solves the identification problem.

Proposition 2 (PI based identification).

Under assumptions A0-A3,

$${\mu }_{0c}\left(X\right)={\mu }_0\left(X\right),$$ (6)

$${\tau }_{0c}=\frac{\mathrm{E}\left[{\pi }_c(X){\mu }_0(X)\right]}{\mathrm{E}\left[{\pi }_c(X)\right]}=\frac{\mathrm{E}\left[\frac{Z}{e(X,Z)}\mathrm{I}(C=c){\mu }_0(X)\right]}{\mathrm{E}\left[\frac{Z}{e(X,Z)}\mathrm{I}(C=c)\right]}=\frac{\mathrm{E}\left[\frac{1-Z}{e(X,Z)}{\pi }_c(X)Y\right]}{\mathrm{E}\left[\frac{1-Z}{e(X,Z)}{\pi }_c(X)\right]}.$$ (7)

We will refer to the observed data functionals in Proposition 2 that identify ${\mu }_{0c}(X)$ and ${\tau }_{0c}$ as ${\mu }_{0c}^{\mathrm{P}\mathrm{I}}(X)$ and ${\tau }_{0c}^{\mathrm{P}\mathrm{I}}$, and the corresponding result for ${\mathrm{\Delta }}_c$ (i.e., ${\tau }_{1c}-{\tau }_{0c}^{\mathrm{P}\mathrm{I}}$) as ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$.

Remark 1 (Sufficient PI version).

A3 involves $Y_0$ but not $Y_1$. Feller et al. (2017)¹¹ call this assumption weak PI to differentiate it from a different assumption (strong PI) that involves both potential outcomes, $C⫫Y_z\mid X$ for $z=0,1$. While these labels suggest a difference in degree, these assumptions are qualitatively different. Strong PI implies that conditional on $X$, the average causal effect is constant across principal strata, which is generally not desired¹¹. As A3 is sufficient (and strong PI is unnecessary), we simply refer to A3 as PI.

PI is untestable. The sensitivity analyses in Sections 4 and 5 will each replace PI with an alternative assumption (sensitivity assumption) indexed by a sensitivity parameter representing deviation from PI. Such an assumption obtains alternative identification results for ${\mu }_{0c}(X)$ and ${\mathrm{\Delta }}_c$. The sensitivity analysis then shows, for a plausible range of the sensitivity parameter, how effect estimates depart from those obtained in a PI-based analysis.

3 |. THREE TYPES OF PI-BASED ESTIMATORS FROM THE LENS OF SENSITIVITY ANALYSIS

It is desirable to develop sensitivity analysis methods that are simple modifications of PI-based methods. With this in mind, in this section we group estimators of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ into three types (each with a few example estimators), which we anticipate can be adapted for sensitivity analysis using different techniques (in subsequent sections). This grouping may be useful generally, say, where it is desirable to use a different sensitivity assumption not covered in this paper.

With three estimator types, the presentation from here through Section 5 is slightly complex. Readers who are mainly looking to add a sensitivity analysis to an already conducted or planned PI-based analysis only need to focus on the type of their estimator and can ignore the others.

Proofs of results in this section are provided in Web Appendix C.

3.1 |. Type A (≈ outcome regression estimators)

As PI-based analysis relies on the identification result ${\mu }_{0c}^{\mathrm{P}\mathrm{I}}(X)={\mu }_0(X)$, an obvious sensitivity analysis technique (applicable to any PI-based method that involves estimating ${\mu }_0(X)$) is to replace ${\mu }_0(X)$ with the alternative formula for ${\mu }_{0c}(X)$ identified under the sensitivity assumption. We aim to use this technique with type A (roughly outcome regression) estimators.

To be precise, type A estimators involve estimating ${\mu }_0(X)$ in order to first estimate the principal causal effect conditional on covariates (which under PI is ${\mu }_{1c}(X)-{\mu }_0(X)$) or a proxy for it, and then aggregate these conditional effects to estimate the average principal causal effect ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$. Examples include the principal-score-weighted outcome-regression estimator (aka the plug-in estimator) (8) and the propensity-score-weighted outcome-regression estimator (9):

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{P}\mathrm{I}}:=\frac{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)\left[{\hat{\mu }}_{1c}\left(X_i\right)-{\hat{\mu }}_0\left(X_i\right)\right]}{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)},$$ (8)

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{P}\mathrm{I}}:=\frac{\sum _{i=1}^n\frac{Z_{i}I\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}\left[Y_i-{\hat{\mu }}_0\left(X_i\right)\right]}{\sum _{i=1}^n\frac{Z_{i}I\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}},$$ (9)

where the hat notation indicates an estimated function. These are justified by the ${\tau }_{1c}$ formulae in (4) and the first two ${\tau }_{0c}^{\mathrm{P}\mathrm{I}}$ formulae in (7). Also included in type A is a multiply robust outcome regression estimator, ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$, which we will present after explaining type B estimators.

For each estimator here we highlight in red the component to be replaced in sensitivity analysis.

3.2 |. Type B (≈ influence function based estimators)

Type B estimators are a subset of estimators constructed based on the nonparametric influence function (IF) of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ (hence the rough label IF-based estimators, although not all IF-based estimators belong in type B). To define this type precisely, let

$${\nu }_{zc}:=\mathrm{E}\left[{\pi }_c(X){\mu }_{zc}(X)\right],$$

$${\pi }_c:=\mathrm{E}\left[{\pi }_c(X)\right],$$

$${\nu }_{0c}^{\mathrm{P}\mathrm{I}}:=\mathrm{E}\left[{\pi }_c(X){\mu }_0(X)\right].$$

In this notation, ${\mathrm{\Delta }}_c=\left({\nu }_{1c}-{\nu }_{0c}\right)/{\pi }_c$ and ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}=\left({\nu }_{1c}-{\nu }_{0c}^{\mathrm{P}\mathrm{I}}\right)/{\pi }_c$. A type B estimator of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ is one that can be expressed as a combination of IF-based estimators of ${\nu }_{1c},{\nu }_{0c}^{\mathrm{P}\mathrm{I}}$ and ${\pi }_c$. The sensitivity analysis technique will be to replace the ${\nu }_{0c}^{\mathrm{P}\mathrm{I}}$ component with an IF-based estimator of ${\nu }_{0c}$ under the sensitivity assumption. To obtain these estimators, we derive the relevant IFs.

Proposition 3 (IFs for PI-based analysis).

The IFs of ${\pi }_c,{\nu }_{1c},{\nu }_{0c}^{\mathrm{P}\mathrm{I}}$, and ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ are

$${\phi }_{{\pi }_c}(O)=\frac{Z}{e(X,Z)}\left[\mathrm{I}(C=c)-{\pi }_c(X)\right]+{\pi }_c(X)-{\pi }_c,$$ (10)

$${\phi }_{{\nu }_{1c}}(O)=\frac{Z}{e(X,Z)}\mathrm{I}(C=c)\left[Y-{\mu }_{1c}(X)\right]+\frac{Z}{e(X,Z)}{\mu }_{1c}(X)\left[\mathrm{I}(C=c)-{\pi }_c(X)\right]+{\pi }_c(X){\mu }_{1c}(X)-{\nu }_{1c},$$ (11)

$${\phi }_{{\nu }_{0c}^{\mathrm{P}\mathrm{I}}}(O)=\frac{1-Z}{e(X,Z)}{\pi }_c(X)[Y-{\mu }_0(X)]+\frac{Z}{e(X,Z)}{\mu }_0(X)[\mathrm{I}(C=c)-{\pi }_c(X)]+{\pi }_c(X){\mu }_0(X)-{\nu }_{0c}^{\mathrm{P}\mathrm{I}},$$ (12)

$${\phi }_{{\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}}(O)=\frac{1}{{\pi }_c}\left\{[{\phi }_{{\nu }_{1c}}(O)+{\nu }_{1c}]-[{\phi }_{{\nu }_{0c}^{\mathrm{P}\mathrm{I}}}(O)+{\nu }_{0c}^{\mathrm{P}\mathrm{I}}]-{\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}[{\phi }_{{\pi }_c}(O)+{\pi }_c]\right\}.$$ (13)

The estimator that uses the IF of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ (with estimated nuisances) as the estimating function is a type B estimator. This is because due to (13), this estimator has the form

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}=\frac{{\hat{\nu }}_{1c,\mathrm{I}\mathrm{F}}-{\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}}{{\hat{\pi }}_{c,\mathrm{I}\mathrm{F}}},$$ (14)

where (with ${\mathrm{P}}_n$ representing sample average)

$$\begin{gathered} {\hat{\nu }}_{1c,\mathrm{I}\mathrm{F}}:={\mathrm{P}}_n\left\{\frac{Z}{\hat{e}(X,Z)}\mathrm{I}(C=c)\left[Y-{\hat{\mu }}_{1c}(X)\right]+\frac{Z}{\hat{e}(X,Z)}{\hat{\mu }}_{1c}(X)\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X){\hat{\mu }}_{1c}(X)\right\}, \\ {\hat{\nu }}_{0c,\text{IF}}^{\mathrm{P}\mathrm{I}}:={\mathrm{P}}_n\left\{\frac{1-Z}{\hat{e}(X,Z)}{\hat{\pi }}_c(X)\left[Y-{\hat{\mu }}_0(X)\right]+\frac{Z}{\hat{e}(X,Z)}{\hat{\mu }}_0(X)\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X){\hat{\mu }}_0(X)\right\}, \\ {\hat{\pi }}_{c,\mathrm{I}\mathrm{F}}:={\mathrm{P}}_n\left\{\frac{Z}{\hat{e}(X,Z)}\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X)\right\} \end{gathered}$$

are IF-based estimators of ${\nu }_{1c},{\nu }_{0c}^{\mathrm{P}\mathrm{I}},{\pi }_c$.

Another type B estimator is the Hájek-type ²³ estimator,

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}=\frac{{\hat{\nu }}_{1c,\mathrm{I}\mathrm{F}\mathrm{H}}-{\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}}{{\hat{\pi }}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}},$$ (15)

where ${\hat{\nu }}_{1c,\text{IFH}}$, ${\hat{\nu }}_{0c,\text{IFH}}^{\mathrm{P}\mathrm{I}}$, ${\hat{\pi }}_{c,\text{IFH}}$ are a modified version of ${\hat{\nu }}_{1c,\text{IF}}$, ${\hat{\nu }}_{0c,\text{IF}}^{\mathrm{P}\mathrm{I}}$, ${\hat{\pi }}_{c,\text{IF}}$, replacing $\frac{Z}{\hat{e}(X,Z)}$ with $\frac{Z}{\hat{e}(X,Z)}/{\mathrm{P}}_n[\frac{Z}{\hat{e}(X,Z)}]$ and $\frac{1-Z}{\hat{e}(X,Z)}$ with $\frac{1-Z}{\hat{e}(X,Z)}/P_n[\frac{1-Z}{\hat{e}(X,Z)}]$. (We call this modification Hájek-ization.)

${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ are multiply robust (see Proposition 4 below). ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ is range-preserving.

Circling back to type A.

We now present the multiply robust outcome regression estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ mentioned earlier. This is a multi-step estimator (the MS subscript is for “multi-step”) that is based on expressing the IF of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ as a sum of three terms:

$$\begin{gathered} {\phi }_{{\mathrm{\Delta }}_c^{\mathrm{P}}}(O)=\frac{1}{{\pi }_c}\{\stackrel{\left(\mathrm{*}\right)}{\overbrace{\frac{Z}{e(X,Z)}\mathrm{I}(C=c)\left[Y-{\mu }_{1c}(X)\right]}}-\stackrel{\left(\mathrm{*}\mathrm{*}\right)}{\overbrace{\frac{1-Z}{e(X,Z)}{\pi }_c(X)\left[Y-{\mu }_0(X)\right]}}+ \\ \underset{\left(\mathrm{*}\mathrm{*}\mathrm{*}\right)}{\underbrace{\left[\frac{Z}{e(X,Z)}\left[\mathrm{I}(C=c)-{\pi }_c(X)\right]+{\pi }_c(X)\right][{\mu }_{1c}(X)-{\mu }_0(X)-{\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}]}}\}, \end{gathered}$$ (16)

and building steps that zero out the sample means of the terms. The resulting estimator is

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}:=\frac{\sum _{i=1}^n\hat{w}\left(O_i\right)\left[{\tilde{\mu }}_{1c}\left(X_i\right)-{\tilde{\mu }}_0\left(X_i\right)\right]}{\sum _{i=1}^n\hat{w}\left(O_i\right)}.$$ (17)

Here $\hat{w}(O):=\frac{Z}{\hat{e}(X,Z)}\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X)$. ${\tilde{\mu }}_{1c}(X)$ and ${\tilde{\mu }}_0(X)$ are specific estimators of ${\mu }_{1c}(X)$ and ${\mu }_0(X):{\tilde{\mu }}_{1c}(X)$ is fit to (non)compliers in the treatment arm weighted by $1/\hat{e}(X,1)$, ${\tilde{\mu }}_0(X)$ is fit to control units weighted by ${\hat{\pi }}_c(X)/\hat{e}(X,0)$, and both are mean-recovering models (i.e., on the sample to which the model is fit, the mean of model predictions equals outcome mean). These models zero out the sample means of $\left(\mathrm{*}\right)$ and $\left(\mathrm{*}\mathrm{*}\right)$, and the weighted averaging in 17 zeros out the sample mean of $\left(\mathrm{*}\mathrm{*}\mathrm{*}\right)$. $\hat{w}(O)$ can also be Hájek-ized, for another version.)

Remark 2.

The tilde notation here refers to this specific method of estimating $\mu$ functions for this estimator. The weighting targets the model to the relevant covariate space where it is used for prediction, and the mean-recovering feature ensures that predictions are on average unbiased (if the weights are correct). This targeted estimation technique can also be used (but is not required) for estimating ${\mu }_{1c}(X)$ and ${\mu }_0(X)$ for other estimators, and for estimating ${\pi }_c(X)$.

${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\text{MS}}^{\mathrm{P}\mathrm{I}}$ shares the same multiply robust property of ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\text{IF}}^{\mathrm{P}\mathrm{I}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\text{IFH}}^{\mathrm{P}\mathrm{I}}$ (see Proposition 4).

Proposition 4 (multiply robust PI-based estimators).

${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ are consistent if one of the following three conditions hold:

1. the propensity score $e(X,Z)$ and principal score ${\pi }_c(X)$ models are correctly specified; or

2. the principal score model ${\pi }_c(X)$ and both outcome models ${\mu }_{1c}(X)$, ${\mu }_0(X)$ are correctly specified; or

3. the propensity score model $e(X,Z)$ and the outcome under control ${\mu }_0(X)$ model are correctly specified.

For simplicity, we presume that estimation uses parametric models. We leave IF-based inference using data-adaptive nuisance estimation to future work, except for a small first step of deriving nonparametric rate conditions (see Section 6.3).

3.3 |. Type C (≈ other/weighting estimators)

Type C estimators do not involve estimating ${\mu }_0(X)$ as a step in the estimation procedure. This type includes the pure weighting estimator

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{P}\mathrm{I}}:=\frac{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}{Y}_i}{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}}-\frac{\sum _{i=1}^n\frac{\left(1-Z_i\right){\hat{\pi }}_c\left(X_i\right)}{\hat{e}\left(X_i,Z_i\right)}{Y}_i}{\sum _{i=1}^n\frac{\left(1-Z_i\right){\hat{\pi }}_c\left(X_i\right)}{\hat{e}\left(X_i,Z_i\right)}},$$ (18)

justified by the second ${\tau }_{1c}$ formula in (4) and the third ${\tau }_{0c}^{\mathrm{P}\mathrm{I}}$ formula in (7). Also included in type C is the estimator that employs this same weighting scheme and uses the weighted sample to fit a model regressing outcome on treatment and covariates, say, to improve precision in estimating ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ (in the spirit of^24,25). For this type, we do not have a specific sensitivity analysis technique in mind, and will need to see whether the identification result under the sensitivity assumption allows a simple modification.

To sum up, we have defined three types of PI-based estimators: type A, whose defining feature is involving ${\mu }_0(X)$ estimation; type B, whose defining feature is having as a component an IF-based estimator of ${\nu }_{0c}^{\mathrm{P}\mathrm{I}}$, and type C, other estimators. We now consider sensitivity analysis.

4 |. SENSITIVITY ANALYSIS BASED ON THREE RATIO-TYPE SENSITIVITY PARAMTERS

Recall that the challenge before invoking PI was that the stratum-specific conditional $Y_0$ means ${\mu }_{01}(X)$ and ${\mu }_{00}(X)$ are not identified, as they are two unknowns in the mixture equation

$${\mu }_{01}\left(X\right){\pi }_1\left(X\right)+{\mu }_{00}\left(X\right){\pi }_0\left(X\right)={\mu }_0\left(X\right).$$

(5)

PI identifies ${\mu }_{01}(X)$ and ${\mu }_{00}(X)$ by equating them to each other. A sensitivity analysis replaces PI with a sensitivity assumption that allows ${\mu }_{01}(X)$ and ${\mu }_{00}(X)$ to differ from each other. The assumption is indexed by a sensitivity parameter indicating how and to what degree they differ. To accommodate different outcome types (binary, bounded, unbounded) and different conceptualizations of how ${\mu }_{01}(X)$ and ${\mu }_{00}(X)$ may differ, we consider different parameterizations. The following assumptions use an odds ratio (OR), a generalized odds ratio (GOR) and a mean ratio (MR) sensitivity parameter. In all of them, $\rho =1$ recovers the PI case.

A4-OR (sensitivity odds ratio): $\frac{{\mu }_{01}(X)/\left[1-{\mu }_{01}(X)\right]}{{\mu }_{00}(X)/\left[1-{\mu }_{00}(X)\right]}=\rho$,

A4-GOR (sensitivity generalized odds ratio): $\frac{\left[{\mu }_{01}(X)-l\right]/\left[h-{\mu }_{01}(X)\right]}{\left[{\mu }_{00}(X)-l\right]/\left[h-{\mu }_{00}(X)\right]}=\rho$, where $l,h$ are the lower and upper $Y_0$ bounds,

A4-MR (sensitivity mean ratio): $\frac{{\mu }_{01}(X)}{{\mu }_{00}(X)}=\rho$,

for some positive range of $\rho$ that is considered plausible.

As mentioned in Section 1.2, a challenge with A4-MR is that it may predict out of the outcome range. For an example, consider an outcome on a 0 to 7 scale. Suppose that for some covariate value $x,{\mu }_0(X)=5$ and ${\pi }_1(X)=0.3$. Then a sensitivity MR value of 1.69 would imply ${\mu }_{01}(X)>7$. A4-MR is thus more suitable if the outcome is single-signed and unbounded. Since most outcomes are practically bounded, if using A4-MR, the parameter range should be carefully selected to avoid predicting extreme ${\mu }_{0c}(X)$ values; we will discuss this in Section 6.1.

For binary outcomes, we propose A4-OR, the assumption that within levels of $X$ (i) the odds of the outcome for compliers is $\rho$ times that for noncompliers, or equivalently (because ORs are symmetric), (ii) the odds of being a complier for those with the outcome is $\rho$ times that for those without the outcome. A4-OR predicts ${\mu }_{0c}(X)$ within [0, 1].

More generally, for outcomes bounded on both ends, we propose A4-GOR, a generalization of A4-OR. (A4-OR is a special case with $l=0$ and $h=1$.) Figure 1 shows the connection between ${\mu }_{00}(X)$ and ${\mu }_{01}(X)$ for several GOR values. If the outcome range varies with $X$, the bounds can be made $X$-value-specific, i.e., $l(X)$ and $h(X)$. A4-GOR always predicts ${\mu }_{0c}(X)$ within the specified bounds. For a non-binary outcome, however, A4-GOR may still contradict with the observed outcome distribution in ways that are not obvious, e.g., predicting ${\mu }_{0c}(X)$ values far from where the outcome mass is concentrated.

FIGURE 1.

Connection between ${\mu }_{00}(X)$ and ${\mu }_{01}(X)$ under A4-GOR for different GOR values

Remark 3 (Exponential tilting connection).

A4-OR can be equivalently expressed as

$$\mathrm{P}\left(Y_0\mid X,C=1\right)=\mathrm{P}\left(Y_0\mid X,C=0\right)\frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(\left(\mathrm{l}\mathrm{n}\rho \right)Y_0\right)}{\mathrm{E}\left[\mathrm{e}\mathrm{x}\mathrm{p}\left(\left(\mathrm{l}\mathrm{n}\rho \right)Y_0\right)\mid X,C=0\right]},$$ (19)

which looks like exponential tilting assumptions used in the context of non-ignorable missingness and unobserved confounding^26,27,28. The difference is that in these other problems, the assumption connects an unobserved distribution (e.g., that of missing data) to an observed distribution (that of non-missing data), whereas here the assumption relates two otherwise unidentified distributions whose mixture (and mixing ratio) is identified. Here the tilting-like assumption (19) achieves identification with a binary outcome but not generally. If $Y_0$ is continuous, for example, 19 (combined with the mixing weights ${\pi }_c(X)$) is not sufficient to identify the component distributions $\mathrm{P}\left(Y_0\mid X,C=c\right)$ (or their means) based on the mixture distribution $\mathrm{P}\left(Y_0\mid X\right)$.

Proofs of all results in this section are provided in Web Appendix D.

4.1 |. Identification

Combining any of the A4- assumptions with (5), we can identify ${\mu }_{0c}(X)$, which then identifies ${\tau }_{0c}$. We present results for A4-GOR (which includes A4-OR as a special case) and A4-MR.

To maintain symmetry, let ${\rho }_1=\rho ,{\rho }_0=1/\rho$.

Proposition 5 (GOR- and MR-based identification).

Under assumptions A0-A2 combined with A4-GOR,

$${\mu }_{0c}\left(X\right)=\left\{\begin{array}{ll}\frac{{\alpha }_c\left(X\right)-{\beta }_c\left(X\right)}{2\left({\rho }_c-1\right){\pi }_c\left(X\right)}\left(h-l\right)+l& \text{if}{\rho }_c\ne 1\\ {\mu }_0\left(X\right)& \text{if}{\rho }_c=1\end{array}=:{\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}\left(X\right)\right.,$$ (20)

and under assumptions A0-A2 combined with A4-MR,

$${\mu }_{0c}\left(X\right)={\gamma }_c\left(X\right){\mu }_0\left(X\right)=:{\mu }_{0c}^{\mathrm{M}\mathrm{R}}\left(X\right),$$ (21)

where

$$\begin{gathered} {\alpha }_c(X):=\left[{\pi }_c(X)+\frac{{\mu }_0(X)-l}{h-l}\right]\left({\rho }_c-1\right)+1, \\ {\beta }_c(X):=\sqrt{{\left[{\alpha }_c(X)\right]}^2-4{\pi }_c\left(X\right)\frac{{\mu }_0\left(X\right)-l}{h-l}{\rho }_c\left({\rho }_c-1\right),} \\ {\gamma }_c(X):=\frac{{\rho }_c}{\left({\rho }_c-1\right){\pi }_c(X)+1}. \end{gathered}$$

Identification of ${\nu }_{0c},{\tau }_{0c},{\mathrm{\Delta }}_c$ follows from ${\mu }_{0c}(X)$ identification. We will label the results of these parameters under A4-GOR and A4-MR with superscripts ^GOR and ^MR, respectively.

4.2 |. Estimation

Based on the above identification results, we now modify the PI-based estimators. We let each resulting estimator inherit the label of the originating estimator, except replacing the superscript ^PI with one indicating the sensitivity assumption.

Figure 2 provides a summary of the key techniques presented here and in the next section.

FIGURE 2.

Flowchart summarizing key sensitivity analysis techniques that are applicable given PI-based estimator type and sensitivity parameterization

4.2.1 |. Type A estimators

These estimators are adapted by replacing the estimate of ${\mu }_0(X)$ with estimates of ${\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)$ or ${\mu }_{0c}^{\mathrm{M}\mathrm{R}}(X)$. For example, this turns the principal score weighted outcome regression estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{P}\mathrm{I}}$ (8) (aka the plug-in estimator) into

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{G}\mathrm{O}\mathrm{R}}:=\frac{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)[{\hat{\mu }}_{1c}\left(X_i\right)-{\hat{\mu }}_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}\left(X_i\right)]}{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{M}\mathrm{R}}:=\frac{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)[{\hat{\mu }}_{1c}\left(X_i\right)-{\hat{\mu }}_{0c}^{\mathrm{M}\mathrm{R}}\left(X_i\right)]}{\sum _{i=1}^n{\hat{\pi }}_c\left(X_i\right)},$$ (22)

where ${\hat{\mu }}_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}\left(X_i\right)$ and ${\hat{\mu }}_{0c}^{\mathrm{M}\mathrm{R}}\left(X_i\right)$ are ${\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}\left(X_i\right)$ (20) and ${\mu }_{0c}^{\mathrm{M}\mathrm{R}}\left(X_i\right)$ (21) evaluated at ${\hat{\mu }}_0\left(X_i\right)$ and ${\hat{\pi }}_c\left(X_i\right)$. The other outcome-regression estimators ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{P}\mathrm{I}}$ (9) and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ (17) are adapted similarly.

4.2.2 |. Type B estimators

Adaptation is based on the IFs of ${\nu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}:=\mathrm{E}\left[{\pi }_c(X){\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\right]$ and ${\nu }_{0c}^{\mathrm{M}\mathrm{R}}:=\mathrm{E}\left[{\pi }_c(X){\mu }_{0c}^{\mathrm{M}\mathrm{R}}(X)\right]$, which are provided in Proposition 6.

Proposition 6 (GOR- and MR-based IFs).

The IFs for ${\nu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\nu }_{0c}^{\mathrm{M}\mathrm{R}}$ are

$${\phi }_{{\nu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}}(O)=\frac{1-Z}{e(X,Z)}{ϵ}_{\mu ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\left[Y-{\mu }_0(X)\right]+\frac{Z}{e(X,Z)}{ϵ}_{\pi ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\left[\mathrm{I}(C=c)-{\pi }_c(X)\right]+{\pi }_c(X){\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)-{\nu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}},$$ (23)

$${\phi }_{{\nu }_{0c}^{\mathrm{M}\mathrm{R}}}(O)=\frac{1-Z}{e(X,Z)}{ϵ}_{\mu ,c}^{\mathrm{M}\mathrm{R}}(X)\left[Y-{\mu }_0(X)\right]+\frac{Z}{e(X,Z)}{ϵ}_{\pi ,c}^{\mathrm{M}\mathrm{R}}(X)\left[\mathrm{I}(C=c)-{\pi }_c(X)\right]+{\pi }_c(X){\mu }_{0c}^{\mathrm{M}\mathrm{R}}(X)-{\nu }_{0c}^{\mathrm{M}\mathrm{R}},$$ (24)

where

$$\begin{array}{ll}{ϵ}_{\mu ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X):=\{\begin{array}{ll}\frac{1}{2}-\frac{{\alpha }_c(X)}{2{\beta }_c(X)}+\frac{{\rho }_c{\pi }_c(X)}{{\beta }_c(X)}& \text{if}{\rho }_c\ne 1\\ {\pi }_c(X)& \text{if}{\rho }_c=1\end{array},& {ϵ}_{\mu ,c}^{\mathrm{M}\mathrm{R}}(X):={\gamma }_c(X){\pi }_c(X),\\ {ϵ}_{\pi ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X):=\{\begin{array}{ll}\left[\frac{1}{2}-\frac{{\alpha }_c(X)}{2{\beta }_c(X)}+\frac{{\rho }_c\frac{{\mu }_0(X)-l}{h-l}}{{\beta }_c(X)}\right](h-l)+l& \text{if}{\rho }_c\ne 1\\ {\mu }_0(X)& \text{if}{\rho }_c=1\end{array},& {ϵ}_{\pi ,c}^{\mathrm{M}\mathrm{R}}(X):={\gamma }_1(X){\gamma }_0(X){\mu }_0(X).\end{array}$$

Based on Proposition 6, under A4-GOR and A4-MR, we obtain estimators ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\text{IF}}^{\mathrm{M}\mathrm{R}}$ by replacing the ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}$ component of ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\text{IF}}^{\mathrm{P}\mathrm{I}}:=\frac{{\hat{\nu }}_{1c,\mathrm{I}\mathrm{F}}-{\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{P}}}{{\hat{\delta }}_{c,\mathrm{I}\mathrm{F}}}$ (14) with ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}}$, respectively, where

$${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}}:={\mathrm{P}}_n\left\{\frac{1-Z}{\hat{e}(X,Z)}{\widehat{ϵ}}_{\mu ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\left[Y-{\hat{\mu }}_0(X)\right]+\frac{Z}{\hat{e}(X,Z)}{\widehat{ϵ}}_{\pi ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X){\hat{\mu }}_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)\right\},$$ (25)

$${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}}:={\mathrm{P}}_n\left\{\frac{1-Z}{\hat{e}(X,Z)}{\widehat{ϵ}}_{\mu ,c}^{\mathrm{M}\mathrm{R}}(X)\left[Y-{\hat{\mu }}_0(X)\right]+\frac{Z}{\hat{e}(X,Z)}{\widehat{ϵ}}_{\pi ,c}^{\mathrm{M}\mathrm{R}}(X)\left[\mathrm{I}(C=c)-{\hat{\pi }}_c(X)\right]+{\hat{\pi }}_c(X){\hat{\mu }}_{0c}^{\mathrm{M}\mathrm{R}}(X)\right\},$$ (26)

and the $ϵ$ functions are estimated by evaluating them at ${\hat{\pi }}_c(X)$ and ${\hat{\mu }}_0(X)$.

${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ (15) is modified similarly to obtain sensitivity estimators ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{M}\mathrm{R}}$ by replacing ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ with ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{M}\mathrm{R}}$, the Hájek-ized version of ${\hat{\nu }}_{0c,\text{IF}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}}$.

4.2.2.1 |. Partial loss of robustness.

Proposition 4 stated that several PI-based estimators are multiply robust, including type B estimators ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}$ (14) and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ (15), and the multi-step type A estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ (17). The adaptation of these estimators for sensitivity analysis results in partial loss of robustness (see Proposition 7). The resulting GOR-based estimators (${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$) depend on correct specification of models for ${\pi }_c(X)$ and ${\mu }_0(X)$ (i.e., they are inconsistent if either model is misspecified). The MR-based counterparts (${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{M}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{M}\mathrm{R}}$) depend on correct specification of the model for ${\pi }_c(X)$.

Proposition 7 (Partial loss of robustness).

${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ are consistent for ${\mathrm{\Delta }}_c^{\mathrm{G}\mathrm{O}\mathrm{R}}$ if

• both the model for ${\pi }_c(X)$ and the model for ${\mu }_0(X)$ are correctly specified, AND

• either the model for $e(X,Z)$ or the model for ${\mu }_{1c}(X)$ is correctly specified.

• ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{M}\mathrm{R}}$ and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{M}\mathrm{R}}$ are consistent for ${\mathrm{\Delta }}_c^{\mathrm{M}\mathrm{R}}$ if

• the model for ${\pi }_c(X)$ is correctly specified, AND

• either the model for $e(X,Z)$ or both outcome models ${\mu }_{1c}(X)$, ${\mu }_0(X)$ are correctly specified.

Remark 4 (Approximate robustness).

Among these sensitivity estimators, the type B estimators (${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{M}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{M}\mathrm{R}}$) are in a sense more robust than the multi-step type A estimators (${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{G}\mathrm{O}\mathrm{R}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{M}\mathrm{R}}$): they have an approximate robustness property with respect to the model component(s) whose correct specification they require for consistency. Specifically, (i) while all six estimators depend on a correct model for ${\pi }_c(X)$, the type B estimators provide a first-order correction of the bias (that would be incurred if simply using the plug-in estimator 22) due to the deviation of the probability limit ${\pi }_c^{†}(X)$ of ${\hat{\pi }}_c(X)$ from the true function ${\pi }_c(X)$. Also, (ii) while all three GOR-based estimators additionally depend on a correct model for ${\mu }_0(X)$, the type B estimators provide a first-order correction of the bias due to the deviation of the probability limit ${\mu }_0^{†}(X)$ of ${\hat{\mu }}_0(X)$ from the true function ${\mu }_0(X)$. (This first-order bias correction feature is also shared by the originating PI-based estimators ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}},{\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$, and results in the robustness of those estimators.)

We give a quick explanation of (ii) to make this concrete. (For full details concerning Remark 4, see the Web Appendix.) If $\hat{e}(X,Z)$ and ${\hat{\pi }}_c(X)$ are correctly specified but ${\hat{\mu }}_0(X)$ is not, the probability limit of both ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ and ${\hat{\nu }}_{0c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{G}\mathrm{O}\mathrm{R}}$ is the sum of two terms

$$\mathrm{E}\left\{{\pi }_c(X){\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}[{\mu }_0^{†}(X),{\pi }_c(X)]\right\}+\mathrm{E}\left\{{ϵ}_{\mu ,c}^{\mathrm{G}\mathrm{O}\mathrm{R}}[{\mu }_0^{†}(X),{\pi }_c(X)][{\mu }_0(X)-{\mu }_0^{†}(X)]\right\}$$ (27)

(which result from the last and first terms in (25)). These are the first two terms in the Taylor expansion of the true parameter ${\nu }_{0c}^{\text{GOR}}$ treated as a function of ${\mu }_0()$ at the point ${\mu }_0^{†}()$. The first term coincides with the probability limit of the plug-in estimator, which is biased due to ${\mu }_0^{†}(X)\ne {\mu }_0(X)$. The second term provides a first-order correction of this bias. For this approximate robustness property to be beneficial, however, ${\mu }_0^{†}(X)$ needs to be close to ${\mu }_0(X)$.

4.2.3 |. Type C estimators

We consider A4-MR and A4-GOR separately. Under A4-MR, the convenient form of ${\mu }_{0c}^{\mathrm{M}\mathrm{R}}(X)$ (21) allows a simple adaptation of type C estimators: to estimate ${\mathrm{\Delta }}_c^{\mathrm{M}\mathrm{R}}$, scale the outcome in control units by a factor of ${\hat{\gamma }}_c(X)$ then use the PI-based analysis method. For the pure weighting estimator specifically, this adaptation results in the estimator

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{M}\mathrm{R}}:=\frac{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}{Y}_i}{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}}-\frac{\sum _{i=1}^n\frac{\left(1-Z_i\right){\hat{\pi }}_c\left(X_i\right)}{\hat{e}\left(X_i,Z_i\right)}{\hat{\gamma }}_c\left(X_i\right)Y_i}{\sum _{i=1}^n\frac{\left(1-Z_i\right){\hat{\pi }}_c\left(X_i\right)}{\hat{e}\left(X_i,Z_i\right)}}.$$

This outcome scaling technique is justified by the result below, a corollary of Proposition 5

Corollary 1 (MR-based outcome scaling).

$${\tau }_{0c}^{\mathrm{M}\mathrm{R}}=\frac{\mathrm{E}\left[\frac{1-Z}{e(X,Z)}{\pi }_c(X){\gamma }_c(X)Y\right]}{\mathrm{E}\left[\frac{1-Z}{e(X,Z)}{\pi }_c(X)\right]}.$$ (28)

Remark 5.

When specializing to the randomized treatment setting, 28p simplifies, and one expression of the specialized version of 28 is $\mathrm{E}[\frac{{\pi }_c(X){\gamma }_c(X)Y}{{\pi }_c}|Z=0]$, which appeared in Ding and Lu (2017, proposition 3)¹³. Based on this expression, this prior work characterizes the MR-based sensitivity analysis as an under/overweighting of the principal score by a factor of ${\gamma }_c(X)$. Interestingly, this characterization breaks the interpretation of ${\tau }_{0c}$ as a weighted average (our starting point in Lemma 1, which we have maintained throughout). Our new insight here is that the appearance of ${\gamma }_c(X)$ in (28) is due to the fact that under A4-MR the outcome mean ${\mu }_{0c}(X)$ is identified by ${\gamma }_c(X){\mu }_0(X)$. It is thus natural to use the scaling the outcome by a factor of ${\gamma }_c(X)$ characterization. Also, by leaving the principal score weights alone, this outcome scaling technique applies to type C estimators generally, not just the pure weighting estimator.

Under A4-GOR, there is no result similar to (28) that separates $Y$ from functions of $X$, therefore no simple modification is available for type C estimators. The pure weighting estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{P}\mathrm{I}}$ (9) (but not type C generally) can be adapted by replacing $Y$ in the second term with an estimate of ${\mu }_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}(X)$ (which requires estimating ${\mu }_0(X)$). For this estimator to reduce to ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{P}\mathrm{I}}$ when $\rho =1,{\mu }_0(X)$ has to be estimated by a ${\tilde{\mu }}_0(X)$ model (defined in 17). However, with ${\mu }_0(X)$ estimated, there are other options for estimating ${\mathrm{\Delta }}_c^{\text{GOR}}$ that one might prefer to such modification, e.g., replacing the whole second term of ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{P}\mathrm{I}}$ with $\frac{\sum _{i=1}^n\frac{Z_i\left(\left(C_i=c\right)\right.}{\hat{e}\left(X_i,Z_i\right)}{\hat{\mu }}_{0c}^{\mathrm{G}\mathrm{O}\mathrm{R}}}{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}}$. This obtains the type A estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{G}\mathrm{O}\mathrm{R}}$, which inconveniently does not reduce to ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\pi }^{\mathrm{P}\mathrm{I}}$ when $\rho =1$. Hence this is one place where we break the convention of respecting the primacy of the main analysis and recommend that, if a GOR-based sensitivity analysis is to be conducted, a type A (or type B) estimator be used for the main analysis.

5 |. SENSITIVITY ANALYSIS BASED ON A DIFFERENCE-TYPE SENSITIVITY PARAMETER

A4-OR, A4-GOR and A4-MR all assume that the means of $Y_0$ differ between compliers and noncompliers in some multiplicative manner. If one believes the difference is additive, it is more appropriate to use a sensitivity parameter that involves ${\mu }_{01}(X)-{\mu }_{00}(X)$. We propose using a standardized mean difference (SMD). For convenient notation, let

$$\begin{gathered} {\sigma }_{0c}^2(X):=\mathrm{v}\mathrm{a}\mathrm{r}\left(Y_0\mid X,C=c\right), \\ {\sigma }_0^2(X):=\mathrm{v}\mathrm{a}\mathrm{r}\left(Y∣X,Z=0\right). \end{gathered}$$

A simple SMD-based assumption is

$$\text{A4-SMD:}\frac{{\mu }_{01}(X)-{\mu }_{00}(X)}{\sqrt{\left[{\sigma }_{01}^2(X)+{\sigma }_{00}^2(X)\right]/2}}=\eta ,\text{for}\text{a}\text{plausible}\text{range}\text{of}\eta \text{.}$$

The denominator here is an “average” standard deviation: the quadratic mean of ${\sigma }_{01}(X)$ and ${\sigma }_{00}(X)$ (the within-stratum conditional standard deviations of $Y_0$). This standard deviation scale helps in selecting a range for $\eta$ and users can tap into intuition about SMDs from other contexts (e.g., measuring effect size ²⁹ or covariate imbalance³⁰. $\eta =0$ recovers the PI case; $\eta =\pm 1$ indicates a substantial complier-noncomplier difference in the outcome under control.

Inconveniently, $\mathrm{A}4-\mathrm{S}\mathrm{M}\mathrm{D}$ combined with A0-A2 only partially identifies ${\mu }_{0c}(X)$. For a simple sensitivity analysis, we consider the stronger assumption below, which supplements $\mathrm{A}4-\mathrm{S}\mathrm{M}\mathrm{D}$ with an equal variance assumption:

$$\mathrm{A}4-\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}\left(\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{S}\mathrm{M}\mathrm{D},\mathrm{e}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{v}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}\right):\mathrm{A}4-\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{a}\mathrm{n}\mathrm{d}{\sigma }_{01}^2\left(X\right)={\sigma }_{00}^2\left(X\right).$$

5.1 |. Identification

For symmetry, let ${\eta }_1=\eta$ and ${\eta }_0=-\eta$.

Proposition 8 (SMDe-based identification).

Under A0-A2 combined with A4-SMDe,

$${\mu }_{0c}(X)={\mu }_0(X)+{\eta }_c\underset{=:{\lambda }_c(X)}{\underbrace{\frac{{\pi }_{1-c}(X){\sigma }_0(X)}{\sqrt{1+{\eta }^2{\pi }_1(X){\pi }_0(X)}}=:{\mu }_{0c}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}(X),}}$$ (29)

$${\tau }_{0c}={\tau }_{0c}^{\mathrm{P}\mathrm{I}}+{\eta }_c\underset{=:{\xi }_c}{\underbrace{\frac{\mathrm{E}\left[{\pi }_c(X){\lambda }_c(X)\right]}{\mathrm{E}\left[{\pi }_c(X)\right]}}}=:{\tau }_{0c}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}},$$ (30)

$${\mathrm{\Delta }}_c={\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}-{\eta }_c{\xi }_c=:{\mathrm{\Delta }}_c^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}.$$ (31)

If equal variance is not assumed, ${\mathrm{\Delta }}_c$ is not point identified, but bounds can be obtained. The bounds can be narrowed if one additionally assumes that ${\sigma }_{01}^2(X)$ and ${\sigma }_{00}^2(X)$ differ from each other by less than a certain factor (see Proposition 8 b in the Web Appendix).

5.2 |. Estimation

This sensitivity analysis requires estimating ${\sigma }_0^2(X)$. For simplicity, in the illustration we use a quasi-likelihood approach assuming the outcome’s conditional variance is proportional to a function of its mean. An alternative is to directly model ${\left[Y-{\hat{\mu }}_0(X)\right]}^2$ based on $X$ in control units.

With the simple result 31, each estimator of ${\mathrm{\Delta }}_c^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}$ we obtain is an estimator of ${\mathrm{\Delta }}_c^{\mathrm{P}\mathrm{I}}$ minus ${\eta }_c$ times an estimator of ${\xi }_c$. This is the case regardless of the type of the PI-based estimator.

5.2.1 |. Simple type A estimators

Adaptation of ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{P}\mathrm{I}}$ (8) and ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{P}\mathrm{I}}$ (9) by replacing ${\mu }_0(X)$ with ${\mu }_{0c}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}(X)$ yields the following estimators:

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}:={\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\pi \mu }^{\mathrm{P}\mathrm{I}}-{\eta }_c\underset{=:{\hat{\xi }}_{c,\pi \sigma }}{\underbrace{\frac{{\sum }_{i=1}^n\frac{{\hat{\pi }}_1\left(X_i\right){\hat{\pi }}_0\left(X_i\right)\sqrt{{\hat{\sigma }}_0^2\left(X_i\right)}}{\sqrt{1+{\eta }^2{\hat{\pi }}_1\left(X_i\right){\hat{\pi }}_0\left(X_i\right)}}}{{\sum }_{i=1}^n{\hat{\pi }}_c\left(X_i\right)}}},$$ (32)

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}:={\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,e\mu }^{\mathrm{P}\mathrm{I}}-{\eta }_c\underset{=:{\hat{\xi }}_{c,e\pi \sigma }}{\underbrace{\frac{\sum _{i=1}^n\frac{Z_i\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}\frac{{\hat{\pi }}_{1-c}\left(X\right)\sqrt{{\hat{\sigma }}_0^2\left(X\right)}}{\sqrt{1+{\eta }^2{\hat{\pi }}_1\left(X\right){\hat{\pi }}_0\left(X\right)}}}{\sum _{i=1}^n\frac{Z_i\mathrm{I}\left(C_i=c\right)}{\hat{e}\left(X_i,Z_i\right)}}}}.$$ (33)

Rather than applying the same adaptation to the multi-step estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ (17), thanks to the special form of ${\mathrm{\Delta }}_c^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}$, we can adapt ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}$ the way we adapt other IF-based estimators.

5.2.2 |. IF-based estimators (including type B and multi-robust type A)

We adapt these estimators using IF-based estimators of ${\xi }_c$. Let $\vartheta (X):=\frac{{\pi }_1(X){\pi }_0(X){\sigma }_0(X)}{\sqrt{1+{\eta }^2{\pi }_1(X){\pi }_0(X)}}$ and $\vartheta :=\mathrm{E}[\vartheta (X)]$. Then ${\xi }_c=\vartheta /{\pi }_c$.

Proposition 9 (SMDe-based IF).

The IFs of $\vartheta$ and ${\xi }_c$ are

$${\phi }_{\vartheta }(O)=\frac{1-Z}{e(X,Z)}{\dot{\vartheta }}_{{\sigma }^2}(X)\{{\left[Y-{\mu }_0(X)\right]}^2-{\sigma }_0^2(X)\}+\frac{Z}{e(X,Z)}{\dot{\vartheta }}_{\pi }(X)\left[C-{\pi }_1(X)\right]+\vartheta (X)-\vartheta ,$$ (34)

$${\phi }_{{\xi }_c}(O)=\frac{1}{{\pi }_c}\left\{\left[{\phi }_{\vartheta }(O)+\vartheta \right]-{\xi }_c\left[{\phi }_{{\pi }_c}(O)+{\pi }_c\right]\right\},$$ (35)

where

$$\begin{gathered} {\dot{\vartheta }}_{{\sigma }^2}(X):=\frac{{\pi }_1(X){\pi }_0(X)}{2{\sigma }_0(X)\sqrt{1+{\eta }^2{\pi }_1(X){\pi }_0(X)}}, \\ {\dot{\vartheta }}_{\pi }(X):=\frac{\left[{\pi }_0(X)-{\pi }_1(X)\right]\left[2+{\eta }^2{\pi }_1(X){\pi }_0(X)\right]{\sigma }_0(X)}{2{\left[1+{\eta }^2{\pi }_1(X){\pi }_0(X)\right]}^{3/2}}. \end{gathered}$$

Based on Proposition 9, we have the estimator

$${\hat{\xi }}_{c,\mathrm{I}\mathrm{F}}:={\hat{\vartheta }}_{\mathrm{I}\mathrm{F}}/{\hat{\pi }}_{c,\mathrm{I}\mathrm{F}},$$ (36)

where

$${\hat{\vartheta }}_{\mathrm{I}\mathrm{F}}:={\mathrm{P}}_n\left[\frac{1-Z}{\hat{e}(X,Z)}{\hat{\dot{\vartheta }}}_{{\sigma }^2}(X)\{{\left[Y-{\hat{\mu }}_0(X)\right]}^2-{\hat{\sigma }}_0^2(X)\}+\frac{Z}{\hat{e}(X,Z)}{\hat{\dot{\vartheta }}}_{\pi }(X)\left[C-{\hat{\pi }}_1(X)\right]+\hat{\vartheta }(X)\right]$$

is the IF-based estimator of $\vartheta$ (where $\vartheta (X)$, ${\dot{\vartheta }}_{{\sigma }^2}(X)$ and ${\dot{\vartheta }}_{\pi }(X)$ are estimated by plugging in ${\hat{\pi }}_1(X)$, ${\hat{\pi }}_0(X)$ and $\sqrt{{\hat{\sigma }}_0^2(X)}$), and ${\hat{\pi }}_{c,\text{IF}}$ is the IF-based estimator of ${\pi }_c$ (defined under (14)). In addition, we have the estimator based on Hájek-ized versions of ${\hat{\vartheta }}_{\text{IF}}$ and ${\hat{\pi }}_{c,\text{IF}}$,

$${\hat{\xi }}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}:={\hat{\vartheta }}_{\mathrm{I}\mathrm{F}\mathrm{H}}/{\hat{\pi }}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}.$$ (37)

Then the adapted IF-based estimators are

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}:={\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}}^{\mathrm{P}\mathrm{I}}-{\eta }_c{\hat{\xi }}_{c,\mathrm{I}\mathrm{F}},$$ (38)

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}:={\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{I}\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}-{\eta }_c{\hat{\xi }}_{c,\mathrm{I}\mathrm{F}\mathrm{H}},$$ (39)

$${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{S}\mathrm{M}\mathrm{D}\mathrm{e}}:={\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,\mathrm{M}\mathrm{S}}^{\mathrm{P}\mathrm{I}}-{\eta }_c{\hat{\xi }}_{c,\mathrm{I}\mathrm{F}}.$$ (40)

Remark 6.

${\hat{\xi }}_{c,\text{IF}}$ and ${\hat{\xi }}_{c,\text{IFH}}$ depend on consistent estimation of ${\pi }_c(X)$ and ${\sigma }_0^2(X)$ (they are inconsistent if either component is inconsistent), but they have the approximately robust property where (i) if ${\hat{\pi }}_1(X)$, ${\hat{\mu }}_0(X)$ and $\hat{e}(X,Z)$ are consistent but ${\hat{\sigma }}_0^2(X)$ is not, the estimator provides a first-order correction of the bias of the plug-in estimator due to the deviation of the probability limit ${\sigma }_0^{2^{†}}(X)$ of ${\hat{\sigma }}_0^2(X)$ from the true ${\sigma }_0^2(X)$; and (ii) if ${\hat{\mu }}_0(X)$, ${\hat{\sigma }}_0^2(X)$ and $\hat{e}(X,Z)$ are consistent but ${\hat{\pi }}_c(X)$ is not, the estimator provides a first-order correction of the bias due to the deviation of the probability limit ${\pi }_c^{†}(X)$ of ${\hat{\pi }}_c(X)$ from the true ${\pi }_c(X)$. (See details in the Web Appendix.)

5.2.3 |. Other estimators

While any PI-based estimator can be paired with any ${\xi }_c$ estimator, to keep things simple it is reasonable to pair non-IF-based estimators with either ${\hat{\xi }}_{c,\pi \sigma }$ (32) or ${\hat{\xi }}_{c,e\pi \sigma }$ (33), which are not IF-based. As outcome modeling is needed to estimate ${\xi }_c$ for the sensitivity analysis, however, we recommend switching to a type A or IF-based estimator for the PI-based main analysis.

6 |. OTHER TOPICS

6.1 |. Using data in considering the range of the MR and SMD parameters

We now return to the issue that certain sensitivity parameters may predict extreme ${\mu }_{0c}(X)$ values. A example concerns the outcome earnings in our illustrative study. Since earnings span a large range, it may be intuitive to think about the earnings as differing in a multiplicative rather than additive manner, so a researcher may choose to use A4-MR for a sensitivity analysis. But earnings are not unbounded, and there is a maximum earning in the dataset, so we would be right to worry that certain sensitivity MR values may predict some ${\mu }_{0c}(X)$ values that are too high. A4-SMDe also has the same issue (to a lesser degree), where predicted ${\mu }_{0c}(X)$ values may be too high or too low. A4-GOR and A4-OR, on the other hand, predict within bounds.

We can use the data to gauge what values of the MR or SMD sensitivity parameter may be extreme, if we are willing to also specify bounds for the stratum-specific conditional $Y_0$ means, ${\mu }_{0c}(X)$. With A4-MR (and a non-negative outcome), we fix an upper bound (B) for ${\mu }_{0c}(X)$. With A4-SMDe, we fix a pair of upper $\left({\mathrm{B}}_h\right)$ and lower $\left({\mathrm{B}}_l\right)$ bounds. These can be informed by the observed outcome distribution, but are not necessarily bounds on the outcome itself. They are required to satisfy $\mathrm{B}\ge {\hat{\mu }}_0(X)$ or ${\mathrm{B}}_l\le {\hat{\mu }}_0(X)\le {\mathrm{B}}_h$ for all $X$ values in the data.

For each $X$ value, we can obtain an interval for the MR/SMD sensitivity parameter that does not predict ${\mu }_{0c}(X)$ outside of these assumed bounds. (This interval is derived in Web Appendix F, see Propositions 10 and 11.) We estimate such intervals for all covariate values and examine the distributions of their upper and lower ends to judge which ranges of the sensitivity parameter should not be allowed – see application in the illustrative example in Section 7

Note that while this helps guard against mathematically implausible values, it does not replace careful consideration based on substantive knowledge, which is important for deciding which range is practically plausible and relevant to the specific application.

6.2 |. Confidence interval estimation

The application in this paper estimates nuisance functions (e.g., propensity score, principal score and outcome mean) parametrically, for simplicity. All the estimators in sections 3, 4 and 5 are M-estimators. With parametric nuisance estimation, they are asymptotically normal and analytic standard errors can be derived using M-estimation calculus³¹, and the bootstrap is also valid. In our illustration below, we bootstrap and construct BCa confidence intervals³².

6.3 |. Rate conditions for nonparametric estimation

With a view to inform nonparametric inference (not the focus of this paper), we derive rate conditions on nonparametric nuisance estimation for IF-based estimators (using sample splitting or cross fitting) to be $\sqrt{n}$-consistent and asymptotically normal. See Propositions 12 and 13 in Web Appendix G for these results under PI and under the sensitivity assumptions, respectively. To our knowledge, our results are the first on rate conditions for sensitivity analyses for PI violation. They show that while PI-based analysis only requires typical rate conditions on several error products of nuisance functions (e.g., ${‖{\hat{e}}_1(X)-e_1(X)‖}_2‖{\hat{\pi }}_c(X)-{\pi }_c(X)‖=o_p(n^{-1/2})$), the sensitivity analyses require rate conditions on single nuisance functions (due to the presence of square errors in the remainder bias term). Specifically, we require ${‖{\hat{\pi }}_c(X)-{\pi }_c(X)‖}_2=o_p(n^{-1/4})$ with all the sensitivity analyses, and additionally ${‖{\hat{\mu }}_0(X)-{\mu }_0(X)‖}_2=o_p(n^{-1/4})$ with the GOR- and SMDe-based sensitivity analyses, and ${\Vert {\hat{\sigma }}_0^2(X)-{\sigma }_0^2(X)\Vert }_2=o_p(n^{-1/4})$ with the SMDe-based sensitivity analyis. These results immediately connect to the earlier results on the robustness under PI, and (partial) loss of robustness under sensitivity assumptions, of IF-based estimation.

6.4 |. Finite-sample bias

There is not an ideal choice for the placement of this topic. It is easier to read after reading the illustrative analysis in the next section. But we put it here for it is a small other topic.

Many consistent estimators are biased in finite samples. Methods to reduce such bias^33,34 are not often used, perhaps because the bias tends to be small, and the correction is complicated. The data example, however, reveals an interesting pattern of bias specific to sensitivity analysis that is worth noting. It is seen with the different outcomes and different estimators. An instance of this pattern is shown in Figure 3, all instances are shown in Web Appendix H.

FIGURE 3.

Point estimate and iterated bootstrap mean estimates. Plots are shown for the outcome work for pay.

In Figure 3 the solid black curve is the point estimate (which we refer to generically as $\hat{\theta }$), the dashed red curve is the mean of bootstrap estimates $({\overline{\hat{\theta }}}^{\mathrm{*}})$, and the dashed orange curve is the mean of estimates from the double bootstrap (bootstrap of bootstrap samples) $({\overline{\hat{\theta }}}^{\mathrm{*}\mathrm{*}})$. The shared pattern in all sensitivity analyses is that the slope of the ${\overline{\hat{\theta }}}^{\mathrm{*}}$ curve is less steep than that of the $\hat{\theta }$ curve, and the slope of the ${\overline{\hat{\theta }}}^{\mathrm{*}\mathrm{*}}$ is even less steep. (Note that the steepness of the curve indicates the degree to which sensitivity analysis estimates depart from the main analysis estimate.) For the two outcomes work and depressive symptoms, where the differences between $\hat{\theta },{\widehat{\hat{\theta }}}^{\mathrm{*}}$ and ${\overline{\hat{\theta }}}^{\mathrm{*}\mathrm{*}}$ are minimal in the main analysis, this means that in the sensitivity analysis ${\overline{\hat{\theta }}}^{\mathrm{*}}$ tends to be less extreme than $\hat{\theta }$, and ${\overline{\hat{\theta }}}^{\mathrm{*}\mathrm{*}}$ tends to be even less extreme; and this gets more pronounced the farther the sensitivity parameter is from its null value.

Finite-sample bias deserves dedicated investigation, which is outside the scope of this paper. This specific pattern, however, begs the question why. Our intuition is that it may be due to the fact that ${\tau }_{0c}$ is a weighted average of ${\mu }_{0c}(X)$ where the weights are ${\pi }_c(X)$, and under sensitivity assumptions the quantity being averaged ${\mu }_{0c}(X)$ depends on the weight ${\pi }_c(X)$. Specifically, with a fixed ${\mu }_0(X)$, ${\mu }_{0c}(X)$ is (i) monotone decreasing in ${\pi }_c(X)$ for ${\rho }_c>1$ or ${\eta }_c>0$, and (ii) monotone increasing in ${\pi }_c(X)$ for ${\rho }_c<1$ or ${\eta }_c<0$ (see Proposition 14 in Web Appendix H). This results in a coupling of (a) any deviation (of the finite sample from the population) in the weight with (b) a deviation in the quantity being averaged – in the opposite direction for case (i) and the same direction for case (ii). The resulting finite-sample bias is an attenuation of the difference between the sensitivity analysis and main analysis estimates.

For the data example we use a bootstrap-based bias correction after conducting a focused simulation study (see Web Appendix H). This bias correction is also implemented in our R-package.

7 |. JOBS II ILLUSTRATION

De-identified JOBS II data were accessed from the Inter-University Consortium for Political and Social Research data archive (www.icpsr.umich.edu). Our analysis focuses on the set of participants who were identified at initial screening as being at high risk for developing depression⁵. For illustrative purposes, we further subset to participants with complete data (n=465) and treat the resulting dataset as if it were an observational study. (Due to this restriction of the sample, analysis results should be seen as merely illustrative and not taken as substantive findings.) We consider three outcomes: working for pay (binary), monthly earnings (non-negative), and depressive symptoms (a score ranging from 1 to 5) at six months post-treatment. The study has a rich set of baseline covariates including demographics, household characteristics, employment history, motivation, and depressive symptoms. Given these covariates, we assume treatment assignment ignorability. We also assume PI in the main analysis.

Table 1 summarizes the covariate distribution (i) in the full analysis sample; (ii) stratified by compliance type in the treatment group (to give a sense of $X-C$ associations); and (iii) stratified by the binary work-for-pay outcome in the control group (to give a sense of $X-Y_0$ associations). Compared to noncompliers, compliers were more likely to be male, White, older and have a college degree. They were more likely to have ever married and have fewer cohabiting children, and less likely to have low household income. They were more likely to have had a professional job as their last steady job, to have been unemployed for a shorter time, and to report slightly higher job-seeking and program-participation motivation. In the control condition, participants who were younger, White, higher educated, unemployed for a shorter period, a manager at their last steady job, or reported higher motivation were more likely to be employed at six months.

TABLE 1.

Baseline covariates in (1) full analysis sample; (2) propensity-score-weighted treatment group, stratified by compliance type; and (3) propensity-score-weighted control group, stratified by outcome work for pay

	Full analysis sample (n=465)		Treatment group propensity-score-weighted				Control group propensity-score-weighted
			compliers (n=172) (n.wt=256.6)		noncompliers (n=139) (n.wt=208.0)		work (n=96) (n.wt=303.3)		not work (n=58) (n.wt=152.1)
	mean or %	(SD) (count)	mean or %	(SD) (count)	mean or %	(SD) (count)	mean or %	(SD) (count)	mean or %	(SD) (count)
Age	36.5	(9.9)	39.0	(9.7)	33.5	(9.8)	35.2	(9.4)	38.6	(11.3)
Sex (female)	57.6%	(268)	53.4%	(137)	62.9%	(130.9)	59.4%	(180.3)	56.7%	(86.3)
Race (white)	81.7%	(380)	85.1%	(218.5)	78.6%	(163.5)	87.1%	(264.3)	73.7%	(112.1)
Education
less than high school	10.5%	(49)	7.3%	(18.8)	16.5%	(34.3)	8.5%	(25.9)	16.3%	(24.8)
high school	29.7%	(138)	26.3%	(67.5)	31.7%	(66.0)	25.6%	(77.5)	35.6%	(54.2)
some college	38.9%	(181)	37.2%	(95.5)	41.7%	(86.7)	42.6%	(129.0)	34.7%	(52.8)
Bachelor’s degree	13.1%	(61)	19.4%	(49.9)	5.7%	(11.8)	14.6%	(44.3)	9.5%	(11.4)
graduate studies	7.7%	(36)	9.8%	(25.0)	4.4%	(9.3)	8.7%	(26.4)	3.9%	(5.9)
Marital status
never married	34.4%	(160)	31.8%	(81.6)	38.0%	(79.0)	35.2%	(106.8)	35.5%	(54.0)
married	38.7%	(180)	37.3%	(95.7)	38.5%	(80.2)	35.4%	(107.3)	39.8%	(60.6)
divorced/separated/widowed	26.9%	(125)	30.9%	(79.2)	23.5%	(48.8)	29.4%	(89.2)	24.6%	(37.5)
Kids in household	0.93	(1.13)	0.85	(1.12)	0.95	(1.17)	0.80	(1.05)	0.98	(1.03)
Household income
under 15K	22.8%	(106)	19.3%	(49.4)	26.7%	(55.6)	19.0%	(57.5)	31.3%	(47.6)
15K to under 25K	24.9%	(116)	22.1%	(56.6)	29.6%	(61.6)	34.3%	(104.1)	15.1%	(23.0)
25K to under 40K	25.8%	(120)	28.6%	(73.3)	23.3%	(48.6)	25.6%	(77.7)	24.0%	(36.5)
40K to under 50K	10.8%	(50)	12.6%	(32.4)	7.7%	(16.0)	6.7%	(20.2)	15.3%	(23.2)
50K or more	15.7%	(73)	17.5%	(44.9)	12.6%	(26.2)	14.4%	(43.8)	14.3%	(21.8)
Economic hardship	3.62	(0.92)	3.52	(0.92)	3.78	(0.92)	3.73	(0.91)	3.52	(1.00)
Occupation (last steady job)
professional	18.5%	(86)	26.7%	(68.5)	9.1%	(18.9)	17.1%	(51.8)	18.8%	(28.6)
managerial	17.2%	(80)	14.7%	(37.6)	19.5%	(40.6)	18.4%	(55.9)	10.8%	(16.4)
clerical	23.4%	(109)	23.9%	(61.3)	23.2%	(48.2)	22.9%	(69.6)	26.3%	(40.1)
sales	6.5%	(30)	5.2%	(13.3)	7.7%	(16.0)	7.9%	(23.8)	3.0%	(4.6)
crafts/foremen	12.9%	(60)	13.6%	(34.8)	12.2%	(25.4)	10.9%	(33.0)	18.8%	(28.6)
operative	9.5%	(44)	5.2%	(13.3)	14.6%	(30.4)	9.5%	(28.9)	7.3%	(11.1)
labor/service	12.0%	(56)	10.8%	(27.7)	13.7%	(28.5)	13.3%	(40.3)	15.0%	(22.7)
Weeks unemployed	9.3	(11.0)	8.1	(10.3)	10.4	(11.1)	8.0	(9.4)	10.5	(12.5)
Motivation to participate	5.34	(0.80)	5.50	(0.79)	5.19	(0.78)	5.41	(0.73)	5.37	(0.83)
Job-seeking motivation	82	(17)	84	(15)	81	(19)	85	(16)	76	(17)
Job-seeking self-efficacy	3.59	(0.83)	3.48	(0.84)	3.70	(0.82)	3.66	(0.76)	3.44	(0.84)
Assertiveness	2.99	(0.82)	2.90	(0.82)	3.07	(0.82)	2.97	(0.81)	2.94	(0.79)
Depressive symptoms	2.34	(0.68)	2.34	(0.69)	2.36	(0.68)	2.42	(0.70)	2.25	(0.60)

n.wt = weighted subsample size. Ranges of continuous/interval variables: age 17 to 77; kids in households 0 to 5 (one observation >5 truncated to 5), economic hardship 1 to 5; weeks unemployed 1 to 52 (12 observations >52 truncated to 52); motivation to participate 1 to 6.5; job-seeking motivation 0 to 100; job-seeking self-efficacy 1 to 5; assertiveness 1 to 5; depressive symptoms 1 to 5.

We aim to illustrate the use of the sensitivity assumptions introduced above with the different outcomes, and show how sensitivity analysis effect estimates depart from PI-based estimates. For this purpose, any type A or type B estimator suffices. We suppose that a researcher has chosen to use the Hájek-type IF-based estimator (15) for the PI-based analysis. We will briefly describe an implementation of this estimator, and then will focus on the sensitivity analyses.

We report bias-corrected point estimates (see Section 6.4) and BCa confidence intervals.

7.1 |. PI-based main analysis

The estimator ${\stackrel{\mathrm{ˆ}}{\mathrm{\Delta }}}_{c,1\mathrm{F}\mathrm{H}}^{\mathrm{P}\mathrm{I}}$ requires estimating several nuisance functions. We make relatively simple choices, keeping in mind what applied researchers may use in practice. We use logistic regression to fit the propensity score ($e(Z,X)$) and principal score $\left({\pi }_c(X)\right.$) models. These models include all baseline covariates, plus squares and square roots of continuous covariates; the inclusion of these additional terms is meant to improve covariate balance to be obtained from principal score and inverse propensity score weighting. We check balance as suggested in¹³. Figure 17 (in Appendix I) shows that covariate balance is improved (i) between the treated and control groups after propensity score weighting, and (ii) between treated (non)compliers and controls after principal score weighting combined with propensity score weighting.

Next, we estimate the conditional outcome mean functions for treated compliers (${\mu }_{11}(X)$), treated noncompliers (${\mu }_{10}(X)$) and controls (${\mu }_0(X)$). With the binary outcome work for pay, we use logistic regression. For the outcome earnings, the means are estimated conditional on working using gamma regression with log link. and then multiplied with the probability of working predicted by the work for pay model. (Small detail: since we use a noncanonical link with the gamma model, the predictions are slightly mean-biased; we calibrate them by a multiplicative constant to remove this bias.) For the depressive symptoms outcome, we use a simple transformation to the [0, 1] interval (by subtracting $l=1$ and dividing by $h-l=4$), fit a quasi-logistic model to the transformed outcome to estimate the conditional means, and then transform the means back to the original scale. These models include all baseline covariates.

We use targeted nuisance estimation (see Remark 2). The ${\pi }_c(X)$, ${\mu }_{11}(X)$ and ${\mu }_{10}(X)$ models are fit to data (treated group, treated compliers and treated noncompliers, respectively) weighted by $1/\hat{e}(Z,X)$. The ${\mu }_0(X)$ model is fit twice, to the control group weighted by ${\hat{\pi }}_1(X)/\hat{e}(Z,X)$ and weighted by ${\hat{\pi }}_0(X)/\hat{e}(Z,X)$, for CACE and NACE estimation, respectively.

Results (see Table 2) suggest that assignment to the intervention resulted in increased employment and earnings and decreased depressive symptoms for compliers. For noncompliers, effect estimates are close to null.

TABLE 2.

Pl-based analysis results: point estimates (and 95% BCa confidence intervals)

outcome	compliers			noncompliers
	mean $Y_1({\mathrm{\tau }}_{11})$	mean $Y_0({\mathrm{\tau }}_{01}^{\mathrm{P}\mathrm{I}})$	CACE (${\mathrm{\Delta }}_1^{\mathrm{P}\mathrm{I}}$)	mean $Y_1({\mathrm{\tau }}_{10})$	mean $Y_0({\mathrm{\tau }}_{00}^{\mathrm{P}\mathrm{I}})$	NACE (${\mathrm{\Delta }}_0^{\mathrm{P}\mathrm{I}}$)
work	75.4% (69.4, 81.4)	61.1% (53.1,68.4)	14.3 percentage points (4.7, 23.1)	68.5% (60.7, 75.2)	64.2% (55.7, 72.2)	4.3 percentage points (−6.2, 14.2)
earnings	$1,279 (1,107, 1,452)	$1,014 (802, 1,221)	$266 (18, 530)	$928 (776, 1,115)	$835 (666, 972)	$92 (−90, 318)
depressive symptoms	1.90 (1.80, 1.99)	2.07 (1.96,2.20)	−0.18 (−0.32, -0.04)	2.05 (1.94,2.16)	2.02 (1.88,2.14)	0.02 (−0.12, 0.18)

Variable work is binary. Actual earnings range is $0–5,667. Scale range of depressive symptoms is 1 to 5.

7.2 |. Sensitivity analysis

We now demonstrate sensitivity analyses that are OR-based for work for pay, MR-based for earnings, and GOR- and SMDe-based for depressive symptoms.

7.2.1 |. OR-based sensitivity analysis: work for pay

We noted above that some baseline factors such as socio-economic advantage and motivation are positively associated both with being a complier $\left(C\right)$ and with the work for pay outcome under control $\left(Y_0\right)$. One might be concerned whether, within subpopulations homogeneous in the observed covariates, there are other advantage type factors that are unobserved that relate to $C$ and $Y_0$ in a similar way; in that case the PI-based analysis might have overestimated the CACE and underestimated the NACE. On the other hand, one might be concerned that among people with the same $X$, some may not have needed to participate in the training because they had good prospects of finding a job; in that case the PI-based analysis might have been biased in the opposite direction. We thus consider a range of sensitivity OR values spanning both sides of 1. The results of this sensitivity analysis (Figure 4, top left) suggest that, even if (within levels of $X$) compliers had double the odds (relative to noncompliers) of getting work without the intervention, the intervention’s effect on having work for compliers would still be positive.

FIGURE 4.

Sensitivity analysis results: point estimates and 95% point-wise CIs for CACE, NACE and stratum-specific potential outcome means, for the range of the sensitivity parameter.

7.2.2 |. GOR-based sensitivity analysis: depressive symptoms

A concern may be that even among people with the same baseline covariate values (including baseline depressive symptoms score), compliers may be those who were more robust in some way (e.g., better at getting out of bed in the morning), and therefore may have better outcome (i.e., lower depressive symptoms at six months) under control than noncompliers. Therefore we consider sensitivity GOR values smaller than 1 (Figure 4, bottom left). The CACE estimate is quite sensitive to PI violation. It is negative (indicating a reduction in depressive symptoms) under PI, but as the sensitivity GOR deviates only slightly from 1, it quickly approaches zero.

7.2.3 |. MR-based sensitivity analysis: earnings

With this outcome, we use the MR sensitivity parameter. To illustrate the method as it would typically be used, we treat earnings as a stand-alone outcome, using ${\hat{\mu }}_0(X)$ as the only input, putting aside its connection with the work for pay outcome.

We start with a tentative MR range from 1/3 to 3, which is covered in Figure 4(top right). As mentioned earlier, it is challenging to choose what range to consider for the sensitivity parameter. Most important to this decision is substantive knowledge, including opinions of experts and study staff (who might know participants better than what is captured in the covariates). Such knowledge should be used, whenever it is available, to help rule in which range of the sensitivity parameter is practical and relevant.

As discussed in Section 6.1, the data can help rule out some implausible ranges. Here we simply use the maximum reported earnings under control ($5,667) as the upper bound B for ${\mu }_{0c}(X)$. After computing covariate-specific “legal” intervals for the sensitivity parameter, we use their end points to make the plot on the left in Figure 5, which shows the proportion of the sample with either ${\hat{\mu }}_{01}(X)$ or ${\hat{\mu }}_{00}(X)$ exceeding B under each MR value. We do not restrict the MR range based on this plot, as it suggests limited bound contradiction. (Alternatives include (i) restricting the MR range, or (ii) modifying the assumption to let the MR be $\rho$ for $X$ values where $\rho$ is in the legal interval, and otherwise be the legal value closest to $\rho$.)

FIGURE 5.

Bounds violation diagnostic: proportion contradicting bounds as a function of the sensitivity parameter

Another way to rely on the data is to examine what the MR values imply about the distributions of ${\mu }_{01}(X)$ values among compliers and of ${\mu }_{00}(X)$ values among noncompliers. Figure 18 (in Appendix I) plots these implied distributions for several MR values on [1/3, 3]. To judge whether such distributions are plausible, again, one should rely substantive knowledge if possible. Also, very large ${\mu }_{01}(X)$ or ${\mu }_{00}(X)$ values (especially those substantially larger than the maximum reported earnings) are suspect. Based on this, one might consider excluding MR values at the low end (1/3) and at the high end (≥ 2).

Another possibility is to supplement the A4-MR with other assumptions based on substantive knowledge. Suppose, for example, that substantive experts think it is unlikely that being assigned to the intervention is harmful to noncompliers (a relaxation of the ER assumption). Based on the results plot in Figure 4, this would narrow attention to the MR range above 1/2.

7.2.4 |. SMDe-based sensitivity analysis: depressive symptoms

Suppose that for the depressive symptoms outcome, an investigator prefers a sensitivity analysis based on A4-SMDe, being more comfortable communicating about mean differences. Here also, we consider a sensitivity SMD range to the left of the null value, where within $X$ levels, complier and noncomplier outcome means under control may differ by up to one standard deviation. Results (Figure 4, bottom right) look similar to those from the GOR-based sensitivity analysis, although using a different sensitivity parameter.

We note two details. First, this sensitivity analysis requires estimating the conditional variance ${\sigma }_0^2(X)$. Using the quasi-likelihood approach, we assume that ${\sigma }_0^2(X)$ is proportional to $\left[{\mu }_0(X)-l\right]\left[h-{\mu }_0(X)\right]$. This is equivalent to assuming that the outcome, after being shifted and rescaled to the [0, 1] interval, follows a quasibinomial model conditional on covariates. Recall that in the PI-based analysis, we transformed this outcome to the [0, 1] interval and fit a model with logit link. We now manually extract the dispersion parameter $\hat{\varphi }$ from this model and use it to compute the variance estimate ${\hat{\sigma }}_0^2(X)=\hat{\varphi }\left[{\hat{\mu }}_0(X)-l\right]\left[h-{\hat{\mu }}_0(X)\right]$. Second, the plot on the right of Figure 5 shows that for the SMD range considered there is minimal contradiction with the ${\mu }_{0c}(X)$ bounds, which here are simply set to the minimum and maximum depressive symptom scores. This is expected, as we consider a modest SMD range.

We do not conduct an SMDe-based sensitivity analysis for the outcome earnings, because the equal variance part of A4-SMDe is likely grossly incorrect for that outcome.

8 |. DISCUSSION

This paper substantially expands options for sensitivity analysis for PI violation in the estimation of complier and noncomplier average causal effects in two ways. First, we consider several sensitivity models with different sensitivity parameters (OR, GOR, MR, SMD) suitable for different outcome types and reflecting different ways compliers and noncompliers may differ with respect to outcome under control. Second, rather than proposing one estimator under the sensitivity model, we tailor sensitivity analysis techniques to different types of estimators (outcome regression, IF-based and weighting) that may be used for the PI-based main analysis.

There are several future directions for this line of sensitivity analysis. One is to incorporate data-adaptive nuisance estimation. As noted, the robustness available for PI-based analysis via IF-based estimation is partially lost for sensitivity analysis, making it more important that we estimate nuisance functions well. We provide rate conditions, but otherwise leave this to future work. Also important is how to handle missing data. Missing-at-random cases can be handled by standard techniques, but given the difference in compliance type observability between treatment arms, one may wish to allow certain not-at-random missingness, e.g., outcome missingness that depends on compliance type³⁵. Another extension is to adapt the methods to accommodate two-sided noncompliance and non-binary $S$, which are also common settings.

For the two-sided noncompliance case, extension is conceptually straightforward: wherever a PI assumption is used to disentangle a mixture it can be replaced with a sensitivity assumption. With binary $Z$ and $S$ there are four mixtures, so if PI assumptions are invoked to disentangle all four, then replacing those assumptions requires four sensitivity parameters. If one assumes away one principal stratum (say, defiers) to identify stratum prevalences and covariate distributions, then two mixtures remain, which means a PI-based analysis requires two PI assumptions and the sensitivity analysis involves two sensitivity parameters – see ²¹ for a sensitivity analysis using two MR parameters. While the idea is simple, works needs to be done to consider different (types of) PI-based estimators and pair them with sensitivity analysis techniques.

The methods in this paper belong to a mean-centric approach to sensitivity analysis. Each assumes a connection between two conditional outcome mean functions of complier and noncompliers. For a binary outcome, the sensitivity analysis based on A4-OR fully respects the observed outcome distribution. For continuous outcomes, however, the sensitivity analyses based on A4-GOR, A4-MR and A4-SMDe alone may conflict with the observed outcome distribution. The MR-based model may predict out of range because it treats the outcome as unbounded. The other two methods use some additional information: the GOR-based model takes in user-specified outcome bounds and respects those bounds; the SMDe-based model is informed about conditional outcome variability and with that information offers a scale-free sensitivity parameter. To mitigate the out-of-range prediction problem that affects the MR-based and to a lesser degree of the SMD-based method, we propose a simple technique that requires an additional assumption of bounds on stratum-specific conditional outcome means. There remains, however, the risk of more subtle conflict (e.g., predicting mean outcome in the tail of the distribution). A different approach is to avoid conflicting with the observed data distribution^27,26,28 all together by anchoring on the conditional distribution of the outcome under control rather than just its mean plus bounds/variance. Such sensitivity analysis (described briefly in the preprint³⁶) will be presented in a separate manuscript.

One last comment:

This paper provides technical solutions for doing sensitivity analysis, but does not address how to choose a relevant range for the sensitivity parameter and how to elicit and use expert opinion for this purpose. This is a topic that should receive more attention.

Abbreviations:

CACE

complier average causal effect

NACE

noncomplier average causal effect

ER

exclusion restriction

PI

principal ignorability

MR

mean ratio

OR

odds ratio

GOR

generalized odds ratio

SMD

standardized mean difference

IF

influence function

DATA AND CODE AVAILABILITY STATEMENT

The de-identified JOBS II data used in this paper can be requested from the Inter-University Consortium for Political and Social Research data archive at www.icpsr.umich.edu. All code to produce the results in this paper and to implement the proposed methods are provided in the R-package PIsens available at https://github.com/trangnguyen74/PIsens.

SUPPORTING INFORMATION

The Web Appendix may be found in the online version of the article at the publisher’s website, and also included on the next pages.