Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part II: Proofs of Results

Abstract

In this companion article to “Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes, Part I: Main Content” [Orellana, Rotnitzky and Robins (2010), IJB, Vol. 6, Iss. 2, Art. 7] we present (i) proofs of the claims in that paper, (ii) a proposal for the computation of a confidence set for the optimal index when this lies in a finite set, and (iii) an example to aid the interpretation of the positivity assumption.

1. Introduction

In this companion article to “Dynamic regime marginal structural mean models for estimation of optimal dynamic treatment regimes. Part I: Main Content” (Orellana, Rotnitzky and Robins, 2010) we present (i) proofs of the claims in that paper, (ii) a proposal for the computation of a confidence set for the optimal index when this lies in a finite set, and (iii) an example to aid the interpretation of the positivity assumption.

The notation, definitions and acronyms are the same as in the companion paper. Througout, we refer to the companion article as ORR-I.

2. Proof of Claims in ORR-I

2.1. Proof of Lemma 1

First note that the consistency assumption C implies that the event

$${\overline{O}}_k={\overline{o}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}({\overline{o}}_{k-1})$$

is the same as the event

$${\overline{O}}_k^g={\overline{o}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}({\overline{o}}_{k-1}).$$

So, with the definitions

$${\underset{\_}{V}}_{k,k+l}\equiv (V_{k+1},\dots ,V_{k+l}),l>0\text{ and }{\underset{\_}{V}}_{k,k}\equiv \text{nil}$$

we obtain

$$=\begin{array}{c}E\left[I_B(O,A){\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K,{\overline{A}}_K\right)|{\overline{O}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}\right)\right]\\ E[I_B\left(\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K+1}\right),\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)\\ \begin{array}{l}{\underset{\_}{\omega }}_{k-1,K}\left(\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K}\right),\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|{\overline{O}}_k^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)]\\ \text{w}\text{.p}\text{.1}\text{.}\end{array}\end{array}$$

Next, note that the fact that ${\underset{\_}{\omega }}_{k-1,K}\left(\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K}\right),\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)=0$ unless $A_k=g_k\left({\overline{O}}_k^g\right)$, $A_{k+1}=g_{k+1}\left({\overline{O}}_k^g,O_{k+1}\right),\dots ,A_K=g_{K+1}\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K}\right)$ implies that

$$\begin{array}{l}{I}_B\left(\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K+1}\right),\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)\times \\ {\underset{\_}{\omega }}_{k-1,K}\left(\left({\overline{O}}_k^g,{\underset{\_}{O}}_{k,K}\right),\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)\\ =I_B\left({\overline{O}}_{K+1}^g,{\overline{g}}_K\left({\overline{O}}_K^g\right)\right){\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right).\end{array}$$

Then, it follows from the second to last displayed equality that

$$=\begin{array}{c}E\left[I_B\left(O,A\right){\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K,{\overline{A}}_K\right)|{\overline{O}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}\right)\right]\\ E[I_B\left({\overline{O}}_{K+1}^g,{\overline{g}}_K\left({\overline{O}}_K^g\right)\right){\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|\\ {\overline{O}}_k^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)]\end{array}$$

$$=\begin{array}{c}E[E\left[{\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\right]\times \\ I_B\left({\overline{O}}_{K+1}^g,{\overline{g}}_K\left({\overline{O}}_K^g\right)\right)|{\overline{O}}_k^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)].\end{array}$$

So, part 1 of the Lemma is proved if we show that

$$E\left[{\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\right]=1.$$ (1)

Define for any k = 0, ..., K,

$${\underset{\_}{\omega }}_{k,k}\left({\overline{O}}_k^g,\left({\overline{g}}_k\left({\overline{O}}_k^g\right),{\underset{\_}{A}}_{k,k}\right)\right)\equiv 1.$$

To prove equality (1) first note that,

$$\begin{array}{l}=\begin{array}{c}E[{\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|\\ {\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}]\\ {\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \\ E[{\underset{\_}{\omega }}_{K-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|\\ {\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}]\end{array}\\ \begin{array}{c}=\hspace{1em}{\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \\ E\left[{\underset{\_}{\omega }}_{K-1,K}\left({\overline{O}}_K^g,{\overline{A}}_K\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right]\end{array}\\ \begin{array}{c}={\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \\ E\left[\frac{I_{\left\{g_K\left({\overline{O}}_K^g\right)\right\}}\left(A_K\right)}{{\lambda }_K\left(g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_K^g,{\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right)}|{\overline{O}}_{K+1}^g,{\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right]\end{array}\\ \begin{array}{c}={\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \\ \frac{E\left[I_{\left\{g_K\left({\overline{O}}_K^g\right)\right\}}\left(A_K\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right]}{{\lambda }_K\left(g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_K^g,{\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right)}\end{array}\\ \begin{array}{c}={\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \\ \frac{\mathbb{P}\left[A_K=g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right]}{{\lambda }_K\left(g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_K^g,{\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right)}\end{array}\end{array}$$

$$\begin{array}{l}\begin{array}{l}={\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\times \hfill \\ \frac{\mathbb{P}\left[A_K=g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_K^g,{\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right]}{{\lambda }_K\left(g_K\left({\overline{O}}_K^g\right)|{\overline{O}}_K^g,{\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)\right)}\hfill \end{array}\\ ={\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)\end{array}$$

where the second to last equality follows because given ${\overline{O}}_K^g$ and ${\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)$, $O_{K+1}^g$ is a fixed, i.e. non-random function of $\mathcal{O}$ and consequently by the sequential randomization assumption, $O_{K+1}^g$ is conditionally independent of A_K given ${\overline{O}}_K^g$ and ${\overline{A}}_{K-1}={\overline{g}}_{K-1}\left({\overline{O}}_{K-1}^g\right)$. The last equality follows by the definition of λ_K (·|·, ·).

We thus arrive at

$$\begin{array}{l}=\begin{array}{l}E\left[{\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\right]\hfill \\ E\{E[{\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K}\right)\right)|\hfill \\ {\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}]|{\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\}\hfill \end{array}\\ =\begin{array}{c}E\{{\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)|\\ {\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\}\end{array}\end{array}$$

This proves the result for the case k = K. If k < K – 1, we analyze the conditional expectation of the last equality in a similar fashion. Specifically, following the same steps as in the long sequence of equalities in the second to last display we arrive at

$$\begin{array}{l}=\begin{array}{l}E\{{\underset{\_}{\omega }}_{k-1,K-1}\left({\overline{O}}_{K-1}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-1}\right)\right)|\hfill \\ {\overline{O}}_{K+1}^g,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-2}\}\hfill \\ {\underset{\_}{\omega }}_{k-1,K-2}\left({\overline{O}}_{K-2}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-2}\right)\right)\times \hfill \\ \times \frac{\mathbb{P}\left[A_{K-1}=g_{K-1}\left({\overline{O}}_{K-1}^g\right)|{\overline{O}}_{K+1}^g,{\overline{A}}_{K-2}={\overline{g}}_{K-2}\left({\overline{O}}_{K-2}^g\right)\right]}{{\lambda }_K\left(g_{K-1}\left({\overline{O}}_{K-1}^g\right)|{\overline{O}}_{K-1}^g,{\overline{g}}_{K-2}\left({\overline{O}}_{K-2}^g\right)\right)}\hfill \end{array}\\ ={\underset{\_}{\omega }}_{k-1,K-2}\left({\overline{O}}_{K-2}^g,\left({\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right),{\underset{\_}{A}}_{k-1,K-2}\right)\right)\end{array}$$

the last equality follows once again from the sequential randomization assumption. This is so because given ${\overline{O}}_{K-1}^g$ and ${\overline{A}}_{K-2}={\overline{g}}_{K-2}\left({\overline{O}}_{K-2}^g\right)$, ${\overline{O}}_K^g$ and ${\overline{O}}_{K+1}^g$ are fixed, i.e. deterministic, functions of $\mathcal{O}$ and the SR assumption ensures then that ${\overline{O}}_K^g$ and ${\overline{O}}_{K+1}^g$ are conditionally independent of A_K–1 given ${\overline{O}}_{K-1}^g$ and ${\overline{A}}_{K-2}={\overline{g}}_{K-2}\left({\overline{O}}_{K-2}^g\right)$.

Equality (1) is thus shown by continuing in this fashion recursively for K – 2, K – 3, ..., K – l until l such that K – l = k – 1.

To show Part 2 of the Lemma, note that specializing part 1 to the case k = 0, we obtain

$$E\left[I_B\left(O^g,A^g\right)|O_0\right]=E\left[I_B\left(O,A\right){\omega }_K\left({\overline{O}}_K,{\overline{A}}_K\right)|O_0\right].$$

Thus, taking expectations on both sides of the equality in the last display we obtain

$$E\left[I_B\left(O^g,A^g\right)\right]=E\left[I_B\left(O,A\right){\omega }_K\left({\overline{O}}_K,{\overline{A}}_K\right)\right].$$

This shows part 2 because B is an arbitrary Borel set.

2.2. Proof of the Assertions in Section 3.2, ORR-I

2.2.1. Proof of Item (a)

Lemma 1, part 2 implies that the densities $p_g^{\text{marg}}$ factors as

$$p_g^{\text{marg}}\left(o,a\right)=\prod _{j=0}^K{I}_{\left\{g_j\left({\overline{o}}_j\right)\right\}}\left(a_j\right)\prod _{j=1}^{K+1}{p}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{a}}_{j-1}\right)p^{\text{marg}}\left(o_0\right).$$

In particular, the event $\left\{{\overline{A}}_{k-1}^g={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}^g\right)\right\}$ has probability 1. Consequently,

$$\begin{array}{l}{p}_g^{\text{marg}}\left(o,a|{\overline{o}}_k\right)=\prod _{j=0}^K{I}_{\left\{g_j\left({\overline{o}}_j\right)\right\}}(a_j)p_g^{\text{marg}}\left(o,a|{\overline{o}}_k,{\overline{a}}_k={\overline{g}}_k\left({\overline{o}}_k\right)\right)\\ =\prod _{j=0}^K{I}_{\left\{g_j\left({\overline{o}}_j\right)\right\}}(a_j)\prod _{j=k+1}^{K+1}{p}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{a}}_{j-1}={\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right).\end{array}$$

Therefore,

$$\begin{array}{l}E\left\{u\left(O^g,A^g\right)|{\overline{O}}_k^g={\overline{o}}_k\right\}=\\ =\sum _{\underset{l=0,\dots ,K}{a_l\in {\mathcal{A}}_l}}\int u\left(o,a\right)\prod _{j=0}^K{I}_{\left\{g_j\left({\overline{o}}_j\right)\right\}}(a_j)\prod _{j=k+1}^{K+1}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\\ =\int u\left(o,a\right)\left[\sum _{\underset{l=0,\dots ,K}{a_l\in {\mathcal{A}}_l}}\prod _{j=0}^K{I}_{\left\{g_j\left({\overline{o}}_j\right)\right\}}(a_j)\right]\prod _{j=k+1}^{K+1}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\\ =\int u\left(o,a\right)\prod _{j=k+1}^{K+1}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\\ =\int \prod _{j=k+1}^{K}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\times \\ \left[\int u\left(o,a\right)d{P}_{O_{K+1}|{\overline{O}}_K,{\overline{A}}_K}^{\text{marg}}\left(o_{K+1}|{\overline{o}}_K,{\overline{a}}_K={\overline{g}}_K\left({\overline{o}}_K\right)\right)\right]\\ =\int \left[\prod _{j=k+1}^{K}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\right]{\phi }_{K+1}\left({\overline{o}}_K\right)\\ =\int \left[\prod _{j=k+1}^{K-1}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right)\right]\times \\ \left[\int {\phi }_{K+1}\left({\overline{o}}_K\right)d{P}_{O_K|{\overline{O}}_{K-1},{\overline{A}}_{K-1}}^{\text{marg}}\left(o_K|{\overline{o}}_{K-1},{\overline{g}}_{K-1}\left({\overline{o}}_{K-1}\right)\right)\right]\\ =\int \prod _{j=k+1}^{K-1}d{P}_{O_j|{\overline{O}}_{j-1},{\overline{A}}_{j-1}}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{g}}_{j-1}\left({\overline{o}}_{j-1}\right)\right){\phi }_{K+1}\left({\overline{o}}_K\right)\\ =\dots ={\phi }_{k+1}\left({\overline{o}}_k\right).\end{array}$$ (2)

2.2.2. Proof of Item (b)

Lemma 1, part 1 implies that

$$\begin{array}{l}\hspace{1em}E\left[u\left(O,A\right){\underset{\_}{\omega }}_{k-1,K}\left({\overline{O}}_K,{\overline{A}}_K\right)|{\overline{O}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}\right)\right]=\\ =E\left[u\left(O^g,A^g\right)|{\overline{O}}_k,{\overline{A}}_{k-1}={\overline{g}}_{k-1}\left({\overline{O}}_{k-1}\right)\right].\end{array}$$

The left hand side of this equality is equal to

$$\sum _{\underset{k=k,\dots ,K}{a_k\in {\mathcal{A}}_k}}\int u\left(o,a\right){\underset{\_}{\omega }}_{k-1,K}\left({\overline{o}}_K,{\overline{a}}_K\right)\prod _{j=k}^K{\lambda }_j\left(a_j|{\overline{o}}_j,{\overline{a}}_{j-1}\right)\prod _{j=k+1}^{K+1}d{P}^{\text{marg}}\left(o_j|{\overline{o}}_{j-1},{\overline{a}}_{j-1}\right)$$

and this coincides with the right hand side of (2) which, as we have just argued, is equal to φ_k+1 (ō_k).

2.3. Proof of Lemma 2 in ORR-I

Let X be the identity random element on $\left(\mathcal{X},\mathcal{A}\right)$ and let E_{P^marg × PX} (·) stand for the expectation operation computed under the product law P^marg × P_X for the random vector (O, A, X). Then the restriction stated in 2) is equivalent to

$$E_{P^{\text{marg}}\times P_X}\left[b\left(X,Z\right){\omega }_K\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\text{par}}\left(X,Z;{\beta }^{*}\right)\right\}\right]=0\text{ for all }b$$ (3)

and the restriction stated in 3) is equivalent to

$$\begin{array}{l}{E}_{P^{\text{marg}}\times P_X}[\left\{b\left(X,Z\right)-E_{P\times P_X}\left[b\left(X,Z\right)|Z\right]\right\}\times \\ {\omega }_K\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\text{sem}}\left(X,Z;{\beta }^{*}\right)\right\}]=0\text{ for all }b.\end{array}$$ (4)

To show 2) let d (O, A, X) ≡ ω_K (Ō_K, Ā_K) {u (O, A) – h_par (X, Z, β*)}.

(ORR-I, (14)) ⇒ (3).

$$\begin{array}{l}{E}_{P^{\text{marg}}\times P_X}\left[b\left(X,Z\right)d\left(O,A,X\right)\right]=E_{P^{\text{marg}}\times P_X}\left[b\left(X,Z\right)E_{P\times P_X}\left[d\left(O,A,X\right)|X,Z\right]\right]\\ =0\end{array}$$

where the last equality follows because E_{P^marg × PX} [d (O, A, X) |X = x, Z] = E_P^marg [d (O, A, x) |Z] by independence of (O, A) with X under the law P^marg × P_X and, by assumption, E_P^marg [d (O, A, x) |Z] = 0 μ-a.e.(x) and hence E_P^marg [d (O, A, x) |Z] because P_X and μ are mutually absolute continuous.

(3) ⇒ (ORR-I, (14)). Define b^* (X; Z) = E_{P^marg × PX} [d(O, A, X)|X, Z]. Then,

$$0=E_{P^{\text{marg}}\times P_X}\left[b^{*}\left(X,Z\right)d\left(O,A,X\right)\right]=E_{P^{\text{marg}}\times P_X}\left[E_{P\times P_X}{\left[d\left(O,A,X\right)|X,Z\right]}^2\right]$$

consequently, E_{P^marg × PX} [d (O, A, X) |X, Z] = 0 with P^marg × P_X prob. 1 which is equivalent to (ORR-I, (14)) because P_X is mutually absolutely continuous with μ.

To show 3) redefine d (O, A, X) as ω_K (Ō_K, Ā_K) {u (O, A) − h_sem (X, Z, β*)}.

(ORR-1, (15)) ⇒ (4)

$$\begin{array}{l}{E}_{P^{\text{marg}}\times P_X}\left[\left\{b\left(X,Z\right)-E_{P^{\text{marg}}\times P_X}\left[b\left(X,Z\right)|Z\right]\right\}d\left(O,A,X\right)\right]\\ =E_{P^{\text{marg}}\times P_X}\left[\left\{b\left(X,Z\right)-E_{P\times P_X}\left[b\left(X,Z\right)|Z\right]\right\}{E}_{P^{\text{marg}}\times P_X}\left\{d\left(O,A,X\right)|X,Z\right\}\right]\\ =E_{P^{\text{marg}}\times P_X}\left[\left\{b\left(X,Z\right)-E_{P^{\text{marg}}\times P_X}\left[b\left(X,Z\right)|Z\right]\right\}q\left(Z\right)\right]=0\end{array}$$

where the third equality follows because E_{P^marg × PX} {d (O, A, X) |X = x, Z} = E_P^marg {d (O, A, x) |Z} and E_P^marg {d (O, A, x) |Z}= q (Z) μ-a.e.(x) and hence P_X-a.e.(x) by absolute continuity.

(4) ⇒ (ORR-I, (15)). Define b^* (X; Z) = E_{P × PX} [ d(O, A, X)|X, Z]. Then,

$$\begin{array}{l}0=E_{P^{\text{marg}}\times P_X}\left[\left\{b^{*}\left(X,Z\right)-E_{P^{\text{marg}}\times P_X}\left[b^{*}\left(X,Z\right)|Z\right]\right\}d\left(O,A,X\right)\right]\\ =E_{P^{\text{marg}}\times P_X}\left[\left\{b^{*}\left(X,Z\right)-E_{P^{\text{marg}}\times P_X}\left[b^{*}\left(X,Z\right)|Z\right]\right\}{b}^{*}\left(X,Z\right)\right]\\ =E_{P^{\text{marg}}\times P_X}\left[{\left\{b^{*}\left(X,Z\right)-E_{P^{\text{marg}}\times P_X}\left[b^{*}\left(X,Z\right)|Z\right]\right\}}^2\right].\end{array}$$

Consequently, b^* (X, Z) = E_{P^marg × PX} [b^* (X, Z) |Z] ≡ q (Z) P_X – a.e. (X) and hence μ_X –a.e. (X) by absolute continuity. The result follows because b^* (x, Z) = E_{P^marg × PX} [d (O, A, X) |X = x, Z] = E_P^marg [d (O, A, X) |Z].

2.4. Derivation of Some Formulas in Section 5.3, ORR-I

2.4.1. Derivation of Formula (26) in ORR-I

Any element

$$\sum _{k=0}^K\left\{d_k\left({\overline{O}}_k,{\overline{A}}_k\right)-E\left[d_k\left({\overline{O}}_k,{\overline{A}}_k\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\right\}$$

of the set Λ is the sum of K + 1 uncorrelated terms because for any l, j such that 0 ≤ l < l + j ≤ K + 1,

$$\begin{array}{l}E[\left\{d_{l+j}\left({\overline{O}}_{l+j},{\overline{A}}_{l+j}\right)-E\left[d_{l+j}\left({\overline{O}}_{l+j},{\overline{A}}_{l+j}\right)|{\overline{O}}_{l+j},{\overline{A}}_{l+j-1}\right]\right\}\times \\ \left\{dl\left({\overline{O}}_l,{\overline{A}}_l\right)-E\left[d_l\left({\overline{O}}_l,{\overline{A}}_l\right)|{\overline{O}}_l,{\overline{A}}_{l-1}\right]\right\}]\\ =E[E\left[\left\{d_{l+j}\left({\overline{O}}_{l+j},{\overline{A}}_{l+j}\right)-E\left[d_{l+j}\left({\overline{O}}_{l+j},{\overline{A}}_{l+j}\right)|{\overline{O}}_{l+j},{\overline{A}}_{l+j-1}\right]\right\}|{\overline{O}}_{l+j},{\overline{A}}_{l+j-1}\right]\times \\ \left\{d_l\left({\overline{O}}_l,{\overline{A}}_l\right)-E\left[d_l\left({\overline{O}}_l,{\overline{A}}_l\right)|{\overline{O}}_l,{\overline{A}}_{l-1}\right]\right\}]\\ =E\left[0\times \left\{d_l\left({\overline{O}}_l,{\overline{A}}_l\right)-E\left[d_l\left({\overline{O}}_l,{\overline{A}}_l\right)|{\overline{O}}_l,{\overline{A}}_{l-1}\right]\right\}\right]=0.\end{array}$$

Thus, Λ is equal to Λ₀ ⊕ Λ₁ ⊕ . . . ⊕ Λ_K where

$${\mathrm{\Lambda }}_k\equiv \left\{d_k\left({\overline{O}}_k,{\overline{A}}_k\right)-E\left[d_k\left({\overline{O}}_k,{\overline{A}}_k\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]:d_k\text{ arbitrary scalar function}\right\}$$

and ⊕ stands for the direct sum operator. Then,

$$\mathrm{\Pi }\left[Q|\mathrm{\Lambda }\right]=\sum _{k=0}^K\mathrm{\Pi }\left[Q|{\mathrm{\Lambda }}_k\right]$$

and it can be easily checked that Π [Q|Λ_k] = E (Q|Ō_k, Ā_k) – E [Q|Ō_k, Ā_k–1].

2.4.2. Derivation of Formula (27) in ORR-I

Applying formula (26, in ORR-I) we obtain

$$\mathrm{\Pi }\left[S.\left(\beta ,{\gamma }^{*},b\right)|\mathrm{\Lambda }\right]=\sum _{k=0}^K\left\{E\left[S.\left(\beta ,{\gamma }^{*},b\right)|{\overline{O}}_k,{\overline{A}}_k\right]-E\left[S.\left(\beta ,{\gamma }^{*},b\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\right\}.$$

So, for k = 0, ..., K,

$$d_{\cdot ,opt,k}^b\left({\overline{O}}_k,{\overline{A}}_k\right)=E\left[S.\left(\beta ,{\gamma }^{*},b\right)|{\overline{O}}_k,{\overline{A}}_k\right].$$

But,

$$\begin{array}{l}\hspace{1em}E\left[S.\left(\beta ,{\gamma }^{*},b\right)|{\overline{O}}_k,{\overline{A}}_k\right]\\ ={\int }_{{\mathcal{X}}_{pos}}{b}_{\cdot }\left(x,z\right)E\left[{\omega }_K^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_k,{\overline{A}}_k\right]d{P}_X\left(x\right)\\ ={\int }_{{\mathcal{X}}_{pos}}{b}_{\cdot }\left(x,z\right){\omega }_k^x\left({\overline{O}}_k,{\overline{A}}_k\right)\times \\ \times E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_k,{\overline{A}}_k\right]d{P}_X\left(x\right)\\ ={\int }_{{\mathcal{X}}_{pos}}{b}_{\cdot }\left(x,z\right){\omega }_k^x\left({\overline{O}}_k,{\overline{A}}_k\right)\times \\ \times E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]d{P}_X\left(x\right).\end{array}$$

So formula ((27), ORR-I) is proved if we show that

$$\begin{array}{l}E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]=\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left\{{\phi }_{k+1}^x\left({\overline{O}}_k\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}.\end{array}$$ (5)

This follows immediately from the preceding proof of Result (b) of Section 3.2. Specifically, it was shown there that

$$E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_{k+1},{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]={\phi }_{k+2}\left({\overline{O}}_{k+1}\right).$$

Consequently, the left hand side of (5) is equal to

$$\begin{array}{l}\hspace{1em}\hspace{1em}E[E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)\left\{u\left(O,A\right)-h_{\cdot }\left(x,Z;\beta \right)\right\}|{\overline{O}}_{k+1},{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]|\\ \hspace{1em}\hspace{1em}{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)]\\ =E\left[{\phi }_{k+2}^x\left({\overline{O}}_{k+1}\right)|{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]\\ \hspace{1em}-h_{\cdot }\left(x,Z;\beta \right)E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)|{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]\\ ={\phi }_{k+1}^x\left({\overline{O}}_k\right)-h_{\cdot }\left(x,Z;\beta \right)\end{array}$$

where the last equality follows by the definition of ${\phi }_{k+1}^x\left({\overline{O}}_k\right)$ and the fact that $E\left[{\underset{\_}{\omega }}_{k,K}^x\left({\overline{O}}_K,{\overline{A}}_K\right)|{\overline{O}}_k,{\overline{A}}_k=g_x\left({\overline{O}}_k\right)\right]=1$ (as this is just the function ${\phi }_{k+1}^x\left({\overline{O}}_k\right)$ resulting from applying the integration to the utility u (O, A) = 1).

2.4.3. Derivation of Formula (31) in ORR-I

It suffices to show that $S_{aug}\left(\gamma ,d_{\cdot ,\beta ,\gamma ,\tau ,opt}^b\right)={\mathrm{\Sigma }}_{k=0}^K{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)M_k\left(x;\beta ,\gamma ,\tau \right)d{P}_X\left(x\right)$ where

$$M_k\left(x;\beta ,\gamma ,\tau \right)\equiv \left\{{\omega }_k^x\left(\gamma \right)-{\omega }_{k-1}^x\left(\gamma \right)\right\}\left\{{\phi }_{k+1}^x\left({\overline{O}}_k;\tau \right)-h_{\cdot }\left(x,Z;\beta \right)\right\}.$$

But by definition

$$\begin{array}{l}{S}_{aug}\left(\gamma ,d_{\cdot ,\beta ,\gamma ,\tau ,opt}^b\right)=\\ =\sum _{k=0}^K\left\{d_{\cdot ,\beta ,\gamma ,\tau ,opt,k}^b\left({\overline{O}}_k,{\overline{A}}_k\right)-E_{\gamma }\left[d_{\cdot ,\beta ,\gamma ,\tau ,opt,k}^b\left({\overline{O}}_k,{\overline{A}}_k\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\right\}\\ =\sum _{k=0}^K\{{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right){\omega }_k^x\left(\gamma \right)\left\{{\phi }_{k+1}^x\left({\overline{O}}_k;\tau \right)-h_{\cdot }\left(x,Z;\beta \right)\right\}d{P}_X\left(x\right)-\\ \hspace{1em}-E\left[{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right){\omega }_k^x\left(\gamma \right)\left\{{\phi }_{k+1}^x\left({\overline{O}}_k;\tau \right)-h_{\cdot }\left(x,Z;\beta \right)\right\}d{P}_X\left(x\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\}\\ =\sum _{k=0}^K{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)\left\{{\omega }_k^x\left(\gamma \right)-E_{\gamma }\left[{\omega }_k^x\left(\gamma \right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\right\}\times \\ \hspace{1em}\hspace{1em}\hspace{1em}\left\{{\phi }_{k+1}^x\left({\overline{O}}_k;\tau \right)-h_{\cdot }\left(x,Z;\beta \right)\right\}d{P}_X\left(x\right)\\ =\sum _{k=0}^K{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)\left\{{\omega }_k^x\left(\gamma \right)-{\omega }_{k-1}^x\left(\gamma \right)\right\}\left\{{\phi }_{k+1}^x\left({\overline{O}}_k;\tau \right)-h_{\cdot }\left(x,Z;\beta \right)\right\}d{P}_X\left(x\right)\end{array}$$

where the last equality follows because

$$\begin{array}{l}{E}_{\gamma }\left[{\omega }_k^x\left(\gamma \right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]=\\ ={\omega }_{k-1}^x\left(\gamma \right)E_{\gamma }\left[\frac{I_{\left\{g_{x,k}\left({\overline{O}}_k\right)\right\}}\left(A_k\right)}{{\lambda }_k\left(A_k|{\overline{O}}_k,{\overline{A}}_{k-1}\right)}|{\overline{O}}_k,{\overline{A}}_{k-1}\right]\\ ={\omega }_{k-1}^x\left(\gamma \right)\frac{E_{\gamma }\left[I_{\left\{g_{x,k}\left({\overline{O}}_k\right)\right\}}\left(A_k\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right]}{{\lambda }_k\left(g_{x,k}\left({\overline{O}}_k\right)|{\overline{O}}_k,{\overline{A}}_{k-1}\right)}={\omega }_{k-1}^x\left(\gamma \right).\end{array}$$

2.5. Proof that b_{·, opt} is Optimal

Write for short, β̂_· (b) ≡ β̂_· (b, d̂_{·, opt}),

$$\begin{array}{l}{Q}_{par}\left(b\right)\equiv {\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)Q_{par}\left(x;{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)d{P}_X\left(x\right)\text{ and}\\ Q_{sem}\left(b\right)\equiv {\int }_{{\mathcal{X}}_{pos}}\left\{b\left(x,Z\right)-\overline{b}\left(Z\right)\right\}{Q}_{sem}\left(x;{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)d{P}_X\left(x\right).\end{array}$$

We will show that J_· (b) = E {Q_· (b) Q_· (b_{·, opt})^′} for · = par and · = sem. When either model (16, ORR-I) or (29, ORR-I) are correct, β^* = β^†. Consequently, for · = par we have that J_par (b) is equal to

$$\begin{array}{l}-E\left\{{{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)\frac{\partial }{\partial \beta }{h}_{\text{par}}\left(x,Z;\beta \right)|}_{{\beta }^{†}}d{P}_X\left(x\right)\right\}\\ =E[{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)d{P}_X\left(x\right)\times \\ \times \left\{{\int }_{{\mathcal{X}}_{pos}}{b}_{par,opt}\left(x,Z\right)E\left\{Q_{par}\left(x;{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)Q_{par}{\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)}^{\prime }|Z\right\}d{P}_X\left(\tilde{x}\right)\right\}]\\ =E[\left\{{\int }_{{\mathcal{X}}_{pos}}b\left(x,Z\right)Q_{par}\left(x;{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)d{P}_X\left(x\right)\right\}\times \\ \times \left\{{\int }_{{\mathcal{X}}_{pos}}{b}_{par,opt}\left(x,Z\right)Q_{par}{\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)}^{\prime }d{P}_X\left(\tilde{x}\right)\right\}]\\ =E\left\{Q_{par}\left(b\right)Q_{par}{\left(b_{par,opt}\right)}^{\prime }\right\}.\end{array}$$

For · = sem and with the definitions b͂ (x, Z) ≡ b (x, Z) – b̄ (Z) and Q͂_sem (x͂; β^†, γ^†, τ^†) ≡ Q_sem (x͂; β^†, γ^†, τ^†) – Q̄_sem (x͂; β^†, γ^†, τ^†), the same argument yields J_sem (b) equal to

$$\begin{array}{l}E[\left\{{\int }_{{\mathcal{X}}_{pos}}\tilde{b}\left(x,Z\right){\tilde{Q}}_{\mathit{\text{sem}}}\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)d{P}_X\left(x\right)\right\}\\ \times \left\{{\int }_{{\mathcal{X}}_{pos}}{\tilde{b}}_{sem,opt}\left(x,Z\right){\tilde{Q}}_{sem}{\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)}^{\prime }d{P}_X\left(\tilde{x}\right)\right\}]\\ =E[\left\{{\int }_{{\mathcal{X}}_{pos}}\tilde{b}\left(x,Z\right)Q_{sem}\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)d{P}_X\left(x\right)\right\}\\ \left\{{\int }_{{\mathcal{X}}_{pos}}{\tilde{b}}_{sem,opt}\left(x,Z\right)Q_{sem}{\left(\tilde{x};{\beta }^{†},{\gamma }^{†},{\tau }^{†}\right)}^{\prime }d{P}_X\left(\tilde{x}\right)\right\}]\\ =E\left\{Q_{sem}\left(b\right)Q_{sem}{\left(b_{par,opt}\right)}^{\prime }\right\}.\end{array}$$

Now, with var_A (β̂_· (b)) denoting the asymptotic variance of β̂_· (b), we have that from expansion ((32) in ORR-I)

$$va{r}_{\mathit{\text{A}}}\left({\widehat{\beta }}_{\cdot }\left(b\right)\right)=\mathit{\text{var}}\left\{E{\left[Q_{\cdot }\left(b\right)Q_{\cdot }{\left(b_{\cdot ,opt}\right)}^{\prime }\right]}^{-1}{Q}_{\cdot }\left(b\right)\right\}$$

and consequently

$$\begin{array}{l}co{v}_A\left({\widehat{\beta }}_{\cdot }\left(b\right),{\widehat{\beta }}_{\cdot }\left(b_{\cdot ,opt}\right)\right)=\\ =E{\left[Q_{\cdot }\left(b\right)Q_{\cdot }{\left(b_{\cdot ,opt}\right)}^{\prime }\right]}^{-1}\mathit{\text{cov}}\left(Q_{\cdot }\left(b\right),Q_{\cdot }\left(b_{\cdot ,opt}\right)\right)E{\left[Q_{\cdot }{\left(b_{\cdot ,opt}\right)}^{\otimes 2}\right]}^{-1}\\ =E{\left[Q_{\cdot }{\left(b_{\cdot ,opt}\right)}^{\otimes 2}\right]}^{-1}=va{r}_A\left({\widehat{\beta }}_{\cdot }\left(b_{\cdot ,opt}\right)\right).\end{array}$$

Thus, 0 ≤ var_A (β̂_· (b) – β̂_· (b_{·, opt})) = var_A (β̂_· (b)) + var_A (β̂_· (b_{·, opt}) – 2cov_A (β̂_· (b), β̂_· (b_{·, opt})) = var_A (β̂_· (b)) – var_A (β̂_· (b_{·, opt})) which concludes the proof.

3. Confidence Set for x_opt (z) when $\mathcal{X}$ is Finite and h_· (z, x; β) is Linear in β

We first prove the assertion that the computation of the confidence set B_b entails an algorithm for determining if the intersection of $\#\left(\mathcal{X}\right)-1$ half spaces in ℜ^p and a ball in ℜ^p centered at the origin is non-empty. To do so, first note that linearity implies that $h_{\cdot }\left(z,x;\beta \right)={\mathrm{\Sigma }}_{j=1}^p{s}_j\left(x,z\right){\beta }_j$ for some fixed functions s_j, j = 1, ..., p. Let $N=\#\left(\mathcal{X}\right)$ and write $\mathcal{X}=\left\{x_1,\dots ,x_N\right\}$. The point x_l is in B_b iff

$$\text{there exists} \beta \text{ in }{C}_b:\sum _{j=1}^p\left[s_j\left(x_l,z\right)-s_j\left(x_k,z\right)\right]{\beta }_i\ge 0\text{ for all }{x}_k\in \mathcal{X}-\left\{x_l\right\}.$$ (6)

Define the p × 1 vector ${\mathbf{\text{v}}}_l^k$ whose j^th entry is equal to s_j (x_l, z) – s_j (x_k, z), j = 1, ..., p. Define also the vectors ${\mathbf{\text{v}}}_l^{*k}={\mathbf{\text{v}}}_l^k{}^{\prime }{\widehat{\mathrm{\Gamma }}}_{\cdot }\left(b\right)$ and the constants $a_l^k={\mathbf{\text{v}}}_l^k{}^{\prime }{\widehat{\beta }}_{\cdot }\left(b,{\widehat{d}}_{\cdot ,opt}^b\right)$. Then ${\mathrm{\Sigma }}_{j=1}^p\left[s_j\left(x_l,z\right)-s_j\left(x_k,z\right)\right]{\beta }_j>0$ iff ${\mathbf{\text{v}}}_j^{*k}{\widehat{\mathrm{\Gamma }}}_{\cdot }{\left(b\right)}^{-1/2}\times \left(\beta -{\widehat{\beta }}_{\cdot }\left(b,{\widehat{d}}_{\cdot ,opt}^b\right)\right)>a_l^k$. Noting that β in C_b iff ${\widehat{\mathrm{\Gamma }}}_{\cdot }{\left(b\right)}^{-1/2}\left(\beta -{\widehat{\beta }}_{\cdot }\left(b,{\widehat{d}}_{\cdot ,opt}^b\right)\right)$ is in the ball

$$\mathcal{U}\equiv \left\{\mathbf{\text{u}}\in {ℝ}^p:{\mathbf{\text{u}}}^{\prime }\mathbf{\text{u}}\le {\chi }_{p,1-\mathit{\alpha }}^2\right\}$$

we conclude that the condition in the display (6) is equivalent to

$$\text{there exists }\mathbf{\text{u}}\text{ in }\mathcal{U}\text{ such that }{\mathbf{\text{v}}}_l^{*k}{}^{\prime }\mathbf{\text{u}}>a_l^k\text{ for }k=1,\dots ,N,k\ne l.$$

The set $\left\{\mathbf{\text{u}}\in {ℝ}^p:{\mathbf{\text{v}}}_j^{*k}{}^{\prime }\mathbf{\text{u}}=a_l^k\right\}$ is a hyper-plane in ℜ^p which divides the Euclidean space ℜ^p into two half-spaces, one of which is $\left\{\mathbf{\text{u}}\in {ℝ}^p:{\mathbf{\text{v}}}_j^{*k}{}^{\prime }\mathbf{\text{u}}>a_l^k\right\}$. Thus, the condition in the last display imposes that the intersection of N – 1 half-spaces (each one defined by the condition ${\mathbf{\text{v}}}_l^{*k}{}^{\prime }\mathbf{\text{u}}>a_l^k$ for each k) and the ball $\mathcal{U}$ is non-empty.

Turn now to the construction of a confidence set $B_b^{*}$ that includes B_b. Our construction relies on the following Lemma.

Lemma. Let

$$\mathcal{D}=\left\{\mathbf{\text{u}}\in {\mathbf{\text{R}}}^p:{\left(\mathbf{\text{u}}-{\mathbf{\text{u}}}_0\right)}^{\prime }{\mathrm{\Sigma }}^{-1}\left(\mathbf{\text{u}}-{\mathbf{\text{u}}}_0\right)\le c_0\right\}$$

where u₀ is a fixed p × 1 real valued vector and Σ is a fixed non-singular p × p matrix.

Let α be a fixed, non-null, p×1 real valued vector. Let τ₀ ≡ α′ u₀ and α^* = Σ^1/2α. Assume that α₁ ≠ 0. Let, ${\mathbf{\text{v}}}_1^{*}$ be the p×1 vector ${\left(-{\mathit{\alpha }}_1^{*-1}{\tau }_0,0,\dots ,0\right)}^{\prime }$. Let ϒ be the linear space generated by the p×1 vectors ${\mathbf{\text{v}}}_2^{*}={\left({\mathit{\alpha }}_1^{*-1}{\mathit{\alpha }}_2^{*},1,0,0,\dots ,0\right)}^{\prime }$, ${\mathbf{\text{v}}}_3^{*}={\left({\mathit{\alpha }}_1^{*-1}{\mathit{\alpha }}_3^{*},0,1,0,\dots ,0\right)}^{\prime },\dots ,{\mathbf{\text{v}}}_p^{*}={\left({\mathit{\alpha }}_1^{*-1}{\mathit{\alpha }}_p^{*},0,0,0,\dots 1\right)}^{\prime }$ and define

$$\begin{array}{l}{\mathbf{\text{v}}}_{1,proj}^{*}={\mathbf{\text{v}}}_1^{*}-\mathrm{\Pi }\left[{\mathbf{\text{v}}}_1^{*}|\mathrm{\Upsilon }\right]\\ ={\mathbf{\text{v}}}_1^{*}-{\mathbf{\text{V}}}^{*}{\left({\mathbf{\text{V}}}^{*}{}^{\prime }{\mathbf{\text{V}}}^{*}\right)}^{-1}{\mathbf{\text{V}}}^{*}{}^{\prime }{\mathbf{\text{v}}}_1^{*}\end{array}$$

where

$${\mathbf{\text{V}}}^{*}=\left({\mathbf{\text{v}}}_2^{*},\dots ,{\mathbf{\text{v}}}_p^{*}\right).$$

Then there exists $\mathbf{\text{u}}\in \mathcal{D}$ satisfying

$${\mathit{\alpha }}^{\prime }\mathbf{\text{u}}=0$$

if and only if

$$c_0-{\Vert {\mathbf{\text{v}}}_{1,proj}^{*}\Vert }^2\ge 0.$$

Proof

$${\mathit{\alpha }}^{\prime }\mathbf{\text{u}}=0\iff {\mathit{\alpha }}^{\prime }{\mathrm{\Sigma }}^{1/2}{\mathrm{\Sigma }}^{-1/2}\left(\mathbf{\text{u}}-{\mathbf{\text{u}}}_0\right)=-{\mathit{\alpha }}^{\prime }{\mathbf{\text{u}}}_0.$$

Then, with τ₀ ≡ −α′ u₀ and α* = Σ^1/2α, we conclude that there exists $\mathbf{\text{u}}\in \mathcal{D}$ satisfying α′ u = 0 if and only if there exists u* ∈ R^p such that

$${\mathbf{\text{u}}}^{*}{}^{\prime }{\mathbf{\text{u}}}^{*}\le c_0\text{ and }-{\mathit{\alpha }}^{*}{}^{\prime }{\mathbf{\text{u}}}^{*}={\tau }_0.$$

Now, by the assumption ${\mathit{\alpha }}_1^{*}\ne 0$ we have −α*^′ u* = τ₀ iff $u_1=-{\mathit{\alpha }}_1^{*-1}\times \left[{\tau }_0+{\mathrm{\Sigma }}_{j=2}^p{\mathit{\alpha }}_j^{*}{u}_j^{*}\right]$. Thus, the collection of all vectors u* satisfying −α*^′ u* = τ₀ is the linear variety

$${\mathbf{\text{v}}}_1^{*}+\mathrm{\Upsilon }={\mathbf{\text{v}}}_{1,proj}^{*}+\mathrm{\Upsilon }$$

where ${\mathbf{\text{v}}}_j^{*}{}^{\prime }s$ and ϒ are defined in the statement of the lemma. The vector ${\mathbf{\text{v}}}_{1,proj}^{*}$ is the residual from the (Euclidean) projection of ${\mathbf{\text{v}}}_1^{*}$ into the space ϒ.

Thus, −α*^′ u* τ₀ iff ${\mathbf{\text{u}}}^{*}={\mathbf{\text{v}}}_{1,proj}^{*}+{\mathbf{\text{v}}}_{\mathrm{\Upsilon }}^{*}$ for some ${\mathbf{\text{v}}}_{\mathrm{\Upsilon }}^{*}\in \mathrm{\Upsilon }$. Consequently, by the orthogonality of ${\mathbf{\text{v}}}_{1,proj}^{*}$ with ϒ we have that for u* satisfying −α*^′ u* = τ₀ it holds that

$$\begin{array}{l}{\mathbf{\text{u}}}^{*}{}^{\prime }{\mathbf{\text{u}}}^{*}={\Vert {\mathbf{\text{u}}}^{*}\Vert }^2\\ \hspace{1em}={\Vert {\mathbf{\text{v}}}_{1,proj}^{*}\Vert }^2+{\Vert {\mathbf{\text{v}}}_{\mathrm{\Upsilon }}^{*}\Vert }^2.\end{array}$$

Therefore, since ${\Vert {\mathbf{\text{v}}}_{\mathrm{\Upsilon }}^{*}\Vert }^2$ is unrestricted,

$${\mathbf{\text{u}}}^{*}{}^{\prime }{\mathbf{\text{u}}}^{*}\le c_0\text{ for some }{\mathbf{\text{u}}}^{*}\text{ satisfying }-{\mathit{\alpha }}^{*}{}^{\prime }{\mathbf{\text{u}}}^{*}={\tau }_0$$

if and only if

$$c_0-{\Vert {\mathbf{\text{v}}}_{1,proj}^{*}\Vert }^2\ge 0.$$ (7)

This concludes the proof of the Lemma.

To construct the set $B_b^{*}$ we note that the condition in the display (6) implies the negation, for every subset ${\mathcal{X}}_{(-l)}$ of $\mathcal{X}-\left\{x_l\right\}$, of the statement

$$\sum _{j=1}^p\sum _{k\in {\mathcal{X}}_{(-1)}}\left[s_j\left(x_l,z\right)-s_j\left(x_k,z\right)\right]{\beta }_j<0\text{ for all }\beta \in C_b.$$ (8)

Thus, suppose that for a given x_l we find that (8) holds for some subset ${\mathcal{X}}_{\left(-1\right)}$ of $\mathcal{X}-\left\{x_l\right\}$, then we know that x_l cannot be in B_b. The proposed confidence set $B_b^{*}$ is comprised by the points in $\mathcal{X}$ for which condition (8) cannot be negated for all subsets ${\mathcal{X}}_{\left(-1\right)}$. The set $B_b^{*}$ is conservative (i.e. it includes B_b but is not necessarily equal to B_b) because the simultaneous negation of the statement (8) for all ${\mathcal{X}}_{\left(-l\right)}$ does not imply the statement (6). To check if condition (8) holds for any given subset ${\mathcal{X}}_{\left(-l\right)}$ and x_l, we apply the result of Lemma as follows. We define the vector α ∈ ℝ^p whose j^th component is equal to ${\mathrm{\Sigma }}_{k\in {\mathcal{X}}_{\left(-l\right)}}\left[s_j\left(x_l,z\right)-s_j\left(x_k,z\right)\right]$, j = 1,..., p and the vector ${\mathbf{\text{u}}}_0={\widehat{\beta }}_{\cdot }\left(b,{\widehat{d}}_{\cdot ,opt}^b\right)\in {ℝ}^p$. We also define the constant $c_0={\chi }_{p,1-\mathit{\alpha }}^2$, and the matrix Σ = Γ̂_· (b). We compute the vectors ${\mathit{\alpha }}^{*}={\mathrm{\Sigma }}^{1/2}\mathit{\alpha }$, ${\mathbf{\text{v}}}_1^{*},\dots ,{\mathbf{\text{v}}}_p^{*}$ and the matrix V* as defined in Lemma. We then check if the condition (7) holds. If it holds then this implies that the hyperplane comprised by the set of β’s that satisfy the condition in display (8) with the < sign replaced by the = sign, intersects the confidence ellipsoid C_b, in which case we know that (8) is false. If it does not hold, then we check if condition

$$\sum _{j=1}^p\sum _{k\in {\mathcal{X}}_{\left(-1\right)}}\left[s_j\left(x_l,z\right)-s_j\left(x_k,z\right)\right]{\widehat{\beta }}_{\cdot }{\left(b,{\widehat{d}}_{\cdot ,opt}^b\right)}_j<0$$ (9)

holds. If (9) does not hold, then we conclude that (8) is false for this choice of ${\mathcal{X}}_{\left(-1\right)}$. If (9) holds, then we conclude that (8) is true and we then exclude x_l from the set $B_b^{*}$.

4. Positivity Assumption: Example

Suppose that K = 1 and that $R_k=R_k^g=1$ with probability 1 for k = 0, 1, so that no subject dies in neither the actual world nor in the hypothetical world in which g is enforced in the population. Thus, for k = 0, 1, O_k = L_k since both T_k and R_k are deterministic and hence can be ignored. Suppose that L_k and A_k are binary variables (and so are therefore $A_k^g$ and $L_k^g$) and that the treatment regime g specifies that

$$g_0\left(l_0\right)=1-l_0\text{ and }{g}_1\left(l_0,l_1\right)=l_0\left(1-l_1\right).$$

Assume that

$$0<\mathbb{P}\left(L_0^g=l_0,L_1^g=l_1\right)<1,l_0=0,1;l_1=0,1.$$ (10)

Assumption PO imposes two requirements,

$$\mathbb{P}\left[{\lambda }_0\left(A_0^g|L_0^g\right)>0\right]=1\text{ and}$$ (11)

$$\mathbb{P}\left[{\lambda }_1\left(A_1^g|L_0^g,L_1^g,A_0^g\right)>0\right]=1.$$ (12)

Because by definition of regime g, $A_0^g=1-L_0^g$, then requirement (11) can be re-expressed as

$$1=\mathbb{P}\left(L_0^g=0\right)I_{\left(0,1\right]}\left({\lambda }_0\left(1|0\right)\right)+\mathbb{P}\left(L_0^g=1\right)I_{\left(0,1\right]}\left({\lambda }_0\left(0|1\right)\right).$$

Since indicators can only take the values 0 or 1 and $\mathbb{P}\left(L_0^g=l_0\right)<1$, l₀ = 0, 1 (by assumption (10)), the preceding equality is equivalent to

$$I_{\left(0,1\right]}\left({\lambda }_0\left(1|0\right)\right)=1\text{ and }{I}_{\left(0,1\right]}\left({\lambda }_0\left(0|1\right)\right)=1,$$

that is to say,

$${\lambda }_0\left(1|0\right)>0\text{ and }{\lambda }_0\left(0|1\right)>0.$$

By the definition of λ₀ (·|·) (see (3) in ORR-I), the last display is equivalent to

$$\mathbb{P}\left(A_0=1|L_0=0\right)>0\text{ and }\mathbb{P}\left(A_0=0|L_0=1\right)>0.$$ (13)

Likewise, because $A_1^g=L_0^g\left(1-L_1^g\right)$, and because $\mathbb{P}\left(L_0^g=l_0,L_1^g=l_1,A_0^g=l_0\right)=0$ by the fact that $A_0^g=1-L_0$, requirement (12) can be re-expressed as

$$\begin{array}{l}1=\mathbb{P}\left(L_0^g=0,L_1^g=0,A_0^g=1\right)I_{\left(0,1\right]}\left({\lambda }_1\left(0|0,0,1\right)\right)\\ +\mathbb{P}\left(L_0^g=0,L_1^g=1,A_0^g=1\right)I_{\left(0,1\right]}\left({\lambda }_1\left(0|0,1,1\right)\right)\\ +\mathbb{P}\left(L_0^g=1,L_1^g=0,A_0^g=0\right)I_{\left(0,1\right]}\left({\lambda }_1\left(1|1,0,0\right)\right)\\ +\mathbb{P}\left(L_0^g=1,L_1^g=1,A_0^g=0\right)I_{\left(0,1\right]}\left({\lambda }_1\left(0|1,1,0\right)\right)\end{array}$$

or equivalently, (again because the events $\left(L_0^g=l_0,L_1^g=l_1,A_0^g=1-l_0\right)$ and $\left(L_0^g-l_0,L_1^g-l_1\right)$ have the same probability by $\mathbb{P}\left(L_0^g=l_0,L_1^g=l_1,A_0^g=l_0\right)=0$,

$$\begin{array}{l}1=\mathbb{P}\left(L_0^g=0,L_1^g=0\right)I_{\left(0,1\right]}\left({\lambda }_1\left(0|0,0,1\right)\right)+\mathbb{P}\left(L_0^g=0,L_1^g=1\right)\\ \hspace{0.17em}\hspace{0.17em}\times I_{\left(0,1\right]}\left({\lambda }_1\left(0|0,1,1\right)\right)+\mathbb{P}\left(L_0^g=1,L_1^g=0\right)I_{\left(0,1\right]}\left({\lambda }_1\left(1|1,0,0\right)\right)\\ \hspace{0.17em}\hspace{0.17em}+\mathbb{P}\left(L_0^g=1,L_1^g=1\right)I_{\left(0,1\right]}\left({\lambda }_1\left(0|1,1,0\right)\right).\end{array}$$

Under the assumption (10), the last display is equivalent to

$$\begin{array}{l}{\lambda }_1\left(0|0,0,1\right)>0,{\lambda }_1\left(0|0,1,1\right)>0,\\ {\lambda }_1\left(1|1,0,0\right)>0\text{ and }{\lambda }_1\left(0|1,1,0\right)>0\end{array}$$

which, by the definition of λ₀ (·|·, ·, ·) in ((3), ORR-I), is, in turn, the same as

$$\begin{array}{l}\mathbb{P}\left(A_1=0|L_0=0,L_1=0,A_0=1\right)>0,\mathbb{P}\left(A_1=0|L_0=0,L_1=1,A_0=1\right)>0\\ \mathbb{P}\left(A_1=1|L_0=1,L_1=0,A_0=0\right)>0,\mathbb{P}\left(A_1=0|L_0=1,L_1=1,A_0=0\right)>0.\end{array}$$ (14)

We conclude that in this example, the assumption PO is equivalent to the conditions (13) and (14). We will now analyze what these conditions encode.

Condition (13) encodes two requirements:

• i) the requirement that in the actual world there exist subjects with L₀ = 1 and L₀ = 0 (i.e. that the conditioning events L₀ = 1 and L₀ = 0 have positive probabilities), for otherwise at least one of the conditional probabilities in (13) would not be defined, and

• ii) the requirement that in the actual world there be subjects with L₀ = 0 that take treatment A₀ = 1 and subjects with L₀ = 1 that take treatment A₀ = 0, for otherwise at least one of the conditional probabilities in (13) would be 0.

Condition i) is automatically satisfied, i.e. it does not impose a restriction on the law of L₀, by the fact that $L_0^g=L_0$ (since baseline covariates cannot be affected by interventions taking place after baseline) and the fact that we have assumed that $\mathbb{P}\left(L_0^g=l_0\right)>0$, l₀ = 0, 1.

Condition ii) is indeed a non-trivial requirement and coincides with the interpretation of the PO assumption given in section 3.1 for the case k = 0. Specifically, in the world in which g were to be implemented there would exist subjects with L₀ = 0. In such world the subjects with L₀ = 0 would take treatment $A_0^g=1$, then the PO assumption for k = 0 requires that in the actual world there also be subjects with L₀ = 0 that at time 0 take treatment A₀ = 1. Likewise the PO condition also requires that for k = 0 the same be true with 0 and 1 reversed in the right hand side of each of the equalities of the preceding sentence. A key point is that (11) does not require that in the observational world there be subjects with L₀ = 0 that take A₀ = 0, nor subjects with L₀ = 1 that take A₁ = 1. The intuition is clear. If we want to learn from data collected in the actual (observational) world what would happen in the hypothetical world in which everybody obeyed regime g, we must observe people in the study that obeyed the treatment at every level of L₀ for otherwise if, say, nobody in the actual world with L₀ = 0 obeyed regime g there would be no way to learn what the distribution of the outcomes for subjects in that stratum would be if g were enforced. However, we don t care that there be subjects with L₀ = 0 that do not obey g, i.e. that take A₀ = 0, because data from those subjects are not informative about the distribution of outcomes when g is enforced.

Condition (14) encodes two requirements:

• iii) the requirement that in the actual world there be subjects in the four strata (L₀ = 0, L₁ = 0, A₀ = 1), (L₀ = 0, L₁ = 1, A₀ = 1), (L₀ = 1, L₁ = 0, A₀ = 0) and (L₀ = 1, L₁ = 1, A₀ = 0) (i.e. that the conditioning events in the display (14) have positive probabilities), for otherwise at least one of the conditional probabilities would not be defined, and

• iv) the requirement that in the actual world there be subjects in every one of the strata (L₀ = 0, L₁ = 0, A₀ = 1), (L₀ = 0, L₁ = 1, A₀ = 1), (L₀ = 1, L₁ = 1, A₀ = 0) that have A₁ = 0 at time 1 and the requirement that there be subjects in stratum (L₀ = 1, L₁ = 0, A₀ = 0) that have A₁ = 1 at time 1, for otherwise at least one of the conditional probabilities in (14) would be 0.

Given condition ii) and the sequential randomization (SR) and consistency (C) assumptions, condition iii) is automatically satisfied, i.e. it does not impose a further restriction on the joint distribution of (L₀, L₁, A₀). To see this, first note that by condition (ii) the strata (L₀ = 0, A₀ = 1) and (L₀ = 1, A₀ = 0) are non-empty. So condition (iii) is satisfied provided

$$\mathbb{P}\left(L_1=l_1|L_0=0,A_0=1\right)>0\text{ and }\mathbb{P}\left(L_1=l_1|L_0=1,A_0=0\right)>0\text{ for }{l}_1=0,1.$$

But

$$\begin{array}{l}\mathbb{P}\left(L_1=l_1|L_0=0,A_0=1\right)=\mathbb{P}\left(L_1^g=l_1|L_0=0,A_0=1\right)\text{ by assumption (C)}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=\mathbb{P}\left(L_1^g=l_1|L_0=0\right)\text{ by assumption (SR)}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=\mathbb{P}\left(L_1^g=l_1|L_0^g=0\right)\text{ by assumption (C)}\end{array}$$

and $\mathbb{P}\left(L_1^g=l_1|L_0^g=0\right)>0$ by (10). An analogous argument shows that $\mathbb{P}\left(L_1=l_1|L_0=1,A_0=0\right)>0$. Finally, condition (iv) is a formalization our interpretation of assumption PO in section 3.1 for k = 1. In the world in which g was implemented there would exist subjects that would have all four combination of values for $\left(L_0^g,L_1^g\right)$. However, subjects with $L_0^g=l_0$ will only have $A_0^g=1-l_0$, so in this hypothetical world we will see at time 1 only four possible recorded histories, $\left(L_0^g=0,L_1^g=0,A_0^g=1\right)$, $\left(L_0^g=0,L_1^g=1,A_0^g=1\right)$, $\left(L_0^g=1,L_1^g-0,A_0^g=0\right)$ and $\left(L_0^g=1,L_1^g=1,A_0^g=0\right)$. In this hypothetical world subjects with any of the first three possible recorded histories will take $A_1^g=0$ and subjects with the last one will take $A_1^g=1$. Thus, in the actual world we must require that there be subjects in each of the first three strata (L₀ = 0, L₁ = 0, A₀ = 1), (L₀ = 0, L₁ = 1, A₀ = 1), (L₀ = 1, L₁ = 0, A₀ = 0) that take A₁ = 0 and subjects in the stratum (L₀ = 1, L₁ = 1, A₀ = 0) that take A₁ = 1. A point of note is that we don t make any requirement about the existence of subjects in strata other than the four mentioned in (iii) or about the treatment that subjects in these remaining strata take. The reason is that subjects that are not in the four strata of condition (iii) have already violated regime g at time 0 so they are uninformative about the outcome distribution under regime g.