Table 2.

Simulation 2: Comparison between the marginal-mean-model-based estimators and the assisted estimators, with respect to the performance in estimating the policy contrasts.

N = 100
Estimation of the first contrast, (1, 1, 1) vs (0,0,0)
Scenario	True value	Bias × 100				Coverage of 95% CI × 100				Relative MSE
		MM	Assist1	Assist2	Assist3	MM	Assist1	Assist2	Assist3	Assist1	Assist2	Assist3
(none,none)	0	2.4	4.9	5.2	4.9	95.2	96.2	96	96.1	0.94	0.93	0.99
(none,low)	−2.4	5.8	4.6	4.7	6	94.5	96	95.4	95.2	0.95	0.94	1.04
(none,med)	−5.2	12	−6.8	−6.8	−2.6	93.6*	93.9	93.6*	94.6	0.95	0.95	1.01
(low,none)	−1.4	−1.9	2.5	1.7	4.8	95.6	94.6	94	95	1.01	1.01	1.09
(low,low)	−3.8	−12.5	−10.8	−11	−10.3	94.3	94.5	93.5*	94.6	0.92	0.92	0.97
(low,med)	−6.6	11	−9.9	−10.4	−5.8	93.9	94.8	94.7	95.5	0.84	0.84	0.93
(med,none)	−3.6	8.9	4.2	5.4	3.4	95.5	95.9	95.3	96.2	0.89	0.87	0.89
(med,low)	−6.0	9.7	−1.9	−2.7	−7.1	94.3	94.8	94.1	94.9	0.85	0.85	0.93
(med,med)	−8.8	28.9*	4.2	5.4	4.7	93.7	94.9	95.2	94.9	0.8	0.79	0.85

Estimation of the second contrast, the tailored policy vs (0,0,0)
Scenario	True value	Bias × 100				Coverage of 95% CI × 100				Relative MSE
		MM	Assist1	Assist2	Assist3	MM	Assist1	Assist2	Assist3	Assist1	Assist2	Assist3
(none,none)	0	6	1	2.4	2.3	96.2	97*	96.6*	96.1	0.78	0.76	0.57
(none,low)	−2.2	6.4	4.8	−2.8	16.7*	95.6	96	95.7	94.7	0.79	0.77	0.59
(none,med)	−3.9	11.5	−2.8	−22.1*	−43.9*	94.9	95.8	95.1	94.4	0.78	0.77	0.67
(low,none)	−1.1	5.3	11.2*	9.7	42.9*	95.5	95.3	94.8	93.8	0.81	0.8	0.69
(low,low)	−3.3	−7.3	−6.3	−15.1*	46.3*	93.9	95.3	93.9	95	0.77	0.74	0.59
(low,med)	−5	6.7	−1.8	−23.6*	−2.8	94	96.3	94.9	95.7	0.7	0.69	0.5
(med,none)	−2.3	9.3	8	9.1	50*	95.9	96.5*	95.8	95.4	0.76	0.74	0.57
(med,low)	−4.4	13.7*	9.4	−0.3	53.3*	93.2*	95	95.2	94.1	0.7	0.67	0.57
(med,med)	−6.2	24.7*	5.2	−15.1*	9.9*	93.1*	95.5	95.3	95.6	0.66	0.64	0.49

MM = Marginal-mean-model-based estimator. Assist1 = Assisted estimator with correctly specified SNMM. Assist2 = Assisted estimator with mis-specified SNMM that excludes X₁₁, X₂₁, RX₂₁. Assist3 = Assisted estimator with mis-specified SNMM that excludes all the covariates interacting with treatments. Bias significantly different from 0, and coverage proportion significantly different from 95%, are marked with an asterisk. Relative MSE is calculated as the ratio of MSE with that of MM.