Table 4.2.

Summary statistics and empirical coverage probability of 95% nominal percentile confidence intervals for ${\beta }_{11}^0$ and ${\beta }_{12}^0$ using the oracle estimator, the proposed penalized Q-learning based estimator, the hard max estimator and the soft-threshold estimator. “PQ” refers to the penalized Q-learning based estimator, “HM” refers to the hard max estimator, “MSE” refers to the mean squares error, “Std” refers to the average of the 2000 standard error estimates and “CP” refers to the empirical coverage probability of 95% nominal percentile confidence interval. A “*” indicates a significantly different coverage rate from the nominal rate.

	β11				β12
	bias(×1000)	MSE(×1000)	std(×100)	CP	bias(×1000)	MSE(×1000)	std(×100)	CP
Setting 1
Oracle	1	2.0	4.5	94.7	1	2.0	4.5	94.9
PQ	1	2.0	4.5	94.7	1	2.0	4.5	94.9
HM	61	6.6	–	88.7*	1	2.1	–	95.2
ST	7	2.3	–	96.2*	−1	2.0	–	94.9
Setting 2
Oracle	52	5.5	6.2	90.0*	1	2.1	4.6	95.3
PQ	−9	2.1	4.5	94.7	1	2.0	4.5	94.8
HM	52	5.5	–	90.8*	1	2.1	–	95.1
ST	−3	2.2	–	94.8	−1	2.0	–	95.1
Setting 3
Oracle	0	3.0	5.5	94.6	1	2.0	4.5	95.1
PQ	0	3.1	5.5	94.2	2	2.0	4.5	95.2
HM	30	4.3	–	93.0*	2	2.1	–	95.2
ST	−5	3.3	–	93.5*	−1	2.1	–	94.9
Setting 4
Oracle	26	4.0	6.2	93.8*	2	2.1	4.6	95.2
PQ	−5	3.1	5.5	94.3	2	2.0	4.5	95.1
HM	26	4.0	–	93.4*	2	2.1	–	95.1
ST	−10	3.4	–	93.5*	−1	2.1	–	95.0
Setting 5
Oracle	0	3.8	6.1	94.7	2	2.7	5.2	95.1
PQ	−2	3.8	6.1	94.4	0	2.7	5.2	95.0
HM	15	4.1	–	94.1	−5	2.7	–	94.3
ST	−8	4.0	–	94.9	−3	2.8	–	94.9
Setting 6
Oracle	2	3.8	6.2	94.8	−1	2.3	4.8	94.7
PQ	1	3.8	6.2	94.7	−1	2.3	4.8	94.7
HM	2	3.8	–	94.3	−1	2.3	–	95.2
ST	45	6.3	–	87.2*	−1	2.3	–	95.2