Skip to main content
. Author manuscript; available in PMC: 2019 Jun 1.
Published in final edited form as: Psychol Methods. 2017 Apr 13;23(2):337–350. doi: 10.1037/met0000138

Table 2.

The empirical statistical properties (bias, standard deviation (SD), and mean squared error (MSE)) of estimates of treatment effects when the subject was certain of treatment (X|π=1) and rejection rates of the associated hypothesis tests, for the given data generating models

# True Model True X|π=1 Fitted Model Bias (X|π=1) SD (X|π=1) MSE (X|π=1) Reject, Omnibus Reject, X|π=1
1 y = e 0 X   0.00084 0.1001 0.0100 0.0502 0.0502
X+π   0.00081 0.1233 0.0152 0.0507 0.0507
X+π+X*π   0.00035 0.2463 0.0607 0.0504 0.0496
X (RCT)   0.00041 0.0998 0.0100 0.0499 0.0499
2 y = 0.5X + e 0.5 X   0.00049 0.1000 0.0100 0.9987 0.9987
X+π   0.00060 0.1228 0.0151 0.9820 0.9820
X+π+X*π   0.00106 0.2463 0.0607 0.9600 0.5275
X (RCT)   0.00012 0.1001 0.0100 0.9986 0.9986
3 y = 0.3π + e 0 X   0.10029 0.1007 0.0202 0.1691 0.1691
X+π   0.00008 0.1232 0.0152 0.0505 0.0505
X+π+X*π   0.00066 0.2465 0.0607 0.0506 0.0501
X (RCT) −0.00016 0.1004 0.0101 0.0504 0.0504
4 y = 0.5X +
0.3π + e 		0.5 X   0.10034 0.1005 0.0202 1.0000 1.0000
X+π   0.00010 0.1230 0.0151 0.9817 0.9817
X+π+X*π −0.00007 0.2468 0.0609 0.9606 0.5266
X (RCT) −0.00014 0.0997 0.0099 0.9984 0.9984
5 y = 0.5X +
0.3π + 0.2Xπ
+ e 		0.7 X   0.03443 0.1009 0.0114 1.0000 1.0000
X+π −0.09914 0.1233 0.0250 0.9979 0.9979
X+π+X*π −0.00010 0.2464 0.0607 0.9946 0.8086
X (RCT) −0.09992 0.1001 0.0200 1.0000 1.0000
6 y = 0.5X +
0.2Xπ + e 		0.7 X −0.06671 0.1002 0.0145 1.0000 1.0000
X+π −0.10008 0.1226 0.0251 0.9979 0.9979
X+π+X*π −0.00046 0.2472 0.0611 0.9945 0.8082
X (RCT) −0.10028 0.1002 0.0201 1.0000 1.0000
7 y = 0.3π +
0.2Xπ + e 		0.2 X   0.03272 0.1008 0.0112 0.6360 0.6360
X+π −0.10016 0.1234 0.0253 0.1286 0.1286
X+π+X*π −0.00100 0.2466 0.0608 0.1203 0.1273
X (RCT) −0.09976 0.1000 0.0199 0.1690 0.1690
8 y = 0.2Xπ +
e 		0.2 X −0.06667 0.1001 0.0145 0.2628 0.2628
X+π −0.10043 0.1231 0.0252 0.1272 0.1272
X+π+X*π   0.00048 0.2464 0.0607 0.1215 0.1279
X (RCT) −0.10045 0.1003 0.0201 0.1687 0.1687
9 y = f1(π) + e 0 X   0.21427 0.1019 0.0563 0.5605 0.5605
X+π −0.00015 0.1231 0.0151 0.0497 0.0497
X+π+X*π   0.00071 0.2469 0.0610 0.0502 0.0505
X (RCT) −0.00013 0.1002 0.0100 0.0506 0.0506
10 y = 0.5X +
f1(π) + e 		0.5 X   0.21470 0.1012 0.0563 1.0000 1.0000
X+π   0.00025 0.1230 0.0151 0.9819 0.9819
X+π+X*π −0.00108 0.2464 0.0607 0.9607 0.5245
X (RCT)   0.00026 0.1004 0.0101 0.9987 0.9987
11 y = 0.5X +
f1(π) +
0.2f1(π)X + e 		0.601 X   0.18527 0.1014 0.0446 1.0000 1.0000
X+π −0.05061 0.1229 0.0177 0.9938 0.9938
X+π+X*π   0.01352 0.2459 0.0606 0.9850 0.7014
X (RCT) −0.05007 0.0997 0.0125 0.9998 0.9998
12 y = 0.5X +
0.2f1(π)X + e 		0.601 X −0.02910 0.0998 0.0108 0.9999 0.9999
X+π −0.05019 0.1226 0.0175 0.9936 0.9936
X+π+X*π   0.01462 0.2455 0.0605 0.9849 0.7050
X (RCT) −0.05097 0.1000 0.0126 0.9998 0.9998
13 y = f1(π) +
0.2f1(π)X + e 		0.101 X   0.18521 0.1013 0.0446 0.8029 0.8029
X+π −0.05043 0.1226 0.0176 0.0678 0.0678
X+π+X*π   0.01460 0.2469 0.0612 0.0693 0.0758
X (RCT) −0.05022 0.1002 0.0126 0.0803 0.0803
14 y = 02f1(π)X
+ e 		0.101 X −0.02916 0.0998 0.0108 0.1095 0.1095
X+π −0.05055 0.1225 0.0176 0.0685 0.0685
X+π+X*π   0.01309 0.2462 0.0608 0.0688 0.0738
X (RCT) −0.05027 0.1000 0.0125 0.0791 0.0791
15 y = f2(π) + e 0 X   0.17542 0.1014 0.0411 0.4083 0.4083
X+π −0.00006 0.1236 0.0153 0.0489 0.0489
X+π+X*π −0.25583 0.2482 0.1271 0.1701 0.1766
X (RCT) −0.00006 0.1000 0.0100 0.0504 0.0504
16 y = 0.5X +
f2(π) + e 		0.5 X   0.17587 0.1012 0.0412 1.0000 1.0000
X+π −0.00004 0.1236 0.0153 0.9810 0.9810
X+π+X*π −0.25506 0.2479 0.1265 0.9715 0.1694
X (RCT)   0.00052 0.1001 0.0100 0.9988 0.9988
17 y = 0.5X +
f2(π) +
02f2(π)X + e 		0.610 X   0.16782 0.1017 0.0385 1.0000 1.0000
X+π −0.01722 0.1236 0.0156 0.9971 0.9971
X+π+i*π −0.24525 0.2485 0.1219 0.9943 0.3161
X (RCT) −0.00515 0.1006 0.0101 1.0000 1.0000
18 y = 0.5X +
02f2(π)X + e 		0.610 X −0.00763 0.1000 0.0101 1.0000 1.0000
X+π −0.01646 0.1227 0.0153 0.9977 0.9977
X+π+X*π   0.01083 0.2466 0.0609 0.9939 0.7094
X (RCT) −0.00516 0.0999 0.0100 1.0000 1.0000
19 y = f2(π) +
0.2f2(π)X + e 		0.110 X   0.16775 0.1013 0.0384 0.7797 0.7797
X+π −0.01740 0.1235 0.0156 0.1161 0.1161
X+π+X*π −0.24615 0.2484 0.1223 0.1962 0.0848
X (RCT) −0.00530 0.1003 0.0101 0.1818 0.1818
20 y = 0.2f2(π)X
+ e 		0.110 X −0.00793 0.0998 0.0100 0.1736 0.1736
X+π −0.01682 0.1230 0.0154 0.1182 0.1182
X+π+X*π   0.00985 0.2474 0.0613 0.0970 0.0781
X (RCT) −0.00514 0.1005 0.0101 0.1815 0.1815
21 y = f3(π) + e 0 X   0.13768 0.1007 0.0291 0.2769 0.2769
X+π   0.00020 0.1230 0.0151 0.0495 0.0495
X+π+X*π   0.12139 0.2470 0.0757 0.0748 0.0778
X (RCT)   0.00013 0.1004 0.0101 0.0511 0.0511
22 y = 0.5X +
f3(π) + e 		0.5 X   0.13765 0.1009 0.0291 1.0000 1.0000
X+π −0.00031 0.1231 0.0152 0.9822 0.9822
X+π+X*π   0.12178 0.2462 0.0755 0.9631 0.7123
X (RCT) −0.00055 0.1000 0.0100 0.9988 0.9988
23 y = 0.5X +
f3(π) +
0.2f3(π)X + e 		0.581 X   0.09457 0.1011 0.0192 1.0000 1.0000
X+π −0.06127 0.1231 0.0189 0.9870 0.9870
X+π+X*π   0.11257 0.2469 0.0736 0.9760 0.8006
X (RCT) −0.06805 0.0999 0.0146 0.9992 0.9992
24 y = 0.5X +
0.2f1(π)X + e 		0.581 X −0.04363 0.1000 0.0119 0.9997 0.9997
X+π −0.06154 0.1229 0.0189 0.9869 0.9869
X+π+X*π −0.00839 0.2465 0.0608 0.9720 0.6392
X (RCT) −0.06833 0.1003 0.0147 0.9990 0.9990
25 y = f3(π) +
0.2f3(π)X + e 		0.081 X   0.09406 0.1008 0.0190 0.4072 0.4072
X+π −0.06145 0.1227 0.0188 0.0529 0.0529
X+π+X*π   0.11269 0.2473 0.0738 0.1053 0.1227
X (RCT) −0.06823 0.0999 0.0146 0.0516 0.0516
26 y = 0.2f3(π)X
+ e 		0.081 X −0.04344 0.1000 0.0119 0.0657 0.0657
X+π −0.06127 0.1227 0.0188 0.0519 0.0519
X+π+X*π −0.00878 0.2461 0.0606 0.0555 0.0591
X (RCT) −0.06906 0.1000 0.0148 0.0507 0.0507
27 y = 0.3π
+ 0.2f1(π)X + e 		0.101 X   0.07084 0.1007 0.0152 0.4001 0.4001
X+π −0.05063 0.1232 0.0177 0.0699 0.0699
X+π+X*π   0.01266 0.2460 0.0607 0.0696 0.0737
X (RCT) −0.00072 0.1001 0.0100 0.1699 0.1699
28 y = f1(π) +
0.2Xπ + e 		0.2 X   0.14793 0.1018 0.0322 0.9265 0.9265
X+π −0.09929 0.1233 0.0251 0.1295 0.1295
X+π+X*π   0.00047 0.2471 0.0611 0.1236 0.1280
X (RCT) −0.14967 0.1000 0.0324 0.0787 0.0787
29 y = 0.5X +
0.3π +
0.2f1(π)X + e 		0.601 X   0.07037 0.1003 0.0150 1.0000 1.0000
X+π −0.05132 0.1227 0.0177 0.9932 0.9932
X+π+X*π   0.01249 0.2465 0.0609 0.9846 0.6991
X (RCT) −0.00126 0.1003 0.0101 1.0000 1.0000
30 y = 0.5X +
f1(π) + 0.2Xπ
+ e 		0.7 X   0.14802 0.1011 0.0321 1.0000 1.0000
X+π −0.09986 0.1226 0.0250 0.9982 0.9982
X+π+X*π −0.00040 0.2460 0.0605 0.9951 0.8075
X (RCT) −0.14932 0.0999 0.0323 0.9998 0.9998

Note. The two hypothesis tests only differ for the model with the interaction term.