Skip to main content
. Author manuscript; available in PMC: 2017 Jan 1.
Published in final edited form as: Stat Interface. 2016;9(2):239–253. doi: 10.4310/SII.2016.v9.n2.a11

Table 2.

Simulation study results: U measures of constructed trees, means (SD) of the U* measure, expected value/reward and classification error for each combination of scenario (sizeable treatment-covariate interaction, small treatment-covariate interaction, and no interaction effects) and level of noise in the data (design 1 = no noise variables; design 2 = some noise variables; design 3 = many noise variables)

Proposed method Foster’s Virtual Twin method
Design Ua E(A)b Classification
error		 U*c
Mean(SD)		 E(A)*d
Mean(SD)		 E(A) Classification
error		 Mean(SD)
E(A)*
Sizable treatment by covariate interactions
Design 1 0.65 0.85 < 1% 0.54 (0.03) 0.79 (0.02) 0.85 < 1% 0.79 (0.02)
Design 2 0.65 0.85 < 1% 0.36 (0.05) 0.68 (0.02) 0.85 < 1% 0.68 (0.02)
Design 3 0.65 0.85 < 1% 0.32 (0.07) 0.67 (0.03) 0.85 < 1% 0.67 (0.03)
Small treatment by covariate interactionse
Design 1 0.13 0.56 24% 0.16 (0.03) 0.59 (0.02) 0.54 22% 0.60 (0.02)
Design 2 0.24 0.62 51% 0.00 (0.03) 0.50 (0.02) 0.55 39% 0.52 (0.02)
Design 3 0.30 0.65 49% −0.02 (0.03) 0.49 (0.02) 0.52 50% 0.50 (0.02)
No treatment by covariate interactions (main effects model)
Design 1 0.05 0.55 0% 0.11 (0.03) 0.62 (0.03) 0.60 29% 0.64 (0.02)
Design 2 0.14 0.60 31% 0.03 (0.03) 0.55 (0.02) 0.56 19% 0.55 (0.02)
Design 3 0.11 0.59 19% 0.02 (0.03) 0.54 (0.02) 0.55 0% 0.55 (0.02)
a

U - weighted difference in proportions of subjects with good outcome on the better treatment compared to the worse treatment based on the constructed tree on the original sample.

b

E(A) - expected value/reward based on the original sample.

c

U* - weighted difference in proportions of subjects with good outcome on the better treatment compared to the worse treatment based on the constructed tree on the original sample, evaluated on the counterfactual samples.

d

E(A)* - expected value/reward calculated on the counterfactual samples.

e

Alpha level of 0.20 was used for pruning. In the remaining scenarios alpha level of 0.05 was used for pruning. For sizeable treatment by covariate interactions, the results were the same regardless whether alpha level was 0.05 or 0.20.