Table 2.

Marginal effects of experimental design factors on key engagement metrics (95% CI).

Factor		Key Engagement Metric
		Feature utilization, mean	Page views ≥25, %	Time on site ≥15 min, %	Returned to website, %
Personalization
	On	2.38 (2.24 to 2.53)	31.6 (27.2 to 36.0)	31.3 (27.0 to 35.6)	37.9 (33.3 to 42.4)
	Off	2.43 (2.28 to 2.58)	32.9 (28.5 to 37.3)	28.4 (24.1 to 32.6)	39.1 (34.6 to 43.7)
	Raw difference	−0.05 (−0.25 to 0.16)	−1.3 (−7.5 to 4.9)	2.9 (−3.2 to 9.0)	−1.2 (−7.7 to 5.2)
	Standard mean difference	−0.021 (−0.115 to 0.074)	−0.017 (−0.111 to 0.077)	0.045 (−0.049 to 0.139)	−0.019 (−0.113 to 0.075)
Integration
	On	2.59 (2.44 to 2.74)	30.8 (26.5 to 35.2)	29.7 (25.4 to 34.0)	37.0 (32.4 to 41.5)
	Off	2.22 (2.08 to 2.37)	33.7 (29.3 to 38.1)	29.9 (25.6 to 34.0)	40.0 (35.5 to 44.6)
	Raw difference	0.36 (0.16 to 0.57)a	−2.9 (−9.1 to 3.3)	−0.2 (−6.3 to 5.9)	−3.0 (−9.5 to 3.4)
	Standard mean difference	0.166 (0.071 to 0.260)a	−0.043 (−0.137 to 0.051)	−0.003 (−0.097 to 0.091)	−0.044 (−0.138 to 0.048)
Dynamic Tailoring
	On	2.55 (2.40 to 2.70)	35.9 (31.4 to 40.4)	32.3 (28.0 to 36.7)	41.6 (36.9 to 46.2)
	Off	2.26 (2.12 to 2.40)	28.6 (24.3 to 32.9)	27.3 (23.1 to 31.0)	35.4 (30.9 to 39.9)
	Raw difference	0.29 (0.09 to 0.50)b	7.3 (1.1 to 13.5)c	5.0 (−1.0 to 11.1)	6.2 (−0.3 to 12.6)
	Standard mean difference	0.134 (0.040 to 0.229)b	0.110 (0.016 to 0.205)c	0.078 (−0.016 to 0.173)	0.089 (−0.004 to 0.184)
Intensity
	Tapered	2.42 (2.27 to 2.57)	33.1 (28.72 to 37.6)	31.3 (27.1 to 35.7)	40.0 (35.5 to 44.6)
	Abrupt	2.39 (2.25 to 2.54)	31.4 (27.0 to 35.8)	28.3 (24.1 to 32.6)	36.9 (32.5 to 41.5)
	Raw difference	0.03 (−0.18 to 0.23)	1.7 (−4.5 to 8.0)	3.0 (−3.0 to 9.1)	3.1 (−3.3 to 9.5)
	Standard mean difference	0.012 (−0.083 to 0.106)	0.028 (−0.066 to 0.122)	0.047 (−0.047 to 0.141)	0.045 (−0.049 to 0.139)

^aSMD: standardized mean difference.

^bP<.001.

^cP<.01.