Table 3.

Simulation scenarios and results for continuous outcomes

Scenarioa	Parameters	Results	Methods
			IEEb	IEEWc	EEEd	EEEWe
Homogeneous treatment effect
1		Mean Estimate	2.0025	2.0009	2.0014	2.0003
	βl = 2	Mean Variance of Estimate	0.0101	0.0100	0.0089	0.0140
	βs = 2	Mean Bias	0.0025	0.0009	0.0014	0.0003
	σb = 0.2	Mean Relative Bias (%)	0.1274	0.0439	0.0725	0.0159
	σe = 1	Monte-Carlo Variance	0.0117	0.0105	0.0099	0.0163
	ρ = 0.038	Empirical Coverage	0.9200	0.9430	0.9340	0.9210
		Mean ICC	-	-	0.0338	-
2		Mean Estimate	2.0047	2.0032	2.0040	2.0030
	βl = 2	Mean Variance of Estimate	0.0170	0.0230	0.0164	0.0289
	βs = 2	Mean Bias	0.0047	0.0032	0.0040	0.0030
	σb = 0.2	Mean Relative Bias (%)	0.2341	0.1621	0.2006	0.1517
	σe = 2	Monte-Carlo Variance	0.0195	0.0237	0.0187	0.0333
	ρ = 0.010	Empirical Coverage	0.9210	0.9460	0.9230	0.9320
		Mean ICC	-	-	0.0075	-
Heterogeneous treatment effects
3		Mean Estimate	3.6505	2.9963	3.0363	2.3495
	βl = 4	Mean Variance of Estimate	0.0381	0.0839	0.0830	0.0449
	βs = 2	Mean Bias	0.6505	−0.0037	0.0363	−0.6505
	σb = 0.2	Mean Relative Bias (%)	21.6832	−0.1246	1.2101	−21.6828
	σe = 1	Monte-Carlo Variance	0.0278	0.0517	0.0553	0.0360
	ρ = 0.038	Empirical Coverage	0.1360	0.9860	0.9800	0.1740
		Mean ICC	-	-	0.3212	-
4		Mean Estimate	3.6526	2.9986	3.1603	2.3786
	βl = 4	Mean Variance of Estimate	0.0450	0.0968	0.0889	0.0752
	βs = 2	Mean Bias	0.6526	−0.0014	0.1603	−0.6214
	σbs = 0.2	Mean Relative Bias (%)	21.7544	−0.0458	5.3440	−20.7122
	σe = 2	Monte-Carlo Variance	0.0362	0.0655	0.0785	0.0703
	ρ = 0.010	Empirical Coverage	0.1670	0.9750	0.9030	0.3910
		Mean ICC	-	-	0.0878	-

^aScenarios 1 and 3 have same values of σ_b and σ_e, but Scenario 1 has homogenous treatment effect and 3 has heterogeneous treatment effects, resulting in the difference in the mean empirical ICCs. Similar patterns for Scenarios 2 and 4. The mean UATE estimand of Scenarios 1 and 2 (homogenous treatment effect) is 2; the mean UATE estimand of Scenarios 3 and 4 (heterogeneous treatment effects) is 3. The true ICC for homogeneous treatment effect (Scenarios 1 and 2) are estimated directly using $\rho =\frac{{\sigma }_b^2}{{\sigma }_b^2+{\sigma }_e^2}$, yielding values of 0.0385 and 0.0099, respectively. In contrast, the true ICC for heterogeneous treatment effects (Scenarios 3 and 4) are obtained via a weighted average of the ICC for the large and small clusters.

^bIEE: Independence working correlation matrix with no cluster size weighting.

^cIEEW: Independence working correlation matrix with inverse cluster size weighting.

^dEEE: Exchangeable working correlation matrix with no cluster size weighting.

^eEEEW: Exchangeable working correlation matrix with inverse cluster size weighting.