Table 4.

D-study estimates using the MTurk average ratings.

Estimate	n	Alternative uses	Consequences
${\sigma }_p^2$	1	0.167	0.068
${\sigma }_r^2$	3	0.060	0.117
${\sigma }_i^2$	2	0.035	0.000
${\sigma }_{pr}^2$	3	0.003	0.005
${\sigma }_{pi}^2$	2	0.069	0.044
${\sigma }_{ri}^2$	6	0.004	0.001
${\sigma }_{pri,e}^2$	6	0.016	0.015
${\sigma }_{\delta }^2$		0.088	0.064
${\sigma }_{\Delta }^2$		0.187	0.182
𝔼ρ2		0.65	0.52
Φ		0.47	0.27

Estimates of variance components, relative error variance (${\sigma }_{\delta }^{\mathit{\text{2}}}$), absolute error variance (${\sigma }_{\Delta }^{\mathit{\text{2}}}$), generalizability coefficient (𝔼ρ²), and dependability coefficient (Φ) are for the current design (3 raters, 2 items). Universe-score variance (${\sigma }_{\tau }^{\mathit{\text{2}}}$) is ${\sigma }_{\mathit{\text{p}}}^{\mathit{\text{2}}}$.