Skip to main content
. Author manuscript; available in PMC: 2015 Aug 1.
Published in final edited form as: Sociol Methodol. 2014 Aug;44(1):229–272. doi: 10.1177/0081175013490188

Table 1.

Performance measures for logit models fitted to 100 simulated data sets with 4-fold cross validation.

Partially Out-of-Sample Fully Out-of-Sample
Scenario 1 Scenario 1 Scenario 2
TV AE PB CR MSE TV AE PB CR MSE TV AE PB CR MSE
β0 Standard Multilevel −0.5 −0.4 −22.4 0.5 0.6 −0.5 −0.4 −15.9 0.5 0.6 −0.5 −0.4 22.7 0.4 1.1
Pure Spatiala −0.5 −0.4 −25.8 0.9 0.7 −0.5 −0.4 −21.2 0.9 0.8 −0.5 −0.4 27.3 1.0 0.4
Hybrid CAR −0.5 −0.4 −20.4 0.4 0.6 −0.5 −0.5 −7.5 0.5 0.8 −0.5 −0.4 16.5 0.6 0.3
Hybrid Spatiala −0.5 −0.4 −24.7 0.9 1.0 −0.5 −0.3 −35.9 0.9 1.3 −0.5 −0.4 29.5 1.0 0.7
β1 Standard Multilevel 0.8 0.7 −12.5 0.8 0.0 0.8 0.7 −13.0 0.7 0.0 0.8 0.7 12.4 0.8 0.0
Pure Spatial 0.8 0.7 −12.5 0.8 0.0 0.8 0.7 −13.0 0.7 0.0 0.8 0.7 13.6 0.7 0.0
Hybrid CAR 0.8 0.7 −12.3 0.8 0.0 0.8 0.7 −12.1 0.9 0.0 0.8 0.7 13.0 0.8 0.0
Hybrid Spatial 0.8 0.7 −12.4 0.8 0.0 0.8 0.7 −12.8 0.7 0.0 0.8 0.7 13.5 0.7 0.0
γ1 Standard Multilevel −0.5 −0.4 −13.1 1.0 0.1 −0.5 −0.5 −6.9 0.9 0.1 −0.5 −0.5 6.6 1.0 0.1
Pure Spatial −0.5 −0.4 −12.3 1.0 0.1 −0.5 −0.5 −10.0 0.9 0.1 −0.5 −0.4 21.3 0.9 0.1
Hybrid CAR −0.5 −0.4 −11.7 1.0 0.1 −0.5 −0.5 −4.1 1.0 0.2 −0.5 −0.4 20.9 1.0 0.1
Hybrid Spatial −0.5 −0.4 −11.8 1.0 0.1 −0.5 −0.5 −8.9 0.9 0.1 −0.5 −0.4 21.7 0.9 0.1
σu2
 Standard Multilevel 3.0 4.3 43.7 0.7 3.1 3.0 4.4 45.1 0.8 3.6 3.0 6.3 110.4 0.2 14.2
Hybrid CAR 3.0 2.2 −25.6 0.9 1.5 3.0 3.2 5.1 1.0 15.5 3.0 2.4 20.6 0.9 1.0
Hybrid Spatial 3.0 2.2 −28.1 0.9 1.5 3.0 2.2 −27.6 0.9 1.6 3.0 2.1 30.1 0.9 1.4
σs2
 Pure Spatial 3.0 6.0 98.9 0.6 15.3 3.0 6.5 116.4 0.6 22.9 1.5 4.8 217.9 0.1 15.0
Hybrid CAR 3.0 9.3 208.8 0.9 62.8 3.0 9.7 223.3 0.9 98.2 1.5 5.7 280.8 0.9 25.4
Hybrid Spatial 3.0 4.7 56.6 1.0 15.7 3.0 5.2 72.5 1.0 23.2 1.5 3.2 111.0 1.0 11.9
ϕ Pure Spatial 3.0 7.2 138.4 0.5 18.4 3.0 6.7 123.6 0.7 15.3 3.0 7.4 145.9 0.5 19.8
Hybrid Spatial 3.0 4.6 54.0 1.0 3.8 3.0 4.5 49.8 1.0 3.4 3.0 4.8 60.4 1.0 4.2
Fully Out-of-Sample
Scenario 3 Scenario 4
TV AE PB CR MSE TV AE PB CR MSE
β0 Standard Multilevel −0.5 −0.4 −20.9 0.7 0.3 −0.5 −0.5 −8.7 0.9 0.1
Pure Spatiala −0.5 −0.4 −21.8 1.0 0.5 −0.5 −0.4 −11.1 1.0 0.2
Hybrid CAR −0.5 −0.4 −17.4 0.7 0.3 −0.5 −0.5 −7.0 0.8 0.2
Hybrid Spatiala −0.5 −0.4 −11.4 1.0 0.6 −0.5 −0.4 −14.5 1.0 0.5
β1 Standard Multilevel 0.8 0.7 −13.8 0.7 0.0 0.8 0.7 −14.1 0.7 0.0
Pure Spatial 0.8 0.7 −13.9 0.7 0.0 0.8 0.7 −14.1 0.7 0.0
Hybrid CAR 0.8 0.7 −12.9 0.8 0.0 0.8 0.7 −13.5 0.8 0.0
Hybrid Spatial 0.8 0.7 −13.6 0.7 0.0 0.8 0.7 −13.9 0.7 0.0
γ1 Standard Multilevel −0.5 −0.4 −11.7 0.9 0.1 −0.5 −0.5 −8.3 0.9 0.1
Pure Spatial −0.5 −0.4 −15.2 0.9 0.1 −0.5 −0.5 −9.7 0.9 0.1
Hybrid CAR −0.5 −0.5 −6.2 1.0 0.2 −0.5 −0.5 −8.5 1.0 0.1
Hybrid Spatial −0.5 −0.4 −12.8 1.0 0.1 −0.5 −0.5 −9.3 0.9 0.1
σu2
 Standard Multilevel 3.0 4.9 62.9 0.6 5.4 3.0 3.7 22.0 0.9 1.4
Hybrid CAR 3.0 3.9 30.9 1.0 13.9 3.0 2.7 −11.3 1.0 0.8
Hybrid Spatial 3.0 2.8 −6.7 1.0 1.2 3.0 2.3 −24.3 0.9 1.2
σs2
 Pure Spatial 3.0 6.8 127.9 0.4 22.4 1.5 5.1 240.3 0.0 15.7
Hybrid CAR 3.0 10.1 237.7 1.0 116.5 1.5 6.1 304.1 1.0 31.5
Hybrid Spatial 3.0 4.3 43.1 1.0 10.3 1.5 3.0 102.6 1.0 10.7
ϕ Pure Spatial 6.0 7.2 20.0 1.0 2.2 6.0 7.4 22.9 1.0 2.6
Hybrid Spatial 6.0 5.0 −17.4 1.0 1.7 6.0 5.0 −16.3 1.0 1.5

Note: TV = true value; AE = average estimate; PB = percentage bias; CR = coverage rate with 95% credible interval; MSE = mean square error.

a

Exponential spatial correlation function.