Table 3.
Out‐of‐sample predictive performance measures under measurement heterogeneity in a single‐predictor logistic regression model. Mean c‐statistic, median calibration slope, mean calibration‐in‐the‐large, and mean Brier score (standard deviation) at external validation of a single‐predictor logistic regression model transported from a derivation set (n= 2000) where measurement procedures were described by the random measurement error model (Equation 2) to validation sets (n= 2000) with various measurement structures under Equation (1). Predictive performance measures were averaged over 10 000 repetitions. All calibration slopes in the derivation set were equal to 1.0 (0.0) and are therefore not reported
Measurement Structure | C‐statistic | Calibration | Calibration‐in‐ | Brier Score | ||||
---|---|---|---|---|---|---|---|---|
at Validation | Derivation | Validation | Slope | the‐large (×10) | Derivation | Validation | ||
|
ψ = 0, θ = 0.5 | 0.745 (0.033) | 0.590 (0.034) | 0.247 (0.153) | ‐0.002 (0.006) | 0.204 (0.012) | 0.281 (0.033) | |
ψ = 0, θ = 1.0 | 0.745 (0.033) | 0.655 (0.045) | 0.380 (0.180) | 0.008 (0.014) | 0.204 (0.012) | 0.257 (0.031) | ||
ψ = 0, θ = 2.0 | 0.745 (0.033) | 0.726 (0.033) | 0.428 (0.125) | ‐0.009 (0.003) | 0.204 (0.012) | 0.232 (0.023) | ||
ψ = 0.25, θ = 0.5 | 0.745 (0.033) | 0.589 (0.034) | 0.247 (0.153) | ‐2.202 (0.643) | 0.204 (0.012) | 0.283 (0.032) | ||
ψ = 0.25, θ = 1.0 | 0.745 (0.033) | 0.655 (0.045) | 0.380 (0.180) | ‐2.210 (0.652) | 0.204 (0.012) | 0.258 (0.031) | ||
ψ = 0.25, θ = 2.0 | 0.745 (0.033) | 0.726 (0.033) | 0.428 (0.125) | ‐2.205 (0.651) | 0.204 (0.012) | 0.233 (0.023) | ||
|
ψ = 0, θ = 0.5 | 0.700 (0.068) | 0.635 (0.069) | 0.812 (0.291) | 0.001 (0.006) | 0.217 (0.020) | 0.235 (0.015) | |
ψ = 0, θ = 1.0 | 0.700 (0.068) | 0.700 (0.068) | 1.000 (0.000) | 0.001 (0.008) | 0.217 (0.020) | 0.218 (0.020) | ||
ψ = 0, θ = 2.0 | 0.700 (0.068) | 0.753 (0.042) | 0.955 (0.377) | ‐0.002 (0.013) | 0.217 (0.020) | 0.204 (0.014) | ||
ψ = 0.25, θ = 0.5 | 0.700 (0.068) | 0.635 (0.069) | 0.811 (0.293) | ‐1.529 (1.027) | 0.217 (0.020) | 0.237 (0.014) | ||
ψ = 0.25, θ = 1.0 | 0.700 (0.068) | 0.700 (0.068) | 1.002 (0.002) | ‐1.530 (1.033) | 0.217 (0.020) | 0.219 (0.019) | ||
ψ = 0.25, θ = 2.0 | 0.700 (0.068) | 0.753 (0.042) | 0.955 (0.377) | ‐1.526 (1.024) | 0.217 (0.020) | 0.205 (0.013) | ||
|
ψ = 0, θ = 0.5 | 0.655 (0.045) | 0.681 (0.045) | 3.147 (1.991) | 0.003 (0.007) | 0.230 (0.011) | 0.234 (0.009) | |
ψ = 0, θ = 1.0 | 0.655 (0.045) | 0.745 (0.034) | 3.106 (1.563) | 0.000 (0.006) | 0.230 (0.011) | 0.220 (0.014) | ||
ψ = 0, θ = 2.0 | 0.655 (0.045) | 0.781 (0.014) | 2.160 (0.969) | 0.005 (0.009) | 0.230 (0.011) | 0.203 (0.013) | ||
ψ = 0.25, θ = 0.5 | 0.655 (0.045) | 0.681 (0.045) | 3.156 (2.001) | ‐0.846 (0.528) | 0.230 (0.011) | 0.235 (0.008) | ||
ψ = 0.25, θ = 1.0 | 0.655 (0.045) | 0.745 (0.034) | 3.102 (1.559) | ‐0.846 (0.532) | 0.230 (0.011) | 0.221 (0.013) | ||
ψ = 0.25, θ = 2.0 | 0.655 (0.045) | 0.781 (0.014) | 2.159 (0.967) | ‐0.851 (0.535) | 0.230 (0.011) | 0.203 (0.013) |