Table 6.

Performance of model selection with main/internal validation study-based analysis under a strong positive association

Model	$log(\widehat{\text{OR}})$ (SD)	Mean SE	95% CI coverage
Setting 7: SEX = 0.60, SPX = 0.90, SEY = 0.70, SPY = 0.80
Naïve	0.46(0.14)	0.14	0
Gold standard	1.80(0.14)	0.15	96.4%
Model 1	1.82(0.28)	0.30	96.8%
Model 2	1.82(0.28)	0.29	96.8%
Model 3 (underlying model)	1.82(0.27)	0.28	96.4%
Model selectiona	1.82(0.28)	0.28	96.4%
Setting 8: SEX1 = 0.60, SPX1 = 0.60, SE X0 = 0.90, SPX0 = 0.90, SEY1 = 0.40, SPY1 = 0.98, SEY0 = 0.70, SPY0 = 0.80
Naïve	−0.20(0.15)	0.15	0
Gold standard	1.80(0.16)	0.15	93.8%
Model 1	1.81(0.31)	0.29	93.6%
Model 2 (underlying model)	1.81(0.31)	0.29	93.4%
Model 3	1.59(0.30)	0.29	85.8%
Model selectionb	1.81(0.31)	0.29	93.2%
Setting 9: SEX1 = 0.60, SPX1 = 0.91, SE X0 = 0.48, SPX0 = 0.94, SEY11 = 0.50, SPY11 = 0.98, SEY10 = 0.21, SPY10 = 0.99, SEY01 = 0.63, SPY01 = 0.97, SE Y00 = 0.31, SPY00 = 0.99
Naïve	1.98(0.18)	0.18	68.2%
Gold standard	1.79(0.14)	0.15	96.8%
Model 1 (underlying model)	1.80(0.28)	0.29	97.0%
Model 2	1.95(0.28)	0.28	92.8%
Model 3	2.57(0.27)	0.27	19.8%
Model selectionc	1.80(0.28)	0.29	97.0%

Notes: 500 simulation studies; 229 internal validation observations and 687 main study observations per simulation. Data were generated from a multinomial distribution with cell probabilities of (π₁₁ = 0.30, π₁₀ = 0.10, π₀₁ = 0.20, π₀₀ = 0.40). True log (OR) = 1.79. Naïve model uses (Y^*, X^*) data. Gold-standard model uses (Y, X) data. Model 1 assumes dependent and differential misclassification. Model 2 assumes independent and differential misclassification. Model 3 assumes completely nondifferential misclassification. Model selection based on the strategy described in Section 2.7.

^aModel 3 selected 87.2% of the time.

^bModel 2 selected 90.6% of the time.

^cModel 1 selected 95.8% of the time.