Table 1.

Measures of improvement in prediction when risk models, logitP(D = 1|X) = α₀ + α₁X and logit P(D = 1|X, Y) = β₀ + β₁X + β₂Y, are fit to a training dataset and applied to a test dataset. The novel marekr Y does not improve prediction: the true models are linear logistic (2) and (3) with coefficients: β₂ = 0, β₁ = α₁ and β₀ = α₀. Data generation is described in Supplementary Materials. Performance measures are averaged over 1000 simulations. Standard errors are small and are shown in Table A.1 of the Supplementary Materials. In parentheses are shown the % of simulations where the NRI was found to be statistically significantly positive.

One Uninformative Marker
Simulation Scenario					Average Performance Increment ×100
ρ = P(D = 1)	AUCaX	N-trainingb	N-testc	NRId	ΔROC(0.2)	ΔAUC	ΔBrier	ΔSNB(ρ)
0.1	0.6	250	25,000	0.27(12.7)	−1.70	−1.28	−0.044	−1.85
0.1	0.7	250	25,000	1.38(29.4)	−1.37	−0.86	−0.049	−1.31
0.1	0.8	250	25,000	3.22(46.8)	−0.90	−0.48	−0.058	−0.80
0.1	0.9	250	25,000	7.72(60.3)	−0.52	−0.25	−0.066	−0.57
0.5	0.6	50	5,000	0.57(21.3)	−1.67	−1.19	−0.479	−1.69
0.5	0.7	50	5,000	2.78(40.5)	−2.59	−1.69	−0.540	−2.49
0.5	0.8	50	5,000	6.56(55.2)	−1.83	−1.00	−0.492	−1.62
0.5	0.9	50	5,000	17.09(69.0)	−1.11	−0.56	−0.433	−1.17

^aArea under the ROC curve for the baseline model (X)

^bSize of training dataset

^cSize of test dataset

^d% of simulations with NRI statistically significantly positive shown in parentheses