Table 4.—

Predictive Performance of Primary and Penalized Models

		Primary Model		Penalized Model
Dimension	Statistic	Original (Within-Sample)	Cross-Validated (Out-of-Sample)	Original (Within-Sample)	Cross-Validated (Out-of-Sample)
C-statistic (Discrimination)	Within-person	0.58 (0.57, 0.60)	0.56 (0.54, 0.58)	0.58 (0.56, 0.60)	0.56 (0.54, 0.58)
	Overall	0.65 (0.64, 0.67)	0.58 (0.56, 0.6)	0.62 (0.6, 0.63)	0.59 (0.57, 0.60)
Calibration in the large	Mean intercept	0.01 (−0.04, 0.06)	0 (−0.08, 0.07)	−0.01 (−0.07, 0.06)	−0.01 (−0.08, 0.07)
	SD of intercepts	0	0.29	0.18	0.26
Weak calibration	Mean intercept	0.38 (0.20, 0.56)	−0.42 (−0.68, −0.16)	0.41 (0.15, 0.67)	−0.1 (−0.39, 0.19)
	SD of intercepts	0	0.14	0.03	0.12
	Mean slope	1.25 (1.13, 1.37)	0.73 (0.57, 0.9)	1.27 (1.1, 1.45)	0.94 (0.75, 1.13)
	SD of slopes	0	0.12	0.13	0.09
Predicted probabilities	Variance	0.004	0.004	0.002	0.002

The C-statistic was calculated within-person (by calculating the C-statistic for each person separately, and combining the statistics across persons with a random effects meta-analysis)and overall (comparing all pairs of days in the standard way).^16,37 The calibration statistics were calculated based on multilevel logistic regression models for the observed outcome based on the linear predictor as an offset (calibration in the large) or as the univariable random effect (weak calibration) plus a random intercept. Standard deviations of intercepts and slopes from these models represent the standard deviation of the estimated random effects across persons. Wald 95% confidence intervals are shown for the key statistics.