Table 4.

Details of evaluations of tool internal and external validity (n=53 tools)

Performance Measure	Internal Validation (n= 45 tools)	External Validation (n= 79 validations)
Internal Validation Method*
Apparent	27 (60)	--
Cross-Validation	2 (4)	--
Split Sample	6 (14)	--
Bootstrapping	10 (22)	--
External Validation Method
Independent	--	61 (77)
Geographic	--	13 (16)
Temporal	--	3 (4)
Other**		2 (3)
Overall Model Performance
R-squared	2 (4)	2 (3)
Calibration
Graph (Plot/intercept/slope)	11 (25)	4 (5)
Hosmer/Lemeshow statistic	3 (7)	0 (0)
Sub-group calibration***	2 (5)	5 (6)
Discrimination
C-statistic****	21 (48)	33 (42)
Survival Analysis Only with Significance Test	22 (50)	40 (51)

^*One tool applied both split sample and bootstrap methods;

^**An RCT(s) was used to develop the prognostic tool, and an additional RCT was used for validation;

^***Tables comparing predicted and observed values for groups of patients were provided;

^****Concordance index based on the ROC for binary data, Harrell’s C statistic for models using time to event data