Table 1. Definitions of (a) reporting and (b) design characteristics.

(a) Reporting characteristic	Definition
Population	Age, sex and important clinical characteristics are described
Study site	Geographic location, institution type and setting, and how patients were referred are described
Prediction rule	Clear and detailed descriptions of predictor variables and prediction rule are provided with a description of process to ensure accurate assessment of rule such as training
Reliability	Intra-observer or inter-observer reliability of prediction rule is described
Outcome	Clear and detailed definition of outcome is provided
Results	Estimates of predictive performance are presented with confidence intervals
Follow up	Clearly describes what happened to all enrolled patients
(b) Design characteristic	Level	Definition
Sample size	Adequate	At least 100 patients with outcome and 100 patients without outcome
	Inadequate	Less than 100 patients with outcome or 100 patients without outcome
Patient selection	Consecutive	All consecutive patients meeting inclusion criteria are selected
	Nonconsecutive	Selection methods other than consecutive selection were used
Disease spectrum	Cohort	Prediction is ascertained before outcome is determined
	Case-control	Prediction is ascertained after outcome is determined
Validation type	Broad	Validation in different settings, with different patients and by different clinicians
	Narrow	Validation in similar settings, with similar patients or by similar clinicians
	Internal	Validation using methods such as split-sample validation, cross-validation or bootstrap
Assessment	Blind	Assessment of prediction without the knowledge of outcome and assessment of outcome without the knowledge of prediction
	Non-blind	Assessment of prediction with the knowledge of outcome or assessment of outcome with the knowledge of prediction
Verification	Complete	All predictions are verified using the same reference standard
	Partial	Only a subset of predictions are verified
	Differential	A subset of predictions are verified using different reference standard
Data collection	Prospective	Data collection for validation study is planned before prediction and outcome are assessed
	Retrospective	Prediction and outcome are assessed before data collection for validation starts