Table 1.

This table shows the 3 levels in the building process of a clinical decision support system and some examples of where clinical expert knowledge of health care professionals plays a role in each of these levels.

Level and example of issue		Example of expert knowledge
Data level
	Laboratory thresholds	Hemoglobin reference range to diagnose anemia
	Derived measurementsa	Body mass index
	Diagnostic codes	Grouping of related diagnoses in a study population
	Jargon	Same abbreviations having different meanings
	Temporality	Glucose values are highly dependent on the time of day (eg, pre- or postprandial)
Algorithm level
	Methodological choices	How to handle missing data (eg, missing not at random)
	Feature engineeringa	Constructing relevant derived variables from raw data (eg, torsades de pointes, Wolff-Parkinson-White syndrome)
	Artifacts	For example, oxygen saturation of zero caused by a slipping pulse oximeter, switched leads in an electrocardiogram
Decision support level
	Interpretation of model output	Risk probability of 0.75 requires a warning (amber light) in a CDSb system
	Degree of autonomy	Tuning of implantable cardioverter defibrillator
	Knowledge on usefulness	Weighing a CDS system’s advice to treat while considering quality of life versus treatment burden in elderly cancer patients in a shared decision-making context

^aDerived measurements may occur at the data level but also at the algorithm level; the former being undesirable because any manipulation at the data level may result in a loss of information.

^bCDS: clinical decision support.