Skip to main content
. 2022 Jan 18;1(1):e0000006. doi: 10.1371/journal.pdig.0000006

Table 1. Checklist for ideal algorithms in healthcare.

Desiderata Criteria Yes Location No N/A
Explainable
    Yesa
    Partiallyb
    Noc
    N/Ad
Feature importance: conveys the relative importance of features in determining algorithm outputs
Descriptive accuracy: describes what the algorithm has learned (e.g., weights in a neural network)
Simulatability: clinicians can understand and mentally simulate the model’s process for generating predictions
Relevance: describes relevancy as judged by the algorithm’s target human audience
Dynamic
    Yesa
    Partiallyb
    Noc
    N/Ad
Temporality: captures temporal changes in physiologic signals and clinical events
Continuous monitoring: performance is reassessed at several time points, including the point at which performance is expected to plateau
Precise
    Yesa
    Partiallyb
    Noc
    N/Ad
Data frequency: rate of data collection matches the rate of physiologic changes
Complexity: algorithm complexity matches the complexity of the prediction or classification task
Autonomous
    Yesa
    Partiallyb
    Noc
    N/Ad
Efficiency: the algorithm executes without the need for time-consuming, manual data entry by the end user (i.e., patient, provider, or investigator)
Fair
    Yesa
    Partiallyb
    Noc
    N/Ad
Generalizability: algorithm is developed and validated across diverse patient demographics and practice settings
Selectivity: excludes features that lack pathophysiologic or linguistic association with outcomes, but may introduce bias
Objectivity: includes variables that are minimally influenced by clinician judgments (e.g., vital signs)
Reproducible
    Yesa
    Partiallyb
    Noc
    N/Ad
Generalizability: validated externally, prospectively
Collaboration: algorithm is shared with the research community
Compliance: fulfills SPIRIT-AI extension guidelines (if trial) and fulfills CONSORT-AI guidelines

aOverall adjudication is “Yes” when all criteria are either met or not applicable.

bOverall adjudication is “Partially” when some but not all criteria are either met or not applicable.

cOverall adjudication is “No” when no criteria are met.

dOverall adjudication is “N/A” when all criteria are not applicable.

CONSORT-AI, Consolidated Standards of Reporting Trials-Artificial Intelligence; N/A, not applicable; SPIRIT-AI, Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence.