Skip to main content
. 2020 Sep 9;26(9):1364–1374. doi: 10.1038/s41591-020-1034-x
Artificial Intelligence The science of developing computer systems which can perform tasks normally requiring human intelligence.
AI intervention A health intervention that relies upon an AI/ML component to serve its purpose.
CONSORT Consolidated Standards of Reporting Trials.
CONSORT-AI extension item An additional checklist item to address AI-specific content that is not adequately covered by CONSORT 2010.
Class-activation map Class-activation maps are particularly relevant to image classification AI interventions. Class-activation maps are visualizations of the pixels that had the greatest influence on predicted class, by displaying the gradient of the predicted outcome from the model with respect to the input. They are also referred to as ‘saliency maps’ or ‘heat maps’.
Health outcome Measured variables in the trial that are used to assess the effects of an intervention.
Human–AI interaction The process of how users (humans) interact with the AI intervention, for the AI intervention to function as intended.
Clinical outcome Measured variables in the trial which are used to assess the effects of an intervention.
Delphi study A research method that derives the collective opinions of a group through a staged consultation of surveys, questionnaires, or interviews, with an aim to reach consensus at the end.
Development environment The clinical and operational settings from which the data used for training the model is generated. This includes all aspects of the physical setting (such as geographical location, physical environment), operational setting (such as integration with an electronic record system, installation on a physical device) and clinical setting (such as primary, secondary and/or tertiary care, patient disease spectrum).
Fine-tuning Modifications or additional training performed on the AI intervention model, done with the intention of improving its performance.
Input data The data that need to be presented to the AI intervention to allow it to serve its purpose.
Machine learning A field of computer science concerned with the development of models/algorithms that can solve specific tasks by learning patterns from data, rather than by following explicit rules. It is seen as an approach within the field of AI.
Operational environment The environment in which the AI intervention will be deployed, including the infrastructure required to enable the AI intervention to function.
Output data The predicted outcome given by the AI intervention based on modeling of the input data. The output data can be presented in different forms, including a classification (including diagnosis, disease severity or stage, or recommendation such as referability), a probability, a class activation map, etc. The output data typically provide additional clinical information and/or trigger a clinical decision.
Performance error Instances in which the AI intervention fails to perform as expected. This term can describe different types of failures, and it is up to the investigator to specify what should be considered a performance error, preferably based on prior evidence. This can range from small decreases in accuracy (compared to expected accuracy) to erroneous predictions or the inability to produce an output, in certain cases.
SPIRIT Standard Protocol Items: Recommendations for Interventional Trials.
SPIRIT-AI An additional checklist item to address AI-specific content that is not adequately covered by SPIRIT 2013.
SPIRIT-AI elaboration item Additional considerations to an existing SPIRIT 2013 item when applied to AI interventions.