Skip to main content
. 2020 Aug 19;29(157):200010. doi: 10.1183/16000617.0010-2020

TABLE 2.

Criteria for public consultation analysis by CNEDiMTS on the use of artificial intelligence (AI) software in medicine

Final use
 1. Detail the benefit of the information that will be given by the system relying on AI
 2. Describe the characteristics of the population for whom it is intended
Description of the learning process for the AI
 3. Type of learning method

  •  Continuous (autonomous learning and adaptation)?

  •  Initial (algorithm is trained initially with no further updates)?

  •  Or incremental (algorithm updated through learning process)?

 4. Describe the model used

  •  Supervised?

  •  Semi-supervised?

  •  Unsupervised?

  •  By reinforcement?

 5. Describe the algorithm used

  •  Classification

  •  Regression

  •  Clustering

 6. Describe selection method of the model
 7. Describe the different steps of learning phase
 8. Describe the strategy for updating the algorithm
 9. Describe if necessary in which cases humans intervene in the learning process
Detail the data entered in the initial learning phase or data involved for the updates or autonomous learning process
Describe the characteristics of the sample from the targeted population, used to develop the model
 10. Detail the characteristics of the sample
 11. Detail the modalities for separating training data from testing data and validating data
 12. Justify representativeness of the sample chosen compared with the targeted population
Description of the variables
 13. Characteristics of the variables

  •  Type

  •  Distribution

 14. Origins of the variables and methods of acquisition
Detail handling of the data before their use for the learning phase
 15. Describe the statistical tests used
 16. Describe methods of transformation use for the data
 17. Describe handling of missing information
 18. Explain detection of erroneous or aberrant data and their handling
Detail entry data implicated in decision-making
 19. Origins of the variables and methods of acquisition
 20. Characteristics of the variables

  •  Type

  •  Distribution

Performance
 21. Describe and justify the method of measurement adopted for the performance
 22. Describe the potential impact of adjustments measures
 23. Characterise overfitting and underfitting
 24. Describe the methods to handle overfitting and underfitting
Validation
 25. Describe the methods of validation
 26. Report the performance of the algorithms on the data set
Resilience of the system
 27. Describe the mechanisms set up in order to understand model drift
 28. Detail the thresholds chosen
 29. Detail if there is a system to detect any anomaly in the entry data implicated in the decision
 30. Describe the potential impacts of those aberrant entry data
 31. Detail the measure set up in case of model drift
 32. Detail the situations susceptible to alter the system function
Explainability/interpretability
 33. Does the algorithm benefit from a technique of explainability/interpretability for the patients and/or the physicians?
 34. Detail the elements of explainability available
 35. Identify the influential parameters
 36. Detail if the decision-making in the system follows guidelines when they exist