Table 5.
CLAIM adherence of included studies
| CLAIM items (N = 9) | Study, n (%) |
|---|---|
| Overall (excluding item 15a and 27) | 262/459 (57) |
| Section 1: Title and abstract | 15/18 (83) |
| 1. Title or abstract—identification as a study of AI methodology | 9/9 (100) |
| 2. Abstract—structured summary of study design, methods, results, and conclusions | 6/9 (67) |
| Section 2: Introduction | 18/27 (67) |
| 3. Background—scientific and clinical background, including the intended use and clinical role of the AI approach | 9/9 (100) |
| 4a. Study objective | 9/9 (100) |
| 4b. Study hypothesis | 0/9 (0) |
| Section 3: Methods | 193/315 (61) |
| 5. Study design—prospective or retrospective study | 9/9 (100) |
| 6. Study design—study goal, such as model creation, exploratory study, feasibility study, non-inferiority trial | 9/9 (100) |
| 7a. Data—data source | 9/9 (100) |
| 7b. Data—data collection institutions | 9/9 (100) |
| 7c. Data—imaging equipment vendors | 9/9 (100) |
| 7d. Data—image acquisition parameters | 9/9 (100) |
| 7e. Data—institutional review board approval | 7/9 (78) |
| 7f. Data—participant consent | 5/9 (56) |
| 8. Data—eligibility criteria | 8/9 (89) |
| 9. Data—data pre-processing steps | 1/9 (11) |
| 10. Data—selection of data subsets (segmentation of ROI in radiomics studies) | 8/9 (89) |
| 11. Data—definitions of data elements, with references to Common Data Elements | 9/9 (100) |
| 12. Data—de-identification methods | 0/9 (0) |
| 13. Data—how missing data were handled | 0/9 (0) |
| 14. Ground truth—definition of ground truth reference standard, in sufficient detail to allow replication | 9/9 (100) |
| 15a. Ground truth—rationale for choosing the reference standard, if alternatives exist (N = 0) | n/a |
| 15b. Ground truth—definitive ground truth | 9/9 (100) |
| 16. Ground truth—manual image annotation | 5/9 (56) |
| 17. Ground truth—image annotation tools and software | 1/9 (11) |
| 18. Ground truth—measurement of inter- and intra-rater variability; methods to mitigate variability and/or resolve discrepancies | 6/9 (67) |
| 19a. Data partitions—intended sample size and how it was determined | 9/9 (100) |
| 19b. Data partitions—provided power calculation | 0/9 (0) |
| 19c. Data partitions—distinct study participants | 3/9 (33) |
| 20. Data partitions—how data were assigned to partitions; specify proportions | 3/9 (33) |
| 21. Data partitions—level at which partitions are disjoint (e.g., image, study, patient, institution) | 9/9 (100) |
| 22a. Model—provided reproducible model description | 8/9 (89) |
| 22b. Model—provided source code | 0/9 (0) |
| 23. Model—software libraries, frameworks, and packages | 5/9 (56) |
| 24. Model—initialization of model parameters (e.g., randomization, transfer learning) | 0/9 (0) |
| 25. Training—details of training approach, including data augmentation, hyperparameters, number of models trained | 8/9 (89) |
| 26. Training—method of selecting the final model | 7/9 (78) |
| 27. Training—ensembling techniques, if applicable (N = 5) | 5/5 (100) |
| 28. Evaluation—metrics of model performance | 9/9 (100) |
| 29. Evaluation—statistical measures of significance and uncertainty (e.g., confidence intervals) | 6/9 (67) |
| 30. Evaluation—robustness or sensitivity analysis | 1/9 (11) |
| 31. Evaluation—Methods for explainability or interpretability (e.g., saliency maps), and HOW they were validated | 2/9 (22) |
| 32. Evaluation—validation or testing on external data | 1/9 (11) |
| Section 4: Results | 19/54 (35) |
| 33. Data—flow of participants or cases, using a diagram to indicate inclusion and exclusion | 3/9 (33) |
| 34. Data—demographic and clinical characteristics of cases in each partition | 3/9 (33) |
| 35a. Model performance—test performance | 5/9 (56) |
| 35b. Model performance—benchmark of performance | 2/9 (22) |
| 36. Model performance—estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) | 6/9 (67) |
| 37. Model performance—failure analysis of incorrectly classified cases | 0/9 (0) |
| Section 5: Discussion | 17/18 (94) |
| 38. Study limitations, including potential bias, statistical uncertainty, and generalizability | 8/9 (89) |
| 39. Implications for practice, including the intended use and/or clinical role | 9/9 (100) |
| Section 6: Other information | 1/27 (4) |
| 40. Registration number and name of registry | 0/9 (0) |
| 41. Where the full study protocol can be accessed | 0/9 (0) |
| 42. Sources of funding and other support; role of funders | 1/9 (11) |
In the cases where a score of one point per item was obtained, the study was considered to have basic adherence to each item. The adherence rate was calculated as proportion of the number of articles with basic adherence to number of total articles. During the calculation, the “if alternatives exist” item (15a) and “if applicable” item (27) were excluded from both the denominator and numerator. The bolded numbers indicated the sum of sections or CLAIM
CLAIM checklist for artificial intelligence in medical imaging