Table 5. Item-wise correlation between reporting status and online publication year.
|
CLAIM items1 |
Pre- and post-publication of CLAIM |
Post-publication of CLAIM |
||||
|
rho |
P |
flag2 |
rho |
P |
flag2 |
|
|
Item#1 (AI methodology and technology type in title) |
−0.097 |
0.046 |
* |
−0.074 |
0.281 |
- |
|
Item#2 (Structured study summary) |
0.034 |
0.491 |
- |
0.022 |
0.748 |
- |
|
Item#3 (Background and clinical role of AI) |
−0.038 |
0.435 |
- |
0.071 |
0.300 |
- |
|
Item#4 (Study objectives and hypotheses) |
−0.131 |
0.007 |
** |
−0.162 |
0.018 |
* |
|
Item#5 (Prospective or retrospective design) |
0.092 |
0.060 |
- |
0.017 |
0.806 |
- |
|
Item#6 (Study goal, e.g., model creation, feasibility) |
−0.098 |
0.045 |
* |
0.046 |
0.502 |
- |
|
Item#7 (Data sources) |
0.024 |
0.626 |
- |
−0.009 |
0.899 |
- |
|
Item#8 (Eligibility criteria, e.g., inclusion/exclusion) |
−0.055 |
0.261 |
- |
−0.014 |
0.842 |
- |
|
Item#9 (Data pre-processing) |
−0.086 |
0.078 |
- |
−0.217 |
0.001 |
** |
|
Item#10 (Data subset selection, if applicable) |
0.191 |
<0.001 |
*** |
0.225 |
<0.001 |
*** |
|
Item#11 (Definitions of data elements) |
−0.220 |
<0.001 |
*** |
−0.057 |
0.405 |
- |
|
Item#12 (De-identification methods) |
0.099 |
0.042 |
* |
0.068 |
0.322 |
- |
|
Item#13 (Handling of missing data) |
0.134 |
0.006 |
** |
0.110 |
0.110 |
- |
|
Item#14 (Ground truth definition) |
−0.057 |
0.240 |
- |
−0.137 |
0.045 |
* |
|
Item#15 (Rationale for reference standard) |
−0.205 |
<0.001 |
*** |
−0.069 |
0.312 |
- |
|
Item#16 (Source and qualifications of annotators) |
−0.078 |
0.111 |
- |
−0.153 |
0.025 |
* |
|
Item#17 (Annotation tools) |
−0.244 |
<0.001 |
*** |
−0.092 |
0.180 |
- |
|
Item#18 [Variability assessment (inter/intra-rater)] |
−0.211 |
<0.001 |
*** |
−0.121 |
0.078 |
- |
|
Item#19 (Sample size determination) |
0.220 |
<0.001 |
*** |
0.250 |
<0.001 |
*** |
|
Item#20 (Data partitioning method) |
0.140 |
0.004 |
** |
−0.112 |
0.102 |
- |
|
Item#21 (Partition level, e.g., image, patient) |
0.345 |
<0.001 |
*** |
0.116 |
0.091 |
- |
|
Item#22 [Model description (inputs, outputs, layers)] |
0.036 |
0.462 |
- |
−0.109 |
0.110 |
- |
|
Item#23 (Software and frameworks used) |
−0.127 |
0.009 |
** |
−0.134 |
0.050 |
- |
|
Item#24 (Model parameter initialization) |
−0.124 |
0.011 |
* |
−0.176 |
0.010 |
* |
|
Item#25 (Training details, e.g., augmentation, hyperparameters) |
0.141 |
0.004 |
** |
−0.123 |
0.073 |
- |
|
Item#26 (Final model selection) |
−0.057 |
0.246 |
- |
−0.117 |
0.088 |
- |
|
Item#27 (Ensemble techniques, if applicable) |
0.186 |
<0.001 |
*** |
0.167 |
0.014 |
* |
|
Item#28 (Model performance metrics) |
−0.076 |
0.119 |
- |
−0.129 |
0.060 |
- |
|
Item#29 (Statistical significance and uncertainty) |
0.026 |
0.594 |
- |
0.060 |
0.386 |
- |
|
Item#30 (Robustness/sensitivity analysis) |
0.022 |
0.656 |
- |
0.015 |
0.831 |
- |
|
Item#31 (Explainability methods, e.g., saliency maps) |
0.222 |
<0.001 |
*** |
−0.002 |
0.982 |
- |
|
Item#32 (External validation/testing) |
0.009 |
0.846 |
- |
0.098 |
0.152 |
- |
|
Item#33 (Participant flow diagram) |
0.356 |
<0.001 |
*** |
0.202 |
0.003 |
** |
|
Item#34 (Demographic/clinical characteristics by partition) |
0.195 |
<0.001 |
*** |
0.126 |
0.065 |
- |
|
Item#35 (Performance metrics for optimal model) |
−0.020 |
0.684 |
- |
−0.097 |
0.158 |
- |
|
Item#36 (Diagnostic accuracy estimates) |
0.101 |
0.039 |
* |
0.172 |
0.012 |
* |
|
Item#37 (Failure analysis) |
0.075 |
0.125 |
- |
−0.009 |
0.892 |
- |
|
Item#38 [Study limitations (bias, uncertainty, generalizability)] |
0.173 |
<0.001 |
*** |
0.107 |
0.119 |
- |
|
Item#39 (Practice implications and clinical role) |
−0.270 |
<0.001 |
*** |
−0.298 |
<0.001 |
*** |
|
Item#40 (Registration number and registry name) |
−0.141 |
0.004 |
** |
−0.093 |
0.174 |
- |
|
Item#41 (Study protocol access) |
0.160 |
0.001 |
** |
−0.075 |
0.275 |
- |
|
Item#42 (Funding sources and funder roles) |
0.207 |
<0.001 |
*** |
0.092 |
0.178 |
- |
1 Note that item names have been abbreviated; 2* P < 0.05; ** P < 0.01; *** P < 0.001. CLAIM, Checklist for Artificial Intelligence in Medical Imaging.