Table 3.

Agreement metrics between software and panel assessment.

Metric	Value	Note
ICC3k	0.857	High level of agreement; 95% CI = [0.77, 0.91], p < 0.01
Cohen's kappa	0.62	Substantial agreement
Spearman's rho	0.73	Strong positive correlation; p < 0.01
Kendall's tau	0.56	Strong positive correlation; p < 0.01
Agreement on worst 5%	80.00%	—
Agreement on top 20%	65.00%	—
Linear weighted kappa	0.51	Moderate agreement
Quadratic weighted kappa	0.66	Substantial agreement

Note. The intraclass correlation coefficient (ICC3k) measures the overall agreement between the software and the panel. Cohen's kappa, Spearman's rho, and Kendall's tau indicate the consistency in ranking and grading between the two methods. The linear and quadratic weighted kappa values assess the degree of agreement, factoring in the severity of disagreements.