Table 1.
TDS and AIPI domain-specific scores for each LLM and answer
| ChatGPT 4 | Gemini Advanced | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TDS | QAMAI | TDS | QAMAI | |||||||||||
| Acc | Cla | Rel | Com | Ref | Use | Acc | Cla | Rel | Com | Ref | Use | |||
| Case 1—oral cavity SCC | 3 | 3 | 3 | 4 | 3 | 1 | 3 | 3 | 3 | 4 | 2 | 1 | 1 | 2 |
| Case 2—larynx SCC | 3 | 3 | 4 | 3 | 2 | 1 | 2 | 4 | 2 | 3 | 2 | 2 | 1 | 2 |
| Case 3—oropharynx SCC | 2 | 4 | 4 | 4 | 4 | 1 | 4 | 0 | 3 | 4 | 3 | 3 | 1 | 3 |
| Case 4—nasopharynx SCC | 2 | 2 | 3 | 2 | 2 | 1 | 2 | 2 | 3 | 4 | 2 | 2 | 1 | 2 |
| Case 5—salivary gland PDC | 3 | 4 | 5 | 4 | 4 | 1 | 4 | 0 | 3 | 4 | 3 | 2 | 1 | 2 |
Acc accuracy, QAMAI Quality Assessment of Medical Artificial Intelligence, Cla clarity, Com completeness, PCD poorly differentiated carcinoma, Rel relevance, Ref reference, SCC squamous cell carcinoma, TDS total disagreement score, Use usefulness