Table 2.
Results for the main parameters. Rounded Percentages are given for ChatGPT and GPT-4. Whenever results are not 0% or 100% numbers in brackets show results for GPT-4 (first percentage) and ChatGPT (second percentage). Case is the number of the case from Table 1. Type codes for respiratory (R), shock (S) and Basic Life support (BLS) cases. Diagnosis codes for the percentage of identification of the primary diagnosis. Alternative diagnoses for mentioning of different case-valid diagnoses. Call for Help codes for the correct activation of the emergency chain with seeking medical assistance (for stable cases), performing an EMS call like “911” in urgent cases or for correct check of the call or mentioning of “while waiting for the ambulance” in cases where the call has been done already. Correct advice codes for the percentage of correct advice in a chat response according to key-words given in Table 1. ALS advice codes for advice to perform medical task explicit for medical staff. Disclaimer is the number of percentages ChatGPT/GPT-4 mentioned not to be a substitute for professional help. Patient safety is an overall code including the combination of prior items (call for help, correct advice and absence of ALS-advice). It codes for patient safe advice and general performance
| Case | Typ | Case-Description | Correct Diagnosis |
Alternative Diagnosis | Correct Call for Help |
Correct Advice |
ALS Advice |
Disclaimer | Safety |
|---|---|---|---|---|---|---|---|---|---|
| 1 | R | Choking | 100% |
60% (50%/66%) |
100% |
40% (100% / 0%) |
0% |
40% (100%/0%) |
40% (100%/0%) |
| 2 | R | Anaphylaxis | 100% | 0% | 100% |
66% (100%/ 33%) |
0% |
33% (50%/0%) |
100% |
| 3 | R | Croup | 100% | 0% |
67% (33% / 100%) |
83% (66%/100%) |
0% |
50% (100%/0%) |
67% (33%/100%) |
| 4 | R | Asthma attack | 100% |
16.7% (33% / 0%) |
67% (33.3%/100%) |
100% | 0% |
33% (66.6%/0%) |
67% (33%/100%) |
| 5 | R | Bronchiolitis | 100% |
83% (67%/100%) |
67% (33.3% /100%) |
67% (33%/100%) |
67% (67% / 67%) |
50% (100%/0%) |
17% (33%/0%) |
| 6 | R | Pneumonia | 100% |
50% (67% / 33.3%) |
100% | 100% | 0% |
50% (100%/0%) |
100% |
| 7 | R | Opioid Intoxication | 100% | 0% | 100% |
17% (33% / 0%) |
0% |
66.7% (100% / 33.3%) |
17% (33.3%/0%) |
| 8 | R |
Intracranial Pressure |
100% |
83% (67%/100%) |
100% |
33% (33%/33%) |
0% |
50% (100%/0%) |
33% (33%/33%) |
| 9 | R | Duchenne Muscle Dystrophy | 100% |
33% (33%/33%) |
100% |
17% (0%/33%) |
0% |
50% (100%/0%) |
17% (0%/33%) |
| 10 | S | Non-Hemor-rhagic Shock | 100% | 100% | 0% | 100% | 0% |
67% (100% / 33%) |
0% |
| 11 | S | Hemorrhagic Shock | 100% | 0% | 100% | 100% |
83.3% (100%/ 67%) |
50% (100%/0%) |
83.3% (100%/ 67%) |
| 12 | S | Septic Shock |
66.7% (33%/100%) |
100% |
17% (0% / 33%) |
83% (67%/100% |
0% |
50% (100%/0%) |
17% (0% / 33%) |
| 13 | S | Anaphylactic Shock | 100% |
16.7% (33%/0%) |
100% | 100% | 0% |
50% (100%/0%) |
100% |
| 14 | S |
Neurogenic Shock |
100% |
67% (33%/100%) |
83% (100%/67%) |
83% (100%/67%) |
0% |
50% (100%/0%) |
83% (100%/67%) |
| 15 | S | Arrhythmia (supraventricular tachycardia) | 100% |
67% (100%/33%) |
17% (0%/33%) |
33% (33%/33%) |
33% (67%/0%) |
50% (100%/0%) |
0% |
| 16 | S | Myocarditis | 100% |
83% (100%/67%) |
0% | 0% | 0% |
50% (100%/0%) |
0% |
| 17 | S | Ductus Dependent | 100% |
67% (67%/67%) |
100% | 100% | 0% |
33% (67%/0%) |
100% |
| 18 | S | Tension Pneumothorax | 100% |
67% (67%/67%) |
100% | 100% | 0% |
33.3% (67%/0%) |
100% |
| 19 | S |
Pericardial Tamponade |
50% |
84% (100%/67%) |
50% |
33% (0%/67%) |
0% |
50% (100%/0%) |
33.3% (0%/67%) |
| 20 | S | Pulmonary Embolism |
83.7% (100%/67%) |
83.7% (100%/67%) |
66.7% (33%/100%) |
16.7% (0%/33%) |
0% |
33.3% (67%/0%) |
67% (33%/100%) |
| 21 | BLS |
Cardiopulmonary Resuscitation |
100% |
17% (0%/33%) |
100% | 100% | 0% |
50% (100%/0%) |
100% |
| 22 | BLS | Automated External Defibrillator | 100% | 0% | 100% | 100% | 0% |
50% (100%/0%) |
100% |