Table 4. Expert evaluation on simulated conversations.
| Expert 1 average rating (n=30) | Expert 2 average rating (n=30) |
Interrater reliability (%) and average absolute difference in rating | |
|---|---|---|---|
| Dimension (Yes=0; No=1) | |||
| Barrier identification accuracy | 0.93 | 0.90 | 80a |
| Tactic comprehensiveness | 0.70 | 0.90 | 80a |
| Dimension (5-point Likert), mean (SD) | |||
| Tactic personalization | 4.38 (0.94) | 4.79 (0.49) | 0.78b |
| Tactic actionability | 4.17 (1.10) | 4.59 (0.63) | 0.89b |
| Conversation empathy | 4.58 (0.73) | 4.76 (0.44) | 0.56b |
a
Values correspond to interrater reliability.
b
Values correspond to average absolute difference in rating.