. 2019 Oct 11;43(1):57–66. doi: 10.1007/s40264-019-00869-4

Table 3.

Performance of neural networks

Source data type	Seriousness classification (accuracy)	Seriousness categorization (F1 score)	Annotation of seriousness category terms (F1 score)
Post-marketing	83.0% (precision = 0.95, recall = 0.74)	Death—0.78 (precision = 0.88, recall = 0.70) Hospitalization—0.79 (precision = 0.84, recall = 0.74) IME—0.76 (precision = 0.81, recall = 0.72)	NC
Solicited reports	92.9% (precision = 0.87, recall = 0.87)	NC	0.90 (precision = 0.88, recall = 0.91)
Medical literature	86.3% (precision = 0.83, recall = 0.82)	NC	0.75 (precision = 0.62, recall = 0.96)

Source data type

Seriousness classification (accuracy)

Seriousness categorization (F1 score)

Annotation of seriousness category terms (F1 score)

Post-marketing

83.0% (precision = 0.95, recall = 0.74)

Death—0.78 (precision = 0.88, recall = 0.70)

Hospitalization—0.79 (precision = 0.84, recall = 0.74)

IME—0.76 (precision = 0.81, recall = 0.72)

Solicited reports

92.9% (precision = 0.87, recall = 0.87)

0.90 (precision = 0.88, recall = 0.91)

Medical literature

86.3% (precision = 0.83, recall = 0.82)

0.75 (precision = 0.62, recall = 0.96)

IME important medical event, NC not calculated