Table 1.

Number of AEs identified by clinician reviewers using two approaches to AE classification

Approach Used for Medical Record Review	RN/Medic1	RN/Medic2	RN/Medic3	MC-EMP1	MC-EMP2	MC-EMP3	Reference criterion Sr-EMPs
Approach 1: Independent Reviews (n=50 charts)
Medical Records Triggered	21 (42%)	34 (68%)	33 (66%)	28 (56%)	38 (76%)	31 (62%)	32 (64%)
Total Number of Triggered Events	33	65	58	63	69	47	60
Medical Records with AE	12 (24%)	21 (42%)	24 (48%)	18 (36%)	33 (66%)	5 (10%)	20 (40%)
Total Number of AEs	19	30	38	31	50	5	26
-AEs Potential for Harm	19	30	36	30	50	5	25
-AEs Harm Identified	0	0	2	1	0	0	1
Kappa (95% CI)	0.11 (−0.16-0.37)	0.13 (−0.14-0.41)	0.19 (−0.08-0.46)	0.20 (−0.07-0.48)	0.21 (−0.02-0.44)	0.25 (0.02-0.47)	Ref
Percent Agreement	66%	58%	60%	62%	58%	68%	Ref
Sensitivity ∣ Specificity	30% ∣ 80%	50% ∣ 63%	60% ∣ 60%	50% ∣ 70%	80% ∣ 43%	25% ∣ 97%	Ref
PPV ∣ NPV	50% ∣ 63%	48% ∣ 66%	50% ∣ 69%	53% ∣ 68%	48% ∣ 76%	83% ∣ 66%	Ref
Approach 2: Grouped Reviews (n=20 charts)
Medical Records Triggered		17 (85%)			19 (95%)		17 (85%)
Total Number of Triggered Events		50			42		33
Medical Records with AE		12 (60%)			11 (55%)		10 (50%)
Total Number of AEs		36			18		13
-AEs Potential for Harm		35			16		12
-AEs Harm Identified		1			2		1
Kappa (95% CI)		0.40 (0.01-0.79)			0.30 (−0.12-0.72)		Ref
Percent Agreement		70%			65%		Ref
Sensitivity ∣ Specificity		80% ∣ 60%			70% ∣ 60%		Ref
PPV ∣ NPV		67% ∣ 75%			64% ∣ 66%		Ref