Table 3.

Detailed description of 24 studies that assessed rule accuracy

Study	‘Gold standard’			Verification only	Overall rule accuracy in %					PPV range for individual rules
	Chart review (all)	Chart review (rule +)	Other		Subgroup	Sensitivity	Specificity	PPV	NPV
Azaz-Livshits et al, 199813	X					65.8	51.3
Brown et al, 200015				X				11		0–100
Brvar et al, 200916	X							5.8
Dalton-Bunnow et al, 199321				X				∼6
Dormann et al, 200022			X (stimulated voluntary reporting)			74	75	13		0–100
Dormann et al, 200423	X				Single critical laboratory values	91	23	25
					Critical changes in value	40	76	32
Egger et al, 200324	X‡				For detecting ADRs	47.5	1.6	1.8
					For detecting drug interactions	58.3	1.4	0.9
Evans et al, 199126				X				9		0–58
Field et al, 200410 28				X	Rule set			7		0–50
					Text searching			12
Honigman et al, 200111 31		X*				58	88	7.5	99.2	0–66.7
Hope et al, 200332		X			Boston study site			10.2
					Indianapolis study site			9.6
Huang et al, 200533	X†							63.5	99.2
Hwang et al, 200834	X					79.1		21		0–100
Jha et al, 19984	X		X (stimulated voluntary reporting)					17		9–28
Jha et al, 200135				X				3.5		2–100
Jha et al, 200836				X*				50*
Kilbridge et al, 200639				X						0–67
Levy et al, 199942	X					62	42
Nebeker et al, 200744	X				Model for bleeding	86	89	11.5
					Model for delirium/psychosis	94.4	71	2.9
Raschke et al, 199846			X (physician orders)					53		4–97
Seger et al, 200747				X*				12.5
Silverman et al, 200449				X*						0–71
Thuermann et al, 200253	X‡					45.1	78.9			0–100
Whipple et al, 199457				X				3.7

^*Only reviewed a sample of the rule positive alerts.

^†Chart review was conducted in all patients at one of the study sites only.

^‡Chart review was part of a surveillance program.

ADR, adverse drug reaction; NPV, negative predictive value; PPV, positive predictive value.