Table 4.

Accuracy measurements of NLP in identifying Tdap-related local reaction, as compared with chart-confirmed validation data.

Site	Reference Standard (n/N)	Chart confirmation rate (%)	TP	TN	FN	FP	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	LR +	LR−
KPSC	75/250	30.0 24.4–36.1	68	162	7	13	90.7 81.7–96.2	92.6 87.6–96.0	84.0 75.5–89.9	95.9 92.0–97.9	12.2 7.2–20.7	0.10 0.05–0.20
KP site 2	16/69	23.2 13.9–34.9	15	45	1	8	93.8 69.8–99.8	84.9 72.4–93.3	65.2 49.4–78.2	97.8 87.1–99.7	6.2 3.2–11.9	0.07 0.01–0.49
KP site 3	21/79	26.6 17.3–37.3	17	53	4	5	81.0 58.1–94.6	91.4 81.0–97.1	77.3 58.9–89.0	93.0 84.5–97.0	9.4 4.0–22.3	0.21 0.09–0.51
Non-KP site 1	14/75	18.7 10.6–29.3	11	59	3	2	78.6 49.2–95.3	96.7 88.7–99.6	84.6 57.8–95.7	95.2 87.8–98.2	24.0 6.0–96.2	0.22 0.08–0.60
Non-KP site 2	8/27	29.6 13.8–50.2	5	19	3	0	62.5 24.5–91.5	100 82.4–100	100 (NA)	86.4 72.1–93.9	NA	0.38 0.15–0.92
Overalla	134/500	26.8 23.0–30.9	116	338	18	28	86.6 79.6–91.8	92.3 89.1–94.9	80.6 74.3–85.6	94.9 92.4–96.6	11.3 7.9–16.3	0.15 0.09–0.22
Overallb	141/500	28.2 24.3–32.4	124	333	17	26	87.9 81.4–92.8	92.8 89.6–95.2	82.7 76.6–87.4	95.1 92.6–96.8	12.1 8.3–17.7	0.13 0.08–0.20

Note: 95% confidence intervals are displayed beneath point estimates.

N: number of cases reviewed; FN: false negative; FP: false positive; TN: true negative; TP: true positive.

PPV: positive predictive value; NPV: negative predictive value.

LR+/LR−: positive and negative likelihood ratio. A LR + value between 5–10 indicates a moderate increase, while a value larger than 10 indicates a strong and often conclusive increase in the probability. A LR-value between 0.1 – 0.2 indicates a moderate decrease, while a value less than 0.1 indicates a strong and often conclusive decrease of the probability.

^aUnweighted results are reported for the entire validation sample (n = 500).

^bWeighted overall measurements are based on the number of patients with selected diagnosis codes at each site.