Table 3.

Measures of Correspondence For Comparisons of Manual Chart Review vs. Automated Extraction of EHR Data and Medicaid Claims Data

Measure (N=150 patients per measure)	Agreement (%)	Kappa (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
BMI assessment
MCR vs. EHR, received	100	1.0	1.0	1.0	1.0	1.0
MCR vs. claims, received	12.7	0.002 (−0.002, 0.006)	0.008 (0.0002, 0.04)	1.0	1.0	0.12 (0.07, 0.18)
Blood pressure assessment
MCR vs. EHR, received	100	1.0	1.0	1.0	1.0	1.0
MCR vs. claims, received	0.7	--	--	1.0	--	0.007 (0.0002, 0.04)
Smoking status assessment
MCR vs. EHR, received	94.0	0.66 (0.45, 0.86)	0.97 (0.93, 0.99)	0.67 (0.38, 0.88)	0.96 (0.92, 0.99)	0.71 (0.42, 0.92)
MCR vs. claims, received	10.0	--	--	1.0	--	0.10 (0.06, 0.16)
Hyperlipidemia screeninga
MCR vs. EHR, ordered	94.0	0.87 (0.78, 0.95)	0.88 (0.77, 0.96)	0.97 (0.91, 0.99)	0.94 (0.83, 0.99)	0.94 (0.88, 0.98)
MCR vs. EHR, received	99.3	0.98 (0.96, 1.0)	1.0	0.99 (0.95, 1.0)	0.98 (0.89, 1.0)	1.0
MCR vs. claims, received	94.0	0.87 (0.78, 0.95)	0.98 (0.89, 1.0)	0.92 (0.85, 0.97)	0.85 (0.73, 0.94)	0.99 (0.94, 1.0)
Diabetes screeningb
MCR vs. EHR, ordered	92.7	0.85 (0.76, 0.93)	0.93 (0.84, 0.98)	0.92 (0.85, 0.97)	0.89 (0.78, 0.95)	0.95 (0.89, 0.99)
MCR vs. EHR, received	98.0	0.96 (0.91, 1.0)	0.98 (0.90, 1.0)	0.98 (0.93, 1.0)	0.96 (0.88, 1.0)	0.99 (0.94, 1.0)
MCR vs. claims, received	94.0	0.87 (0.79, 0.95)	0.95 (0.85, 0.99)	0.94 (0.87, 0.98)	0.89 (0.79, 0.96)	0.97 (0.91, 0.99)
Influenza vaccinationc
MCR vs. EHR, received	98.0	0.96 (0.91, 1.0)	0.97 (0.88, 1.0)	0.99 (0.94, 1.0)	0.98 (0.91, 1.0)	0.98 (0.92, 1.0)
MCR vs. claims, received	88.0	0.74 (0.63, 0.85)	0.75 (0.62, 0.85)	0.97 (0.91, 0.99)	0.94 (0.83, 0.99)	0.85 (0.77, 0.92)
Chlamydia screeningd
MCR vs. EHR, ordered	98.7	0.97 (0.93, 1.0)	1.0	0.98 (0.92, 1.0)	0.97 (0.89, 1.0)	1.0
MCR vs. EHR, received	98.0	0.96 (0.91, 1.0)	0.98 (0.91, 1.0)	0.98 (0.92, 1.0)	0.97 (0.89, 1.0)	0.99 (0.94, 1.0)
MCR vs. claims, received	79.3	0.59 (0.47, 0.72)	0.92 (0.82, 0.97)	0.71 (0.61, 0.80)	0.68 (0.57, 0.78)	0.93 (0.84, 0.98)
Cervical cancer screeninge
MCR vs. EHR, ordered	93.3	0.83 (0.73, 0.93)	0.81 (0.67, 0.92)	0.98 (0.93, 1.0)	0.95 (0.82, 0.99)	0.93 (0.87, 0.97)
MCR vs. EHR, received	97.3	0.93 (0.87, 1.0)	0.93 (0.80, 0.98)	0.99 (0.95, 1.0)	0.97 (0.86, 1.0)	0.97 (0.92, 1.0)
MCR vs. claims, received	86.7	0.67 (0.54, 0.80)	0.78 (0.62, 0.89)	0.90 (0.83, 0.95)	0.74 (0.59, 0.86)	0.92 (0.85, 0.96)
Breast cancer screeningf
MCR vs. EHR, ordered	98.0	0.95 (0.89, 1.0)	0.93 (0.81, 0.99)	1.0	1.0	0.97 (0.92, 0.99)
MCR vs. EHR, received	80.0	0.42 (0.27, 0.57)	0.36 (0.22, 0.51)	0.99 (0.95, 1.0)	0.94 (0.71, 1.0)	0.78 (0.70, 0.85)
MCR vs. claims, received	91.3	0.80 (0.70, 0.90)	0.91 (0.79, 0.98)	0.91 (0.84, 0.96)	0.82 (0.69, 0.91)	0.96 (0.90, 0.99)
Colorectal cancer screeningg
MCR vs. EHR, ordered	96.7	0.91 (0.84, 0.99)	0.88 (0.73, 0.96)	1.0	1.0	0.96 (0.90, 0.99)
MCR vs. EHR, received	90.7	0.62 (0.44, 0.79)	0.50 (0.31, 0.69)	1.0	1.0	0.90 (0.83, 0.94)
MCR vs. claims, received	93.3	0.79 (0.66, 0.91)	0.85 (0.67, 0.96)	0.95 (0.90, 0.98)	0.80 (0.61, 0.92)	0.97 (0.92, 0.99)

Notes: 95% CIs were obtained using exact binomial CIs for proportions

-- Statistic cannot be computed due to zero cell count in claims data Eligibility was based recommended age, sex, and medical history, not whether patient was due for a service

^aMales and females ages 20-64; cholesterol screening includes LDL, HDL, total cholesterol, and triglycerides

^bMales and females ages 45-64

^cMales and females ages 50-64

^dSexually active females ages 19-24

^eFemales ages 19-64 with no history of hysterectomy

^fFemales ages 40-64 with no history of bilateral mastectomy

^gMales and females ages 50-64 with no history of colorectal cancer or total colectomy; colorectal cancer screening includes fecal occult blood test, flexible sigmoidoscopy, and colonoscopy

EHR, electronic health record; MCR, manual chart review; PPV, positive predictive value; NPV, negative predictive value