Table 1.
Average accuracy for electronic case report form data entry.
| Type of disease and data element | NLPa only | Manual group (median, IQRb) | NLP-MIESc group (median, IQR) | Logarithmic mean difference (95% CI) | Ratio of change in geometric mean (95% CI) | P value | |
| Congenital heart disease |
|
|
|
|
|
||
|
|
True-false Id | 97.50 | 79.17 (66.74, 84.17) | 96.81 (95.69, 97.29) | 0.41 (0.04 to 0.79) | 1.51 (1.03 to 2.20) | .04 |
|
|
True-false IIe | 92.00 | 95.39 (92.67, 95.89) | 97.78 (97.19, 98.44) | 0.21 (–0.01 to 0.10) | 1.10 (0.99 to 1.24) | .10 |
|
|
Multiple choice | 89.33 | 82.80 (73.13, 85.83) | 95.00 (94.58, 97.42) | 0.29 (0.10 to 0.49) | 1.34 (1.10 to 1.63) | .009 |
|
|
Fill-in-the-blank | 94.17 | 96.33 (95.25, 97.00) | 97.00 (95.83, 97.42) | 0.01 (–0.01 to 0.02) | 1.01 (0.99 to 1.02) | .22 |
|
|
Overall | 92.77 | 90.42 (87.75, 92.68) | 97.17 (96.83, 97.44) | 0.14 (0.03 to 0.25) | 1.15 (1.04 to 2.20) | .03 |
| Pneumonia |
|
|
|
|
|
|
|
|
|
True-false I | 88.00 | 70.83 (65.25, 77.75) | 88.17 (87.25, 89.00) | 0.30 (0.11 to 0.50) | 1.35 (1.11 to 1.65) | .009f |
|
|
True-false II | 94.44 | 91.25 (88.26, 93.78) | 95.83 (95.21, 96.81) | 0.11 (0.01 to 0.21) | 1.12 (1.01 to 1.23) | .04 |
|
|
Multiple choice | 80.83 | 67.50 (50.21, 72.50) | 81.25 (77.92, 85.00) | 0.33 (0.14 to 0.52) | 1.39 (1.15 to 1.68) | .003f |
|
|
Overall | 84.15 | 84.21 (80.53, 86.23) | 92.19 (91.49, 93.20) | 0.17 (0.06 to 0.28) | 1.18 (1.06 to 1.32) | .008 |
aNLP: natural language processing.
bIQR: interquartile range.
cNLP-MIES: NLP-driven medical information extraction system.
dTrue-false I: data elements retrieved from admissions records.
eTrue-false II: data elements retrieved from imaging reports (ultrasonic cardiogram or chest x-ray).
fIndependent group t test.