Table 4.
“Out-of-the-box” individual PHI recall results for partial and fully-contained matches using the VHA evaluation corpus
| RULE-BASED SYSTEMS | |||||||
|---|---|---|---|---|---|---|---|
|
PHI type |
#Inst. |
PARTIAL MATCHES |
FULLY-CONTAINED MATCHES |
||||
| |
|
HMS Scrubber |
MeDS |
MIT deid |
HMS Scrubber |
MeDS |
MIT deid |
| Patient Name |
206 |
0.83 |
0.99 |
0.98 |
0.57 |
0.69 |
0.69 |
| Relative Name |
30 |
0.76 |
0.95 |
1 |
0.57 |
0.67 |
0.77 |
| Healthcare Provider Name |
492 |
0.74 |
0.96 |
0.94 |
0.43 |
0.47 |
0.38 |
| Other Person Name |
20 |
0.66 |
0.81 |
0.74 |
0.30 |
0.25 |
0.35 |
| Street City |
137 |
0.90 |
0.96 |
0.81 |
0.70 |
0.78 |
0.78 |
| State Country |
161 |
0.45 |
0.49 |
0.85 |
0.43 |
0.45 |
0.84 |
| Deployment |
43 |
0.34 |
0.49 |
0.27 |
0.07 |
0.02 |
0.05 |
| ZIP code |
4 |
1 |
1 |
1 |
1 |
1 |
1 |
| Healthcare Unit Name |
1453 |
0.45 |
0.51 |
0.12 |
0.24 |
0.23 |
0.03 |
| Other Org Name |
86 |
0.33 |
0.50 |
0.27 |
0.03 |
0.20 |
0.03 |
| Date |
2547 |
0.74 |
0.87 |
0.80 |
0.34 |
0.27 |
0.46 |
| Age > 89 |
4 |
0 |
0 |
1 |
0 |
0 |
1 |
| Phone Number |
90 |
0.73 |
0.79 |
0.80 |
0.42 |
0.5 |
0.48 |
| Electronic Address |
4 |
0 |
0.86 |
0.75 |
0 |
0 |
0.75 |
| SSN |
16 |
1 |
1 |
1 |
1 |
1 |
1 |
| Other ID Number |
123 |
0.66 |
0.82 |
0.41 |
0.43 |
0.61 |
0.27 |
|
MACHINE LEARNING-BASED SYSTEMS | |||||||
| PHI type |
#Inst. |
PARTIAL MATCHES |
FULLY-CONTAINED MATCHES |
||||
| |
|
MIST |
HIDE |
MIST |
HIDE |
||
| Patient Name |
206 |
0.51 |
0.54 |
0.42 |
0.50 |
||
| Relative Name |
30 |
0.13 |
0.13 |
0.13 |
0.13 |
||
| Healthcare Provider Name |
492 |
0.53 |
0.59 |
0.44 |
0.53 |
||
| Other Person Name |
20 |
0 |
0.20 |
0 |
0.15 |
||
| Street City |
137 |
0.26 |
0.29 |
0.26 |
0.27 |
||
| State Country |
161 |
0.14 |
0.22 |
0.14 |
0.21 |
||
| Deployment |
43 |
0.07 |
0.05 |
0.07 |
0.02 |
||
| ZIP code |
4 |
0 |
0.75 |
0 |
0.75 |
||
| Healthcare Unit Name |
1453 |
0.09 |
0.09 |
0.06 |
0.05 |
||
| Other Org Name |
86 |
0.09 |
0.07 |
0.06 |
0.06 |
||
| Date |
2547 |
0.72 |
0.73 |
0.39 |
0.38 |
||
| Age > 89 |
4 |
0 |
0 |
0 |
0 |
||
| Phone Number |
90 |
0.34 |
0.61 |
0.24 |
0.51 |
||
| Electronic Address |
4 |
0 |
0 |
0 |
0 |
||
| SSN |
16 |
0.56 |
0.87 |
0.56 |
0.87 |
||
| Other ID Number | 123 | 0.32 | 0.69 | 0.20 | 0.63 | ||