Table 9.
Final results of portability assessment of code‐based and NLP‐enhanced algorithms
| KPNW | KPW | Optum | TennCare | |
|---|---|---|---|---|
| OOD | ||||
| Sensitivity | 97.2 | 100.0 | 96.9 | 99.2 |
| Specificity | 84.6 | 89.2 | 100.0 | 92.4 |
| PPV | 87.4 | 84.1 | 100.0 | 91.9 |
| NPV | 96.5 | 100.0 | 96.9 | 99.2 |
| F‐score | 0.92 | 0.92 | 0.98 | 0.95 |
| Heroin | ||||
| Sensitivity | 91.8 | 87.5 | 98.5 | N/A |
| Specificity | 99.0 | 99.3 | 100.0 | N/A |
| PPV | 94.7 | 93.3 | 100.0 | N/A |
| NPV | 98.4 | 98.5 | 99.5 | N/A |
| F‐score | 0.93 | 0.90 | 0.99 | N/A |
| Suicide/suicide attempt | ||||
| Sensitivity | 70.5 | 74.1 | 63.2 | 44.9 |
| Specificity | 90.2 | 86.7 | 91.0 | 87.1 |
| PPV | 78.9 | 74.1 | 81.1 | 64.5 |
| NPV | 85.5 | 86.7 | 80.1 | 75.1 |
| F‐score | 0.74 | 0.74 | 0.67 | 0.53 |
| Suicide/suicide attempt—NLP enhanced | ||||
| Sensitivity | 78.7 | 81.5 | N/A | N/A |
| Specificity | 91.0 | 95.2 | N/A | N/A |
| PPV | 81.9 | 89.8 | N/A | N/A |
| NPV | 89.2 | 90.9 | N/A | N/A |
| F‐score | 0.80 | 0.85 | N/A | N/A |
| Abuse involved | ||||
| Sensitivity | 75.3 | N/A | 67.1 | 41.5 |
| Specificity | 79.5 | N/A | 31.9 | 62.5 |
| PPV | 74.0 | N/A | 46.2 | 29.0 |
| NPV | 80.5 | N/A | 52.6 | 74.4 |
| F‐score | 0.76 | N/A | 0.55 | 0.34 |