Table 2.

Chart review performance of alternative sleep apnea phenotyping algorithms

Method	Training recall (sensitivity)	Training precision (PPV)	Training negative predictive value	Training AUC	Validation recall (sensitivity)	Validation precision	Validation negative predictive value	Validation AUC
≥1 PheCode	1.000	0.689	NA	NA	1.000	0.733	NA	NA
≥2 PheCodes	0.823	0.836	0.621	NA	0.795	0.805	0.455	NA
MAP NLP CUIs	0.774	0.850	0.582	0.819	0.727	0.831	0.442	0.786
PheCAP SICDNLP and NLP CUIs	0.427	0.981	0.437	0.893	0.375	0.943	0.353	0.832
PheCAP SICD	0.387	0.941	0.411	0.790	0.341	0.938	0.341	0.756
PheCAP SNLP	0.331	0.911	0.385	0.820	0.250	0.917	0.313	0.753
PheCAP SICDNLP	0.331	0.911	0.385	0.822	0.250	0.917	0.313	0.754
PheCAP SICDNLP, Demographics, and NLP CUIs	0.403	0.980	0.426	0.892	0.352	0.939	0.345	0.830
PheCAP SICDNLP and NLP CUIs plus AHI and CPAP	0.411	0.981	0.430	0.904	0.443	0.951	0.380	0.858

Note: A total of 300 chart reviews were performed for participants with one or more sleep apnea PheCode codings. Therefore, certain PheCode-only rows lack negative predictive values by definition. Of the 300 chart reviews, 180 (60%) of these results were used in the training set, and 120 (40%) were used in the validation set. Results for the best performing PheCAP model are shown as “PheCAP S_ICDNLP and NLP CUIs,” along with chart review performance for PheCode-only definitions using a minimum of 1 and 2 PheCode instances to define a case and a more basic NLP algorithm using MAP. The performance of PheCAP surrogate-only models is shown next (“PheCAP S_ICD,” “PheCAP S_NLP,” “PheCAP S_ICDNLP”) and is followed by the predictive performance using demographic parameters exclusively. Reduced performance was observed when including demographics and the lead PheCAP model (“PheCAP S_ICDNLP, Demographics, and NLP CUIs”). Additional modest performance gains were obtained by forcing case status for participants with separately extracted AHI and/or continuous positive airway pressure (joint CPAP CUI/procedure term) evidence. Full results for all models are presented in Table S5. Recall (sensitivity) = true positives/(true positives + false negatives). Precision (Positive Predictive Value) = true positives/(true positives + false positives); Negative Predictive Value = true negatives/(true negatives + false negatives).

Abbreviations: AHI: apnea-hypopnea index; AUC: area under the curve; CPAP: continuous positive airway pressure ventilation; CUIs: concept unique identifiers; MAP: multimodal automated phenotyping; NLP: natural language processing.