Figure 1.

Overview of PA estimation and evaluation. The phenotyping algorithm (PA) is first estimated to obtain the scores (S). Patient charts from the electronic health record (EHR) warehouse are reviewed to obtain the gold-standard label (Y) for PA evaluation. In classical supervised ROC analysis (supROC), only the labeled data from chart review is used to evaluate the PA’s performance. Semi-supervised ROC analysis (ssROC) uses the labeled data to impute the missing Y as $\widehat{m}(S)$ so that the unlabeled data can be utilized for estimation to yield more precise estimates of the ROC parameters.