Table 7.

Alert thresholds and median lead time.^a

Score	Threshold source	Threshold value	Sensitivity	Specificity	PPVb	NPVc	WDRd	F1 scoree	Lead timef (h:min), median (IQR)
EDIg	Singh et al [11]	64.8	0.448	0.917	0.583	0.865	1.71	0.507	32:26 (4:37-66:08)
PICTUREh
	Align by sensitivity	0.165	N/Ai	0.946	0.683	0.869	1.46	0.541	40:14 (7:51-67:50)
	Align by specificity	0.097	0.616	N/A	0.658	0.902	1.52	0.636	40:04 (7:44-91:00)
	Align by PPV	0.048	0.792	0.851	N/A	0.940	N/A	0.668	54:10 (29:26-115:50)
	Align by NPV	0.173	0.432	0.946	0.675	N/A	1.48	0.527	41:40 (7:31-68:30)

^aSensitivity, specificity, PPV, and NPV were calculated for the EDI at a threshold of 64.8 as suggested in Singh et al [11] and based off encounter-level performance. PICTURE thresholds were then aligned to match these statistics. The WDR is also calculated as 1 / PPV and represents the number of false alarms received for each true positive. This value is important in limiting alert fatigue for clinicians and indicates that PICTURE may yield as much as 17% fewer false alarms for each true positive.

^bPPV: positive predictive value.

^cNPV: negative predicative value.

^dWDR: workup to detection ratio.

^eF1 scores were calculated as the harmonic mean between PPV and sensitivity.

^fLead times were determined using the intersection of true positives between PICTURE and the EDI, and were calculated as the time between a patient first crossing the threshold and their first deterioration event.

^gEDI: Epic Deterioration Index.

^hPICTURE: Predicting Intensive Care Transfers and Other Unforeseen Events.

ⁱN/A: not applicable.