Table 3.

Post-hoc Manual Review of Top Scoring Records

	High Evidence	Some Evidence	Low Evidence
Top comorbidity scores not included in top text-based scores	N = 2 concordant with initial manual review	N = 6 one record reclassified as High Evidence 5 records contained possible problematic opioid use that did not reach diagnostic criteria for OUD (e.g., possible opioid complication, escalating dose, non-opioid substance abuse)	N = 2 No additional evidence of OUD on follow up review Both records contained complex medical histories including severe depression
Top text-based scores not included in top comorbidity scores	N = 2 concordant with initial manual review*	N = 4 Four records contained possible problematic opioid use that did not reach diagnostic criteria for OUD (e.g., potential opioid related complication, escalating dose)	N = 4 No additional evidence of OUD on follow up review Three records contained complex medical histories

Note: OUD- opioid use disorder.

^*of note, although these records contained documentation of OUD (e.g. opioid overdose, violated opioid contact, positive urine drug screen for unprescribed opioids), they did not contain OUD-related ICD Codes (Supplemental Table 1), but did contain alternate ICD codes of ICD-9 965 poisoning by opium-unspecified, and ICD-10 Z79.891 long term [current] use of opiate analgesic.