Fig. 2. Research workflow for developing an active surveillance system. First, total HOI diagnosis dates were extracted using the NHIS claim dataset. The period of 14 days before the HOI diagnosis was set as a risk window. The control window was randomly selected for 14 days excluding the risk window and washout period. If a HOI occurred several times, the HOI that re-occurred within 31 days was considered as the same and a continuous event of the previous HOI. In order to ensure independence between HOIs, a risk window was defined only for recurrent HOI events where the interval between HOIs was more than 6 months apart. Second, the prescription and vaccination information that occurred in each window was collected. If there was no prescription and vaccination information in the window, the window was removed. At the final step, the HOI prediction model was learned using the prescription and vaccination information in each window. After that, using the feature importance ratio and LIME analysis of the model, a suspected drug or vaccine that could cause the HOI was determined.

FI = feature importance, HOI = health outcome of interest, KDCA = Korea Disease Control and Prevention Agency, LIME = Local interpretable Model-agnostic Explanation, NHIS = National Health Insurance Service, SIDER = Side Effect Resource database.