Fig. 2: Overview of the Ensemble Integration (EI) framework for multimodal data.

In the implementation of EI tested in this work, we used ten standard binary classification algorithms, such as support vector machine (SVM), random forest (RF) and logistic regression (LR), as implemented in Weka (Frank et al. (2005)), to derive sets of local predictive models 1, 2, …, N from the data modalities 1, 2, …, N. We then applied the stacking and ensemble selection methods to these local models to generate the EI models. These models generated prediction scores for the entities and multimodal data of interest that were evaluated to assess their performance. Finally, we used our novel interpretation method to identify the features that contributed the most substantially to the best-performing EI model’s predictions.