Fig. 1: Overview of early, intermediate and late approaches for integrating multimodal data.

(a) The early approach combines multiple datasets into an intermediate representation, from which predictive models can be inferred. (b) The intermediate approach jointly models the multiple datasets and their elements through an intermediate representation. (c) The late approach first builds local model(s) for each individual modality, which are then integrated into the final predictive model.