Table 2.

Properties and benefits of different fusion strategies.

	Early	Joint	Late
Able to make predictions when not all modalities are present	×	×a	✓
Able to model interactions between features from different modalities	✓	✓	×
Able to learn more compatible features from each modality	×	✓	×
Does not necessarily require a large amount of training data	×	×	✓
Does not require training multiple models	✓b	✓	×
Does not necessarily require meticulous designing efforts	✓	×	✓
Flexibility to join input at different levels of abstraction	×	✓	×

Different properties and benefits for each fusion strategy.

^aSpecialized joint fusion architecture such as Kawahara et al.’s multi-modal multi-task model is capable of handling missing data.

^bEarly fusion requires training of multiple models when the imaging features are extracted using CNN.