Figure 24:

Tradeoff between performance and complexity. Size of circles shows variance in performance and complexity across (a) all datasets and (b) datasets on which we tested >6 approaches. We plot a dotted blue line of best quadratic fit to show the Pareto frontier between performance and complexity. These strong tradeoffs suggest that future work should focus on lightweight multimodal models that generalize throughout datasets in MultiBench. It remains an open question whether several well-performing methods (such as MFAS or MulT) can be successfully adapted to new domains and tasks, since they work much better on more-studied datasets (b) as compared to over all datasets (a).