Fig. 3.

Illustration of the proposed framework for joint lung lobe segmentation and severity assessment of COVID-19 in 3D CT images. Each raw 3D CT image is first pre-processed, and multiple 2D image patches (with each patch from a specific slice) are then extracted to construct an instance bag for representing each input CT scan. This bag is then fed into the proposed multi-task multi-instance UNet (M${}^2$UNet) for joint lung lobe segmentation and severity assessment of COVID-19, consisting of a shared patch-level encoder, a segmentation sub-network, and a classification sub-network for severity assessment. Here, the segmentation task can provide location and tissue guidance for the task of severity assessment that employs a hierarchical multi-instance learning strategy.