Figure 6.

A breakdown of the inputs, outputs, loss functions, and architecture of the three prediction models. Global average pooled feature responses from the clinical and the dermoscopic images are extracted and concatenated (denoted by the plus symbol) with one-hot encoded patient meta-data, and the three models are trained with $L_{\mathrm{DIAG}}$, $L_{\mathrm{MGMT}}$, and $L_{\mathrm{multi}}$ respectively. The first model predicts the diagnosis labels (${\mathrm{DIAG}}_{\mathrm{pred}}$) which are then used along with the management priors to obtain inferred management decisions (${\mathrm{MGMT}}_{\mathrm{infr}}$), whereas the second model predicts the management decisions directly (${\mathrm{MGMT}}_{\mathrm{pred}}$). Finally, the last model is a multi-task one and is trained to predict the seven-point criteria, the diagnosis, and the management (outputs enclosed in the dashed box).