Algorithm 1. Nash-MTL

Input: ${\theta }_0$- initial parameter vector, ${\left\{l_i\right\}}_{i=1}^K$–differentiable loss functions, $\mu$–learning rate

Output: ${\theta }^T$

for t = 1,…, T do

Compute task gradients $g_i^t={\nabla }_{\theta (t-1)}{l}_i$

Set $G^{(t)}$ the matrix with columns $g_i^{(t)}$

Solve for $\alpha : {(G^t)}^T(G^t)\alpha =1/\alpha$ to obtain ${\alpha }^t$

Update the parameters ${\theta }^{(t)}={\theta }^{(t)}-\mu \ast G^{(t)}{\alpha }^{(t)}$

end for

return ${\theta }^T$