Modality transformation between MR and CT when a fixed anatomy is combined with a modality vector derived from each imaging modality. Specifically let xmr, xct be MR and CT images respectively. The left panel of the figure shows the original MR image xmr, and a ‘reconstruction’ of xmr using the modality component derived from xct, i.e. g(fanatomy(xmr), fmodality(xct, fanatomy(xct))). The right panel of the figure shows the original CT image xct, and a ‘reconstruction’ of xct using the modality component derived from xmr, i.e. g( fanatomy(xct), fmodality(xmr, fanatomy(xmr))).