Table A1. Implementation codes list.

No.	Authors	Pretext task	Implementation
1	Dosovitskiy et al. (2015)	Exemplar CNN	caffe*
2	Doersch, Gupta & Efros (2015)	Relative position prediction	caffe*—pytorch
3	Noroozi & Favaro (2016)	Jigsaw puzzle	pytorch
4	Komodakis & Gidaris (2018)	Rotation prediction	pytorch*
5	Vincent et al. (2008)	Denoising auto-encoder	theano—pytorch
6	Pathak et al. (2016)	Image inpainting	caffe*—tensorflow
7	Zhang, Isola & Efros (2016)	Image colorization	pytorch*
8	Zhang, Isola & Efros (2017)	Split-brain auto-encoder	caffe*—tensorflow
9	Radford, Metz & Chintala (2016)	Deep Convolutional GAN	pytorch—tensorflow
10	Donahue, Krähenbühl & Darrell (2016)	Bi-directional GAN	theano*—tensorflow
11	van den Oord, Li & Vinyals (2018)	CPC	pytorch—tensorflow
12	He et al. (2020)	MoCo	pytorch*—tensorflow*
13	Chen et al. (2020)	SimCLR	tensorflow*—pytorch
14	Grill et al. (2020)	BYOL	tensorflow*—pytorch
15	Caron et al. (2020)	SwAV	pytorch*—tensorflow
16	Taleb et al. (2020)	CPC Jigsaw puzzle Exemplar CNN Rotation Prediction Relative position prediction	tensorflow*
17	Li et al. (2021)	Rotation prediction multi-view instance discriminate	tensorflow*
18	Sriram et al. (2021)	MoCo	pytorch*
19	Sowrirajan et al. (2021)	MoCo	pytorch*
20	Xie et al. (2020)	BYOL	pytorch*
21	Chaitanya et al. (2020)	SimCLR	tensorflow*
22	Zhou et al. (2019)	Models Genesis	tensorflow*
23	Haghighi et al. (2020)	Semantic Genesis	tensorflow*
24	Li et al. (2020a)	Feature-based softmax embedding	pytorch*
25	Holmberg et al. (2020)	Cross modal retinal thickness prediction	tensorflow*
26	Matzkin et al. (2020)	skull reconstruction	pytorch*
27	Zhang et al. (2021)	TS-SSL	tensorflow*
28	Prakash et al. (2020)	Images denoising	tensorflow*