TABLE 5.

Overview of RL in medical image registration

Author	ROI	Modality	Algorithm	State	Action number	Reward design
75	Nasopharynx	CT‐MR	A3C	3D tensor composed of the moving and the fixed image	8	Distance between the transformed landmark and the reference landmark
74	Prostate	MR‐MR	Q‐learning	Spatial transformation parameters	30	The reduction of distance between the current parameters to the ground truth parameters
71	Spine, Cardiac	CT‐CBCT	Q‐learning	The rigid‐body transformation matrix	12	The reduction of distance between the current transformation to the ground truth transformation
84	Chest, Abdomen	CT‐Depth Image	Dueling DQN	3D tensor including cropped image pair	6	Small constant during the exploration, and big constant at termination with sign determined by change of constant
83	Brain, Liver	MR‐MR, CT‐CT	SPAC	Pair of fixed images and moving image	*	The change of the dice score
73	Spine, Cardiac	CT‐CBCT	Dilated FCN	The rigid‐body transformation matrix	12	The reduction of distance between the current transformation to the ground truth transformation
72	Spine	CBCT‐Xray	Q‐learning	Transformations in SE(3)	12	Reduction of distance to the ground truth transformation
85	Nasopharynx	CT‐MR	A3C	Concatenation of the fixed and moving image	8	Distance between the transformed landmark and the reference landmark

*Indicate that the missing part is not clearly defined in the original paper.