Table 4.

Performance comparison of experts’ OC segmentation vs. mean, median, majority, and STAPLE OC ground-truth. The performance indices are the best for the STAPLE algorithm, followed by majority, median, and mean ground-truths.

Experts’ annotation vs. Mean ground-truth
	Se	Sp	Ac	Ep	J	D
Expert 1	0.7556	1.0000	0.9988	0.0012	0.7556	0.8537
Expert 2	0.6202	1.0000	0.9977	0.0023	0.6202	0.7507
Expert 3	0.6709	1.0000	0.9983	0.0017	0.6709	0.7919
Expert 4	0.6100	1.0000	0.9977	0.0023	0.6100	0.7446
Expert 5	0.7890	1.0000	0.9990	0.0010	0.7890	0.8714
Average	0.6891	1.0000	0.9983	0.0017	0.6891	0.8025
Experts’ annotation vs. Median ground-truth
Expert 1	0.9450	0.9994	0.9990	0.0010	0.8092	0.8906
Expert 2	0.8407	0.9998	0.9987	0.0013	0.7894	0.8787
Expert 3	0.9014	0.9997	0.9990	0.0010	0.8328	0.9067
Expert 4	0.8466	0.9999	0.9989	0.0011	0.8112	0.8932
Expert 5	0.9460	0.9992	0.9989	0.0011	0.7803	0.8730
Average	0.8959	0.9996	0.9989	0.0011	0.8046	0.8884
Experts’ annotation vs. Majority ground-truth
Expert 1	0.9561	0.9993	0.9991	0.0009	0.8184	0.8950
Expert 2	0.8546	0.9998	0.9988	0.0012	0.8039	0.8866
Expert 3	0.9284	0.9996	0.9992	0.0008	0.8557	0.9196
Expert 4	0.8672	0.9999	0.9991	0.0009	0.8376	0.9086
Expert 5	0.9589	0.9992	0.9990	0.0010	0.7931	0.8792
Average	0.9130	0.9996	0.9990	0.0010	0.8217	0.8978
Experts’ annotation vs. STAPLE ground-truth
Expert 1	0.9900	0.9988	0.9987	0.0012	0.7782	0.8666
Expert 2	0.9545	0.9995	0.9992	0.0007	0.8742	0.9280
Expert 3	0.9822	0.9991	0.9990	0.0009	0.8313	0.9015
Expert 4	0.9591	0.9995	0.9993	0.0006	0.8853	0.9364
Expert 5	0.9957	0.9985	0.9985	0.0014	0.7460	0.8450
Average	0.9763	0.9990	0.9989	0.0009	0.8230	0.8955