Table 1.
Quantitative comparisons.
NJUD datasets | NLPR datasets | STERE datasets | SIP datasets | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
MAE | FM | SM | EM | MAE | FM | SM | EM | MAE | FM | SM | EM | MAE | FM | SM | EM | |
PCF | 0.059 | 0.872 | 0.877 | 0.924 | 0.044 | 0.841 | 0.874 | 0.925 | 0.064 | 0.86 | 0.875 | 0.925 | 0.071 | 0.838 | 0.842 | 0.901 |
MMCI | 0.079 | 0.852 | 0.858 | 0.915 | 0.059 | 0.815 | 0.856 | 0.913 | 0.068 | 0.863 | 0.873 | 0.927 | 0.086 | 0.818 | 0.833 | 0.897 |
CPFP | 0.053 | 0.877 | 0.879 | 0.926 | 0.036 | 0.867 | 0.888 | 0.932 | 0.051 | 0.874 | 0.879 | 0.925 | 0.064 | 0.851 | 0.85 | 0.903 |
DMRA | 0.051 | 0.886 | 0.886 | 0.927 | 0.031 | 0.879 | 0.899 | 0.947 | 0.066 | 0.847 | 0.835 | 0.911 | 0.085 | 0.821 | 0.806 | 0.875 |
D3Net | 0.041 | 0.9 | 0.9 | 0.95 | 0.025 | 0.897 | 0.912 | 0.953 | 0.046 | 0.891 | 0.899 | 0.938 | 0.063 | 0.861 | 0.86 | 0.909 |
UCNet | 0.043 | 0.895 | 0.897 | 0.936 | 0.025 | 0.903 | 0.92 | 0.956 | 0.039 | 0.899 | 0.903 | 0.944 | 0.051 | 0.879 | 0.875 | 0.919 |
SSF | 0.043 | 0.896 | 0.899 | 0.935 | 0.026 | 0.896 | 0.914 | 0.953 | 0.044 | 0.89 | 0.893 | 0.936 | 0.053 | 0.88 | 0.874 | 0.921 |
S2MA | 0.053 | 0.889 | 0.894 | 0.93 | 0.03 | 0.902 | 0.915 | 0.95 | 0.051 | 0.882 | 0.89 | 0.932 | 0.054 | 0.884 | 0.878 | 0.92 |
CoNET | 0.047 | 0.892 | 0.895 | 0.937 | 0.031 | 0.887 | 0.908 | 0.945 | 0.04 | 0.904 | 0.908 | 0.948 | 0.063 | 0.867 | 0.858 | 0.913 |
cmMS | 0.044 | 0.897 | 0.9 | 0.936 | 0.027 | 0.896 | 0.915 | 0.949 | 0.042 | 0.891 | 0.895 | 0.937 | 0.061 | 0.871 | 0.867 | 0.091 |
DisenFuse | 0.052 | 0.897 | 0.889 | 0.914 | 0.035 | 0.895 | 0.9 | 0.933 | 0.054 | 0.887 | 0.883 | 0.915 | 0.068 | 0.866 | 0.859 | 0.899 |
ICNet | 0.051 | 0.903 | 0.895 | 0.901 | 0.028 | 0.919 | 0.922 | 0.945 | 0.054 | 0.897 | 0.891 | 0.911 | 0.063 | 0.882 | 0.864 | 0.903 |
CMWNet | 0.046 | 0.913 | 0.903 | 0.923 | 0.029 | 0.913 | 0.917 | 0.941 | 0.043 | 0.911 | 0.905 | 0.93 | 0.062 | 0.89 | 0.867 | 0.909 |
BBSNet | 0.039 | 0.926 | 0.916 | 0.937 | 0.026 | 0.921 | 0.923 | 0.948 | 0.046 | 0.901 | 0.896 | 0.928 | 0.056 | 0.892 | 0.874 | 0.912 |
CDNet | 0.038 | 0.919 | 0.913 | 0.94 | 0.024 | 0.925 | 0.93 | 0.954 | 0.041 | 0.909 | 0.903 | 0.938 | 0.06 | 0.888 | 0.862 | 0.905 |
DCF2 | 0.038 | 0.917 | 0.903 | 0.941 | 0.023 | 0.917 | 0.921 | 0.956 | 0.037 | 0.915 | 0.905 | 0.943 | 0.052 | 0.9 | 0.873 | 0.921 |
DANet | 0.048 | 0.88 | 0.891 | 0.932 | 0.029 | 0.903 | 0.915 | 0.953 | 0.048 | 0.881 | 0.892 | 0.93 | 0.054 | 0.884 | 0.878 | 0.92 |
A2dele | 0.052 | 0.872 | 0.868 | 0.914 | 0.031 | 0.875 | 0.89 | 0.937 | 0.043 | 0.885 | 0.885 | 0.935 | 0.07 | 0.834 | 0.829 | 0.889 |
PGAR | 0.045 | 0.905 | 0.906 | 0.94 | 0.028 | 0.898 | 0.918 | 0.948 | 0.044 | 0.893 | 0.903 | 0.936 | 0.059 | 0.877 | 0.875 | 0.914 |
DFM-Net | 0.042 | 0.91 | 0.906 | 0.947 | 0.026 | 0.908 | 0.923 | 0.957 | 0.045 | 0.893 | 0.898 | 0.941 | 0.051 | 0.887 | 0.883 | 0.926 |
DSA2F | 0.039 | 0.917 | 0.904 | 0.937 | 0.024 | 0.916 | 0.918 | 0.952 | 0.039 | 0.91 | 0.897 | 0.942 | 0.057 | 0.891 | 0.862 | 0.911 |
HAINet | 0.038 | 0.92 | 0.909 | 0.931 | 0.025 | 0.917 | 0.921 | 0.952 | 0.038 | 0.919 | 0.909 | 0.938 | 0.048 | 0.916 | 0.886 | 0.925 |
SSL | 0.038 | 0.923 | 0.909 | 0.939 | 0.025 | 0.923 | 0.922 | 0.96 | 0.039 | 0.914 | 0.904 | 0.939 | 0.046 | 0.909 | 0.888 | 0.927 |
Ours | 0.039 | 0.914 | 0.902 | 0.936 | 0.022 | 0.922 | 0.924 | 0.958 | 0.036 | 0.914 | 0.905 | 0.941 | 0.046 | 0.904 | 0.886 | 0.926 |
Rank | 5 | 7 | 10 | 10 | 1 | 3 | 2 | 2 | 1 | 3 | 3 | 5 | 1 | 3 | 2 | 2 |
TOP3 | Average metrics | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
MAE | FM | SM | EM | |||||||||||||
PCF | 0/16 | 0.060 | 0.853 | 0.867 | 0.919 | |||||||||||
MMCI | 0/16 | 0.073 | 0.837 | 0.855 | 0.913 | |||||||||||
CPFP | 0/16 | 0.051 | 0.867 | 0.874 | 0.922 | |||||||||||
DMRA | 0/16 | 0.058 | 0.858 | 0.857 | 0.915 | |||||||||||
D3Net | 1/16 | 0.044 | 0.887 | 0.893 | 0.938 | |||||||||||
UCNet | 1/16 | 0.040 | 0.894 | 0.899 | 0.939 | |||||||||||
SSF | 0/16 | 0.042 | 0.891 | 0.895 | 0.936 | |||||||||||
S2MA | 0/16 | 0.047 | 0.889 | 0.894 | 0.933 | |||||||||||
CoNET | 2/16 | 0.045 | 0.888 | 0.892 | 0.936 | |||||||||||
cmMS | 0/16 | 0.044 | 0.889 | 0.894 | 0.728 | |||||||||||
DisenFuse | 0/16 | 0.052 | 0.886 | 0.883 | 0.915 | |||||||||||
ICNet | 0/16 | 0.049 | 0.900 | 0.893 | 0.915 | |||||||||||
CMWNet | 1/16 | 0.045 | 0.907 | 0.898 | 0.926 | |||||||||||
BBSNet | 3/16 | 0.042 | 0.910 | 0.902 | 0.931 | |||||||||||
CDNet | 5/16 | 0.041 | 0.910 | 0.902 | 0.934 | |||||||||||
DCF2 | 7/16 | 0.038 | 0.912 | 0.901 | 0.940 | |||||||||||
DANet | 0/16 | 0.045 | 0.887 | 0.894 | 0.934 | |||||||||||
A2dele | 0/16 | 0.049 | 0.867 | 0.868 | 0.919 | |||||||||||
PGAR | 0/16 | 0.044 | 0.893 | 0.901 | 0.935 | |||||||||||
DFM-Net | 4/16 | 0.041 | 0.900 | 0.903 | 0.943 | |||||||||||
DSA2F | 1/16 | 0.040 | 0.909 | 0.895 | 0.936 | |||||||||||
HAINet | 9/16 | 0.037 | 0.918 | 0.906 | 0.937 | |||||||||||
SSL | 10/16 | 0.037 | 0.917 | 0.906 | 0.941 | |||||||||||
Ours | 11/16 | 0.036 | 0.914 | 0.904 | 0.940 | |||||||||||
Rank | 1 | 1 | 3 | 3 | 3 |
For MAE, the lower, the better. On the contrary, for FM, SM and EM, the higher, the better. The second row from the bottom refers to the evaluated scores of our proposed method and the last row refers to the rank of our proposed method.