Table 5:
nk = 1000, weakly heterogeneous covariate distribution
CFS | CCOD | CP | s.d.(CP) | wd | s.d.(wd) | CP | s.d.(CP) | wd | s.d.(wd) |
---|---|---|---|---|---|---|---|---|---|
| |||||||||
Homoscedasticity where σ(x) = 1 | |||||||||
Federated I |
Pooled sample |
||||||||
holds | 0.897 | 0.022 | 3.28 | 0.21 | 0.899 | 0.011 | 3.29 | 0.09 | |
ASR | weakly violated | 0.896 | 0.025 | 3.27 | 0.22 | 1.000 | 0.000 | 9.23 | 0.26 |
strongly violated | 0.894 | 0.027 | 3.26 | 0.24 | 1.000 | 0.000 | 24.51 | 0.96 | |
holds | 0.899 | 0.024 | 3.30 | 0.22 | 0.899 | 0.015 | 3.29 | 0.12 | |
Local ASR | weakly violated | 0.897 | 0.027 | 3.28 | 0.23 | 0.832 | 0.022 | 2.76 | 0.14 |
strongly violated | 0.897 | 0.024 | 3.29 | 0.22 | 0.736 | 0.030 | 2.25 | 0.14 | |
holds | 0.909 | 0.035 | 3.62 | 3.28 | 0.899 | 0.015 | 3.30 | 0.13 | |
CQR | weakly violated | 0.911 | 0.037 | 3.56 | 0.91 | 0.903 | 0.020 | 3.34 | 0.20 |
strongly violated | 0.907 | 0.036 | 3.53 | 1.28 | 0.900 | 0.023 | 3.32 | 0.22 | |
Federated II (ours) |
Target site only |
||||||||
holds | 0.899 | 0.019 | 3.29 | 0.18 | 0.901 | 0.025 | 3.33 | 0.23 | |
ASR | weakly violated | 0.898 | 0.021 | 3.28 | 0.19 | 0.900 | 0.025 | 3.31 | 0.23 |
strongly violated | 0.896 | 0.022 | 3.27 | 0.20 | 0.899 | 0.025 | 3.31 | 0.24 | |
holds | 0.900 | 0.022 | 3.32 | 0.21 | 0.903 | 0.032 | 3.37 | 0.31 | |
Local ASR | weakly violated | 0.899 | 0.024 | 3.29 | 0.22 | 0.902 | 0.031 | 3.35 | 0.31 |
strongly violated | 0.899 | 0.022 | 3.30 | 0.20 | 0.901 | 0.032 | 3.35 | 0.32 | |
holds | 0.910 | 0.030 | 3.58 | 2.62 | 0.905 | 0.036 | 3.40 | 0.35 | |
CQR | weakly violated | 0.912 | 0.032 | 3.53 | 0.72 | 0.904 | 0.034 | 3.39 | 0.34 |
strongly violated | 0.908 | 0.030 | 3.50 | 1.01 | 0.904 | 0.035 | 3.39 | 0.35 | |
Federated III |
Equal weights |
||||||||
holds | 0.902 | 0.021 | 3.33 | 0.20 | 0.905 | 0.024 | 3.37 | 0.25 | |
ASR | weakly violated | 0.901 | 0.021 | 3.31 | 0.20 | 0.903 | 0.023 | 3.34 | 0.23 |
strongly violated | 0.899 | 0.022 | 3.30 | 0.20 | 0.902 | 0.024 | 3.34 | 0.23 | |
holds | 0.906 | 0.023 | 3.39 | 0.30 | 0.913 | 0.027 | 3.53 | 0.88 | |
Local ASR | weakly violated | 0.906 | 0.022 | 3.37 | 0.22 | 0.913 | 0.027 | 3.48 | 0.54 |
strongly violated | 0.905 | 0.021 | 3.36 | 0.21 | 0.912 | 0.024 | 3.45 | 0.38 | |
holds | 0.923 | 0.027 | 3.65 | 1.09 | 0.936 | 0.032 | 3.94 | 1.28 | |
CQR | weakly violated | 0.925 | 0.027 | 3.64 | 0.52 | 0.936 | 0.032 | 3.92 | 0.94 |
strongly violated | 0.922 | 0.026 | 3.61 | 0.47 | 0.935 | 0.033 | 3.94 | 1.02 | |
| |||||||||
Heteroscedasticity where | |||||||||
Federated I |
Pooled sample |
||||||||
holds | 0.907 | 0.022 | 4.16 | 0.58 | 0.908 | 0.011 | 4.12 | 0.24 | |
ASR | weakly violated | 0.905 | 0.022 | 4.12 | 0.54 | 0.985 | 0.003 | 9.53 | 0.32 |
strongly violated | 0.901 | 0.025 | 4.03 | 0.57 | 1.000 | 0.000 | 24.61 | 0.95 | |
holds | 0.921 | 0.040 | 3.86 | 0.54 | 0.929 | 0.020 | 3.91 | 0.31 | |
Local ASR | weakly violated | 0.922 | 0.033 | 3.84 | 0.50 | 0.859 | 0.025 | 3.04 | 0.22 |
strongly violated | 0.918 | 0.038 | 3.78 | 0.47 | 0.767 | 0.031 | 2.39 | 0.17 | |
holds | 0.894 | 0.136 | 3.84 | 1.73 | 0.872 | 0.107 | 3.23 | 0.35 | |
CQR | weakly violated | 0.885 | 0.142 | 3.81 | 2.35 | 0.879 | 0.104 | 3.26 | 0.36 |
strongly violated | 0.882 | 0.143 | 3.74 | 1.45 | 0.867 | 0.121 | 3.29 | 0.46 | |
Federated II (ours) |
Target site only |
||||||||
holds | 0.908 | 0.018 | 4.17 | 0.50 | 0.907 | 0.023 | 4.17 | 0.62 | |
ASR | weakly violated | 0.907 | 0.018 | 4.13 | 0.47 | 0.908 | 0.022 | 4.18 | 0.57 |
strongly violated | 0.903 | 0.021 | 4.06 | 0.50 | 0.906 | 0.022 | 4.16 | 0.60 | |
holds | 0.923 | 0.034 | 3.88 | 0.50 | 0.922 | 0.047 | 3.95 | 0.74 | |
Local ASR | weakly violated | 0.925 | 0.029 | 3.87 | 0.45 | 0.926 | 0.041 | 3.97 | 0.68 |
strongly violated | 0.921 | 0.032 | 3.82 | 0.43 | 0.925 | 0.041 | 3.97 | 0.71 | |
holds | 0.904 | 0.121 | 3.77 | 1.37 | 0.849 | 0.179 | 3.47 | 0.81 | |
CQR | weakly violated | 0.899 | 0.119 | 3.75 | 1.86 | 0.857 | 0.172 | 3.51 | 0.77 |
strongly violated | 0.896 | 0.123 | 3.69 | 1.14 | 0.857 | 0.172 | 3.51 | 0.80 | |
Federated III |
Equal weights |
||||||||
holds | 0.911 | 0.019 | 4.26 | 0.54 | 0.914 | 0.020 | 4.36 | 0.60 | |
ASR | weakly violated | 0.910 | 0.019 | 4.23 | 0.52 | 0.913 | 0.021 | 4.33 | 0.60 |
strongly violated | 0.908 | 0.020 | 4.19 | 0.54 | 0.911 | 0.022 | 4.31 | 0.63 | |
holds | 0.932 | 0.030 | 4.04 | 0.55 | 0.940 | 0.030 | 4.24 | 0.87 | |
Local ASR | weakly violated | 0.934 | 0.028 | 4.03 | 0.54 | 0.940 | 0.029 | 4.28 | 1.76 |
strongly violated | 0.932 | 0.028 | 4.01 | 0.50 | 0.939 | 0.030 | 4.30 | 1.88 | |
holds | 0.941 | 0.086 | 4.03 | 1.14 | 0.957 | 0.074 | 4.67 | 1.77 | |
CQR | weakly violated | 0.944 | 0.071 | 3.97 | 0.97 | 0.956 | 0.069 | 4.67 | 2.34 |
strongly violated | 0.941 | 0.088 | 3.96 | 0.87 | 0.958 | 0.067 | 4.80 | 3.17 |
CFS: conformal score; CCOD: common conditional outcome distribution
CP: coverage probability; wd: width; s.d.: standard deviation (over 500 replications)