Table 7:
nk = 300, strongly heterogeneous covariate distribution
CFS | CCOD | CP | s.d.(CP) | wd | s.d.(wd) | CP | s.d.(CP) | wd | s.d.(wd) |
---|---|---|---|---|---|---|---|---|---|
| |||||||||
Homoscedasticity where σ(x) = 1 | |||||||||
Federated I |
Pooled sample |
||||||||
holds | 0.891 | 0.043 | 3.26 | 0.37 | 0.901 | 0.018 | 3.31 | 0.16 | |
ASR | weakly violated | 0.893 | 0.039 | 3.28 | 0.40 | 1.000 | 0.000 | 8.85 | 0.50 |
strongly violated | 0.893 | 0.041 | 3.29 | 0.39 | 1.000 | 0.000 | 24.23 | 1.49 | |
holds | 0.899 | 0.043 | 3.36 | 0.44 | 0.901 | 0.023 | 3.33 | 0.22 | |
Local ASR | weakly violated | 0.897 | 0.044 | 3.34 | 0.48 | 0.843 | 0.037 | 2.86 | 0.27 |
strongly violated | 0.895 | 0.049 | 3.34 | 0.47 | 0.754 | 0.056 | 2.35 | 0.30 | |
holds | 0.923 | 0.064 | 4.55 | 6.23 | 0.903 | 0.025 | 3.35 | 0.24 | |
CQR | weakly violated | 0.922 | 0.055 | 4.49 | 5.18 | 0.904 | 0.043 | 3.41 | 0.42 |
strongly violated | 0.924 | 0.067 | 4.72 | 7.05 | 0.904 | 0.042 | 3.41 | 0.44 | |
Federated II (ours) |
Target site only |
||||||||
holds | 0.893 | 0.038 | 3.28 | 0.34 | 0.898 | 0.048 | 3.35 | 0.47 | |
ASR | weakly violated | 0.895 | 0.034 | 3.29 | 0.34 | 0.898 | 0.043 | 3.34 | 0.41 |
strongly violated | 0.896 | 0.036 | 3.30 | 0.35 | 0.896 | 0.049 | 3.35 | 0.47 | |
holds | 0.903 | 0.040 | 3.39 | 0.43 | 0.904 | 0.058 | 3.50 | 0.66 | |
Local ASR | weakly violated | 0.900 | 0.041 | 3.37 | 0.46 | 0.904 | 0.056 | 3.48 | 0.64 |
strongly violated | 0.900 | 0.043 | 3.37 | 0.44 | 0.905 | 0.058 | 3.50 | 0.63 | |
holds | 0.927 | 0.051 | 4.38 | 4.96 | 0.918 | 0.058 | 3.68 | 0.66 | |
CQR | weakly violated | 0.926 | 0.045 | 4.33 | 4.12 | 0.920 | 0.062 | 3.71 | 0.71 |
strongly violated | 0.928 | 0.055 | 4.52 | 5.62 | 0.920 | 0.060 | 3.71 | 0.69 | |
Federated III |
Equal weights |
||||||||
holds | 0.897 | 0.036 | 3.31 | 0.34 | 0.907 | 0.037 | 3.44 | 0.46 | |
ASR | weakly violated | 0.899 | 0.032 | 3.32 | 0.32 | 0.907 | 0.036 | 3.44 | 0.44 |
strongly violated | 0.900 | 0.034 | 3.34 | 0.35 | 0.908 | 0.039 | 3.46 | 0.47 | |
holds | 0.914 | 0.038 | 3.59 | 1.08 | 0.929 | 0.042 | 4.27 | 3.11 | |
Local ASR | weakly violated | 0.912 | 0.037 | 3.55 | 0.89 | 0.930 | 0.042 | 4.70 | 7.53 |
strongly violated | 0.913 | 0.039 | 3.53 | 0.57 | 0.930 | 0.042 | 4.30 | 4.39 | |
holds | 0.949 | 0.033 | 4.59 | 3.44 | 0.965 | 0.034 | 9.82 | 52.91 | |
CQR | weakly violated | 0.946 | 0.035 | 4.55 | 3.13 | 0.960 | 0.056 | 6.84 | 8.83 |
strongly violated | 0.950 | 0.031 | 4.65 | 4.31 | 0.966 | 0.032 | 6.82 | 7.94 | |
| |||||||||
Heteroscedasticity where | |||||||||
Federated I |
Pooled sample |
||||||||
holds | 0.903 | 0.038 | 4.23 | 1.05 | 0.906 | 0.017 | 4.10 | 0.42 | |
ASR | weakly violated | 0.902 | 0.035 | 4.16 | 0.90 | 0.983 | 0.004 | 9.22 | 0.52 |
strongly violated | 0.904 | 0.034 | 4.21 | 0.95 | 1.000 | 0.000 | 24.37 | 1.53 | |
holds | 0.918 | 0.053 | 3.92 | 0.89 | 0.925 | 0.032 | 3.89 | 0.49 | |
Local ASR | weakly violated | 0.919 | 0.053 | 3.97 | 0.92 | 0.870 | 0.044 | 3.19 | 0.45 |
strongly violated | 0.919 | 0.052 | 3.96 | 0.91 | 0.789 | 0.058 | 2.55 | 0.40 | |
holds | 0.915 | 0.134 | 9.13 | 47.58 | 0.850 | 0.155 | 3.29 | 0.56 | |
CQR | weakly violated | 0.916 | 0.127 | 4.57 | 8.47 | 0.854 | 0.169 | 3.43 | 0.72 |
strongly violated | 0.915 | 0.126 | 5.86 | 16.05 | 0.839 | 0.183 | 3.48 | 0.91 | |
Federated II (ours) |
Target site only |
||||||||
holds | 0.905 | 0.034 | 4.23 | 0.97 | 0.899 | 0.046 | 4.24 | 1.29 | |
ASR | weakly violated | 0.905 | 0.031 | 4.20 | 0.84 | 0.905 | 0.044 | 4.39 | 1.33 |
strongly violated | 0.906 | 0.030 | 4.22 | 0.87 | 0.902 | 0.042 | 4.28 | 1.27 | |
holds | 0.920 | 0.049 | 3.96 | 0.91 | 0.901 | 0.091 | 4.12 | 1.67 | |
Local ASR | weakly violated | 0.923 | 0.050 | 4.04 | 0.94 | 0.913 | 0.085 | 4.31 | 1.70 |
strongly violated | 0.923 | 0.049 | 4.03 | 0.93 | 0.913 | 0.081 | 4.32 | 1.71 | |
holds | 0.925 | 0.120 | 8.10 | 38.03 | 0.855 | 0.213 | 3.97 | 1.46 | |
CQR | weakly violated | 0.926 | 0.116 | 4.49 | 6.77 | 0.877 | 0.197 | 4.18 | 1.51 |
strongly violated | 0.929 | 0.107 | 5.53 | 12.80 | 0.877 | 0.192 | 4.18 | 1.49 | |
Federated III |
Equal weights |
||||||||
holds | 0.910 | 0.031 | 4.38 | 1.00 | 0.918 | 0.031 | 4.69 | 1.25 | |
ASR | weakly violated | 0.909 | 0.030 | 4.34 | 0.86 | 0.915 | 0.030 | 4.52 | 0.96 |
strongly violated | 0.910 | 0.030 | 4.35 | 0.86 | 0.917 | 0.029 | 4.61 | 0.99 | |
holds | 0.936 | 0.044 | 4.27 | 1.02 | 0.951 | 0.038 | 4.99 | 2.81 | |
Local ASR | weakly violated | 0.936 | 0.046 | 4.41 | 1.81 | 0.948 | 0.041 | 5.64 | 9.58 |
strongly violated | 0.938 | 0.047 | 4.42 | 1.66 | 0.950 | 0.040 | 5.40 | 7.17 | |
holds | 0.958 | 0.097 | 7.31 | 18.71 | 0.975 | 0.066 | 16.39 | 86.25 | |
CQR | weakly violated | 0.958 | 0.092 | 5.51 | 7.78 | 0.969 | 0.084 | 13.30 | 53.84 |
strongly violated | 0.969 | 0.057 | 6.29 | 9.57 | 0.979 | 0.043 | 10.97 | 24.62 |
CFS: conformal score; CCOD: common conditional outcome distribution
CP: coverage probability; wd: width; s.d.: standard deviation (over 500 replications)