Table 3.
Agreement scores, weighted kappas, total and variance for each step of the development process. [CI=confidence intervals].
| Agreement score (95% CI) | Weighted kappa (95% CI) | Total variance | Variance of total variance (%) | |||
|---|---|---|---|---|---|---|
|
| ||||||
| Country | Job | Rater | ||||
| Independent expert rating (Step 1) | ||||||
| Number | 0.49 (0.46–0.51) | 0.51 (0.49–0.52) | 1.02 | 0.0 | 51.1 | 48.9 |
| Nature of contacts | 0.70 (0.67–0.72) | 0.64 (0.64–0.64) | 0.69 | 0.3 | 64.3 | 35.3 |
| Contaminated workspaces | 0.59 (0.57–0.62) | 0.65 (0.64–0.66) | 1.21 | 0.0 | 65.1 | 34.9 |
| Location | 0.71 (0.69–0.74) | 0.56 (0.55–0.58) | 1.50 | 2.3 | 55.9 | 41.8 |
| Social distancing | 0.58 (0.55–0.60) | 0.67 (0.67–0.67) | 0.88 | 0.0 | 67.1 | 32.9 |
| Face covering | 0.46 (0.44–0.49) | 0.17 (0.12–0.22) | 0.76 | 14.0 | 19.0 | 67.0 |
| Group expert meeting and revision (Step 2) | ||||||
| Number | 0.50 (0.48–0.53) | 0.54 (0.52–0.55) | 1.05 | 0.0 | 54.1 | 45.9 |
| Nature of contacts | 0.68 (0.66–0.70) | 0.63 (0.63–0.63) | 0.73 | 0.0 | 63.5 | 36.6 |
| Contaminated workspaces | 0.59 (0.57–0.62) | 0.63 (0.62–0.64) | 1.20 | 0.0 | 62.8 | 37.1 |
| Location | 0.71 (0.68–0.73) | 0.55 (0.54–0.56) | 1.54 | 0.7 | 55.0 | 44.4 |
| Social distancing | 0.54 (0.52–0.57) | 0.66 (0.65–0.66) | 0.94 | 0.2 | 65.6 | 34.1 |
| Face covering | 0.54 (0.51–0.56) | 0.34 (0.31–0.37) | 0.79 | 7.5 | 34.3 | 58.2 |
| Independent country rating (Step 3) | ||||||
| Number | 0.55 (0.53–0.58) | 0.60 (0.56–0.64) | 0.99 | 2.7 | 97.3 | |
| Nature of contacts | 0.74 (0.71–0.76) | 0.70 (0.70–0.70) | 0.68 | 0.0 | 100.0 | |
| Contaminated workspaces | 0.68 (0.66–0.71) | 0.75 (0.75–0.75) | 1.27 | 0.0 | 100.0 | |
| Location | 0.74 (0.72–0.76) | 0.55 (0.50–0.60) | 1.43 | 0.9 | 99.2 | |
| Social distancing | 0.63 (0.60–0.65) | 0.74 (0.74–0.74) | 0.89 | 0.0 | 100.0 | |
| Face covering | 0.70 (0.67–0.72) | 0.56 (0.50–0.61) | 0.67 | 2.0 | 97.9 | |
| Consensus between countries (Step 4) | ||||||
| Number | 0.58 (0.55–0.61) | 0.65 (0.63–0.68) | 0.97 | 0.3 | 99.5 | |
| Nature of contacts | 0.76 (0.74–0.78) | 0.76 (0.76–0.76) | 0.68 | 0.4 | 99.6 | |
| Contaminated workspaces | 0.70 (0.68–0.73) | 0.80 (0.80–0.80) | 1.25 | 1.9 | 98.0 | |
| Location | 0.77 (0.75–0.79) | 0.65 (0.64–0.66) | 1.40 | 0.8 | 98.9 | |
| Social distancing | 0.66 (0.63–0.68) | 0.79 (0.79–0.79) | 0.89 | 0.0 | 95.7 | |
| Face covering | 0.72 (0.69–0.74) | 0.60 (0.60–0.60) | 0.66 | 0.3 | 99.5 | |
| Income insecurity | 0.66 (0.64–0.69) | 0.53 (0.42–0.64) | 0.52 | 1.2 | 98.8 | |
| Migrants | 0.27 (0.25–0.29) | 0.14 (-0.07–0.36) | 0.71 | 4.8 | 95.2 | |