TABLE 2.
Interclass Correlation Coefficients by Reviewer Type and Subdomaina
| ASCO | ESMO | ICER | NCCN | |||||
|---|---|---|---|---|---|---|---|---|
| All reviewers (n = 8), ICC (95% CI) | 0.800 | (0.660-0.913) | 0.818 | (0.686-0.921) | 0.652 | (0.466-0.834) | 0.153 | (0.045-0.371) |
| Oncologists vs. non-oncologists | ||||||||
| Oncologists (n = 4) | 0.807 | (0.638-0.920) | 0.842 | (0.699-0.936) | 0.769 | (0.582-0.903) | 0.210 | (0.020-0.501) |
| Other (n = 4) | 0.786 | (0.605-0.911) | 0.816 | (0.655-0.924) | 0.603 | (0.353-0.817) | 0.156 | (0b-0.427) |
| Physicians vs. nonphysicians | ||||||||
| Physicians (n = 6) | 0.825 | (0.686-0.926) | 0.831 | (0.698-0.929) | 0.641 | (0.439-0.830) | 0.156 | (0.031-0.395) |
| Other (n = 2) | 0.740 | (0.375-0.905) | 0.691 | (0.302-0.884) | 0.482 | (0.023-0.784) | 0.198 | (0b-0.597) |
| By subdomain | ||||||||
| Clinical benefit | 0.829 | (0.704-0.927) | 0.809 | (0.673-0.917) | - | (-) | 0.149 | (0.041-0.368) |
| Toxicity | 0.755 | (0.592-0.891) | 0.597 | (0.406-0.800) | - | (-) | 0.194 | (0.067-0.432) |
| Quality of life | 0.671 | (0.490-0.844) | 0.818 | (0.686-0.921) | - | (-) | - | (-) |
| Certainty | - | (-) | - | (-) | 0.062 | (0b-0.247) | 0.022 | (0b-0.129) |
aICC and CI are shown as measure of framework reliability.
bNegative ICC estimate was observed, which suggested that the true ICC is very low; ICC of zero was therefore assumed.34
ASCO = American Society of Clinical Oncology; CI = confidence interval; ESMO = European Society for Medical Oncology; ICC = interclass correlation coefficient; ICER = Institute for Clinical and Economic Review.