We appreciate the readers’ interest in our article and thank them for sharing their academic insights. Limitations of the Cohen’s Kappa are well known and have been described extensively elsewhere [1]. Limitations include that Kappa can under-estimate reliability for rare occurrences (such as the cancer diagnoses we examine, although in fact our calculated Kappa statistics are rather large). Nonetheless, Kappa is a widely used statistic that can be informative, and it serves as a minor component of our results that can be compared to findings in other studies of other populations. Note that our study does include what we consider a gold standard – cancer diagnosis as reported by the Alaska Native Tumor Registry (ANTR) – and, as such, measures of reliability like the Kappa statistic are only presented as an aside to our primary conclusions (examining reliability if the tumour registry were considered to be incomplete). We found that self-reported prior cancer diagnoses showed good agreement with registry-reported cancer history, with some variation in agreement among different demographic groups.
For the reader who may disagree that the ANTR is the gold standard, we also submitted multiple other measures of agreement Positive Predictive Value/Negative Predictive Value (PPV/NPV, sensitivity/specificity). These measures are helpful if one understands the important relationship between them. As noted in Koller, et al. [2], “Sensitivity and specificity of self-report against the medical record is equivalent to the PPV and NPV (respectively) of the medical record [in this case, information in the ANTR abstracted from the medical record] to self-report.” If all four measurements with a single referent are provided, the reciprocal relationship between them allows the reader to visualise the values for both referents, which enhances comparison with other similar studies. We offer Table 1, as shown in Koller, et al. [2].
Table 1.
Equivalents when gold standard is reversed
| Medical record | Self-report |
|---|---|
| Gold standard | |
| Sensitivity | PPV |
| Specificity | NPV |
| PPV | Sensitivity |
| NPV | Specificity |
Funding Statement
This work was supported by the National Cancer Institute [HHSN26120130010I, CA88958, CA96095].
Disclosure statement
No potential conflict of interest was reported by the authors.
References
- [1].Feinstein AR, Cicchetti DV.. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43:543–1. [DOI] [PubMed] [Google Scholar]
- [2].Koller KR, Wilson AS, Asay ED, et al. Agreement between self-report and medical record prevalence of 16 chronic conditions in the Alaska Native EARTH study. J Prim Care Community Health. 2014;5(3):160–165. [DOI] [PubMed] [Google Scholar]