Accuracy of linear intraoral measurements using cone beam CT and multidetector CT: methodological mistake

We were interested to read the paper by Patcas et al published in the December 2012 issue of DMFR. The authors compared the accuracy of linear bone measurements of cone beam CT (CBCT) with multidetector CT (MDCT) and validated intraoral soft-tissue measurements in CBCT.¹ They reported that radiological measurements are accurate with a mean difference from anatomical measurements of 0.14 mm (CBCT) and 0.23 mm (MDCT). However, limits of agreement for soft-tissue measurements in CBCT were smaller (−0.77 mm; 1.07 mm), indicating a slightly higher accuracy. As the authors point out in their conclusion, CBCT is slightly more reliable for linear measurements than MDCT and less affected by metal artefacts.¹ The common practice for assessing validity (accuracy) is to apply well-known statistical tests, and it is unclear why the authors did not consider employing such practice.^2–6 Moreover, the reliability (precision) of a test is being assessed using different methods from those for validity (accuracy).^2,3

Reliability and validity are two completely different methodological issues in research. Sensitivity, specificity, positive predictive value, negative predictive value, likelihood ratio positive (true positive/false positive) and likelihood ratio negative (false negative/true negative) as well as the odds ratio (true results/false results—preferably greater than 50) and diagnostic accuracy are some of the tests to evaluate the validity (accuracy) of a single test compared with a gold standard.^2–6 Reliability (repeatability or reproducibility) is assessed by different statistical tests such as Pearson's r, least squares and paired t-test, all of which can result in some of the common mistakes in reliability analysis.^2–6 Briefly, for a quantitative variable, intraclass correlation coefficient agreement and for qualitative variables weighted kappa should be used with caution because kappa has its own limitations too. Regarding reliability or agreement, it is good to know that, for computing the kappa value, just concordant cells are being considered, whereas discordant cells should also be taken into account to reach the correct estimation of agreement (weighted kappa).^2–6 Two important weaknesses of the kappa value to assess agreement of a qualitative variable are as follows: it depends on the prevalence in each category and it also depends on the number of categories. So the fewer the categories, the higher the kappa value and vice versa, which can easily lead to misinterpretation.^2–6

It is crucial to know that statistics cannot provide a simple substitute for clinical judgment. As a take-home message, for reliability and validity analysis, appropriate tests should be applied with careful interpretation.