Accuracy, Validity, and Reliability of the Infrared Optical Head Tracker (IOHT)

Introduction

We were interested to read the paper by Kim et al., published in the March 2012 issue of Investigative Ophthalmology & Visual Science.¹ The authors aimed to evaluate the accuracy, validity, and reliability of a newly developed infrared optical head tracker (IOHT) using Nintendo Wii remote controllers (WiiMote; Nintendo Co. Ltd., Kyoto, Japan) for measurement of the angle of head posture. They reported that, in comparison with the data from the cervical range of motion (CROM), the IOHT-derived results showed high consistency.¹ Why did the authors not use seven well known statistical tests to evaluate validity of IOHT compared to CROM as gold standard (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV], likelihood ratio positive and negative [LR+ and LR−], and odds ratio)?^2–5 Moreover, high consistency between IOHT and CROM results neither prove reliability nor validity of IOHT.^2–5 They reported that correlation coefficients of 1- and 3-D positions between the IOHT and the CROM were more than 0.99 and 0.96 (P < 0.05, Pearson's correlation test), respectively.¹ Pearson's correlation has nothing to do with reliability as well as validity, and actually is one of the common mistakes in reliability analysis.^2–5 As the authors point out in their conclusion, the IOHT showed strong concordance with the CROM, thus proving its validity and reliability as a head-posture-measuring device.¹ Why did they not use intra-class correlation coefficient (ICC) exact type or weighted kappa to evaluate the reliability of IOHT?^2–5 It is good to know that validity (accuracy) and reliability (precision) are two completely different issues being evaluated with different tests that should be interpreted with caution.^2–5