Errors in Methodology Affect Diagnostic Accuracy of High-Speed Videomicroscopy Analysis in Primary Ciliary Dyskinesia

To the Editor:

The study by Rubbo et al¹ in a recent issue of CHEST (May 2019) explores accuracy of high speed video-microscopy analysis (HSVA) as a primary ciliary dyskinesia (PCD) diagnostic test across three centers. The authors compare 120 randomly selected HSVA interpretations (from six specifically selected video clips per patient out of an unreported clip number) by three blinded experts against two reference standards: 1) The European Respiratory Society (ERS) PCD guidelines (diagnosis by electron microscopy and/or genetic testing) or 2) a multidisciplinary team opinion (MDT), incorporating all available diagnostic tests. The authors report sensitivity and specificity of HSVA at 96% to 100% and 91% to 97%, respectively; however, critical methodology biases are inflating these estimates.²

With technicians selecting six video clips per patient, rather than randomly selecting from an eligible pool of clips, inappropriate exclusion of clips more difficult to diagnose (gray zone) results in overestimation of diagnostic accuracy from selection bias in the HSVA index test. Moreover, by not performing HSVA on three occasions or after cellular regrowth, the authors are not following the strong ERS guideline recommendation, and secondary dyskinesia artifact is probable.³ Adherence to this recommendation would negate the authors’ claim of same-day PCD diagnosis via HSVA.

Data from the ERS reference standard reveal genetic testing was only performed in 16 of 120 patients (13% total) and in 8 of 84 patients (9.5% at most) who are not “PCD positive” by HSVA. This lack of universal genetic testing creates differential verification bias, inflating diagnostic accuracy up to 60%.⁴ Nondiagnostic HSVA exists in at least six PCD genes (HYDIN, CCDC164, DNAH9, GAS8, CCNO, and MCIDAS),⁵ and universal genetic testing may have detected these cases, thus decreasing sensitivity. The alternative MDT reference standard incorporates the index test of HSVA into the MDT reference standard, creating a crucial methodology error and affecting accuracy estimates. Although the authors acknowledge this limitation, this does not justify its use when alternative reference standards are easily performed.

This report shows that HSVA interpretation has good interobserver agreement among world experts when cases result in highly characteristic beat patterns. Yet, agreement decreases when beat patterns are uncharacteristic, with an overall agreement that is moderate at best (ĸ = 0.42). Thus, the true diagnostic accuracy of HSVA in the era of PCD genomics remains unknown. We urge the authors to conduct prospective, multisite HSVA studies using samples postcellular regrowth, using standardized HSVA interpretations, incorporating known disease controls (such as cystic fibrosis or immunodeficiency), and choosing nonbiased reference standards, including extended genetic panel testing, to ensure patients with PCD receive accurate diagnostic testing.