More work needs to be done to ensure that better pneumonia diagnostics aids are developed and launched to better support frontline health workers–A response to “Are respiratory rate counters really so bad” by Ansermino et al.

We thank Ansermino et al for their valuable contribution to the discussion on evaluating respiratory rate (RR) diagnostic aids. We completely agree when they say “Until the performance of automated counters have been established, the use of respiratory rate counters should not be discarded based on this study alone”. While we did not test automated RR diagnostic aids in our study [1], we did see large variations in the agreement between the four RR diagnostic aids we tested and the reference standard presented, i.e the Masimo Root patient monitoring and connectivity platform with Phasein ISA CO2 capnography using nasal cannulas. While we also had expert clinicians as an additional reference standard, we did not present this data in our article, but we also saw large variations in agreement between all test devices and the expert clinicians counting also. We further agree with the authors and others [2,3], that there are limitations with all current types of RR reference standards, and none should be considered a gold standard. We hope to further document our learnings around reference standards in an upcoming publication with the working title “Challenges in selecting a robust reference standard for validating respiratory rate counting aids – lessons learned from two recent performance studies conducted in hospital settings in Cambodia, Ethiopia, Uganda and South Sudan”, where we will present data on the levels of variation we saw in the different reference standards we used.

While we recognize the difficulties with RR reference standards, and as highlighted in the discussion in our original article [1], one of the other points to consider when looking at variability in performance in the field is that while these devices are programmed in a way to most likely produce accurate results if they were used exactly as per instructions in a controlled environment, once taken into the community and in the hands of CHWs, they produce poorer results. This could be the reason for differences in the performance of Rrate in our study compared to its previous published performance [4]. We further agree with the authors, that one of the challenges when evaluating automated RR diagnostic aids is the fact that they identify breaths which a human observer may not see. This was a challenge we also encountered in a recent evaluation of automated RR counters [5]. This is a key consideration when designing future performance evaluations of these types of devices and one we hope will be discussed further at the planned technical consultation hosted by UNICEF focused on automated RR diagnostic aids.