Better late than never: correcting the error in the Exhalyzer nitrogen washout system

Multiple-breath washout (MBW) testing has become well established in cystic fibrosis (CF) over the past decade. It has been used as an end point for clinical trials of new therapies (1), a process which has also driven the expansion of the technique into clinical centers and into clinical practice. The system that has benefited most from this was the Exhalyzer nitrogen washout device (Eco Medics AG, Duernten, Switzerland), selected on the basis that it was easy to source 100% oxygen as the washout gas, and that the device had been validated for accuracy (2). Unfortunately, it appears that this was not the whole story, and as two papers published in the Journal show there were substantial and clinically significant errors in how the device calculated the washout nitrogen signal (3, 4).

The Exhalyzer system measures nitrogen (N₂) indirectly, calculating it from the residual of the oxygen (O₂) and carbon dioxide (CO₂) signals. This makes it highly sensitive to errors in the O₂ signal, particularly at the end of washout where small relative errors in high concentrations of O₂ cause large relative errors in the much lower N₂ signal. Wyler et al. (4) and Sandvik et al. (3) have both reported that an unrecognized source of error was the effect of expired CO₂ on the O₂ signal. The effect of this nonlinear error was to reduce the O₂ signal, thereby increasing the N₂ signal and leading to prolongation of washout. The errors reported in the mean functional residual capacity (FRC) were around 12% in both healthy infants and in school-age children with CF. The primary lung function outcome from MBW testing is the lung clearance index (LCI), which was also affected, being falsely elevated by 19% in healthy infants and 15% in children with CF. These are not trivial errors and are furthermore likely to represent a best-case scenario. The reason for this is that the error is nonlinear and is more significant if LCI is already elevated. Baseline LCI in the cohorts that have been reanalyzed was only modestly elevated (mean LCI < 10) and the effects will, therefore, be even greater for those with higher LCI. Since the Exhalyzer has been used to measure LCI in thousands of subjects and in hundreds of studies in the past 8 or 9 years, this could have wide-ranging and highly significant effects on LCI datasets. Of particular importance are studies assessing interventions and those looking at long-term changes in LCI. It is likely that the direction of change will be preserved when these are reanalyzed, but there is no guarantee that the magnitude or consistency of any changes will be preserved. This could have profound effects for the conclusions of some studies or trials.

The truth is, however, that this is not a newly recognized issue and that opportunities have been missed to correct this earlier and avoid the disruption that large-scale data reanalysis will now cause. The errors reported here are not so very different from the 10% difference in healthy control LCI and 20% difference in FRC reported in 2013 when Exhalyzer was first compared with SF₆ using a mass spectrometer (5). The authors also showed that in many subjects, Exhalyzer measured higher FRCs than those detected by plethysmography, something that cannot be physiologically correct. Various explanations have been proffered to explain the differences between Exhalyzer outputs and those from washout using an exogenous tracer gas (SF₆) (5–8). But the Exhalyzer system also gave much higher FRC and LCI than an apparently equivalent nitrogen washout device (9). The possibility that the oxygen analyzer might be at fault was flagged up in this journal in 2015 (10) and more definitively pinpointed in 2018 (11). The reality is that in the understandable enthusiasm to adopt LCI, and move it rapidly to support clinical trials, the growing evidence that the system was inaccurate was not rigorously explored. It is to the credit of the teams who have now published the correction that they took the care to do this and to work with the manufacturers to do so.

There are important learning points from this, and ones that apply to other areas than multiple-breath washout technologies. This raises an important issue of how well (or not) the Exhalyzer D and other commercial devices are validated for accuracy. In particular, it emphasizes the need to use clinically realistic testing models (the original test model used to assess accuracy did not include continuous addition of CO₂). Although clinicians and researchers should continue to work closely with the manufacturers of devices, we also need to recognize their objectives may be different. Nevertheless, the results presented by Wyler et al. (4) and Sandvik et al.(3) serve as a good basis to call for better processes to ensure accurate measurements in real-life situations.

What does this mean for the future? There have been software updates before, so this is not the first time data analysis has changed, although never before at this magnitude of impact. The bad news is that the level of this error means that previously published datasets, including large longitudinal studies and healthy cohorts, will need to be reanalyzed on updated software. The good news is that this is ultimately a correctable issue, both retrospectively and prospectively. The other good news appears to be, from early reports, that clinical trials using LCI seem to show preserved outcomes, although this needs to be confirmed for studies that only showed incremental improvements in the outcomes. In both studies, the standard deviation of corrected LCI was reduced, which may mitigate some of the impact of smaller effect size. Nonetheless, these studies will need careful reanalysis and validation, but with the assurance that recorded data will be sufficient to identify the end of the test owing to the overestimation (rather than underestimation) of the N₂ signal. In fact the correction of this error now makes nitrogen washout a much more attractive option, including in infants (although work remains to be done to characterize the impact of breathing 100% O₂). The third piece of good news is that correcting the nitrogen signal also reduces one of the major problems with this nitrogen washout device (and barriers to clinical use): the long washout tail that made testing so protracted. In CF, as we move to a postmodulator era, we are going to need reliable and trustworthy lung function measurements that are sensitive to early disease. The reality is that no system is perfect, they all have pros and cons, and we need to be honest about what these are. Now that Exhalyzer outputs are more closely aligned with those of SF₆ washout, it may be that different devices will produce interchangeable results. The Global Lung Function Initiative is currently collecting LCI data to produce normative ranges, and this will allow comparison across tracer gases and devices (https://www.ers-education.org/guidelines/global-lung-function-initiative/about/). The issue of the contribution of body N₂ to the expired signal at end of washout remains an open one (12) and is probably not something that can be reliably settled using indirect calculation of N₂.

In conclusion, an almost decade-long known issue in the calculation of nitrogen signal from the Exhalyzer D washout parameters has finally been addressed. Although reanalysis of a few datasets by Sandvik et al. and Wyler et al. showed unaltered outcomes in terms of separating patients with CF from controls, it is not until all existing datasets from healthy cohorts, longitudinal studies, and clinical trials are reanalyzed that we will be able to fully assess the impact of this error on previous study outcomes. Reanalysis of datasets from healthy cohorts will be central in setting up normative ranges for LCI and FRC for all ages, including in adults where there are known age-related changes. New normative ranges will also need to be generated for other important outcomes such as S_cond and S_acin, which are likely to be altered too. This, however, along with ubiquitous availability of pure oxygen, will go a long way in furthering the use of multiple-breath nitrogen washout testing in the clinic not only in the CF community but also for patients with other lung diseases that could benefit from clinic assessment tools that are more sensitive to the presence of airway disease than spirometry.

GRANTS

A.H. is supported by the National Institute of Health Research (NIHR) Manchester Biomedical Research Centre. C.D. is supported by grant U01 ES028669 from the National Institute of Environmental Health Sciences (NIEHS) at the NIH.

DISCLAIMERS

The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.

DISCLOSURES

A.H. has grants from NIHR, CF Trust, and CF Foundation. A.H. has worked on multiple-breath washout using the Innocor system and has collaborated with Innovision ApS, the manufacturer of that device.

AUTHOR CONTRIBUTIONS

A.H. and C.D. drafted manuscript; A.H. and C.D. edited and revised manuscript; A.H. and C.D. approved final version of manuscript.