Measurement of Fractional Exhaled Nitric Oxide: Comparison of Three Different Analysers

Abstract

Background

Fractional exhaled nitric oxide (FeNO) is a surrogate marker for airway inflammation, supporting the diagnostic pathway and treatment decisions for asthma patients.

Objectives

Aim of this study was to compare the new analyser Vivatmo pro (Bosch, BV) with NIOX VERO (Circassia, CN) and CLD (Ecomedics, EC).

Methods

In 100 asthmatics (median 53 years [range 20–87], 62% female, 86% on inhaled corticosteroids [mean 1,300 μg beclomethasone dipropionate or equivalent], 35% treated with biologics) 2 FeNO measurements per device were performed. Additionally, the success rate to achieve a valid NO value was evaluated.

Results

Sixty-eight percent of the patients had FeNO values below 50 ppb. Median NO concentrations were 31 ppb (range 6–194) for BV, 33 ppb (9–164) for CN and 31ppb (7–353) for EC. Bland-Altman plots suggested an agreement within the predefined limits of ±5 ppb for all analysers within the therapeutically relevant range (0–70 ppb). The highest agreement in FeNO levels were between BV and EC with mean differences of −0.26 (95% CI −1.48 to 0.95) vs. 1.52 (95% CI 0.4–2.6) ppb for CN and EC. The results indicate an equivalence of the methods (two-one sided t test-equivalence test: p < 0.0001, ±5 ppb margins). Acceptance of the measurements was high for all devices (97%). The highest success rate to obtain 2 valid NO values without failed attempts was achieved with the BV analyser (73 vs. 62% for the CN analyser and 46% for the EC analyser).

Conclusions

For the range between 0 and 70 ppb, FeNO concentrations measured with all 3 devices were statistically equivalent within predefined acceptance criteria and did not differ in a clinically relevant way.

Introduction

Measurement of fractional exhaled nitric oxide (FeNO) is a quantitative, noninvasive, simple and safe method to assess Type 2-driven airway inflammation in respiratory diseases [1]. Type 2-driven airway inflammation is a hallmark of bronchial asthma, and several lines of evidence support the relationship between exhaled NO, IL-13 activity and eosinophilic inflammation in asthma [2, 3]. NO concentrations in the exhaled breath correlate well with Type 2 airway inflammation and is therefore recommended in international guidelines as an additional parameter supporting treatment decisions in symptomatic patients [4, 5, 6, 7, 8, 9, 10] In this regard, measurement of exhaled NO is useful to evaluate potential corticosteroid responsiveness, or conversely, helping to identify poor corticosteroid compliance [11, 12]. Elevated levels of NO in asthmatic patients have been shown to correlate with disease activity, and more importantly, with subsequent deterioration in asthma control [13, 14]. FeNO even predicts the response to inhaled corticosteroids in patients with nonspecific respiratory symptoms [15]. The FeNO measurement is easy to perform, reproducible, associated with a high degree of acceptance by patients and thus a useful marker for asthma patients in primary care [16]. Furthermore, in the current NICE clinical guideline on asthma, FeNO is recommended for the management of patients who remain symptomatic on inhaled corticosteroids [7]. The introduction of FeNO testing in primary care settings can be achieved with a very low effort with respect to measurement procedures and data interpretation, while simultaneously improving the quality of care [7, 17].

FeNO can be measured using several commercially available analysers which differ in some aspects, for example, methods of measurements, complexity or set-up [18]. Stationary analysers usually measure FeNO by chemiluminescent techniques, whilst handheld devices measure FeNO using electrochemistry. Independent of the measurement technology and in order to assure reliable measurements the analysers follow the standardized measurement procedures recommended by ATS (American Thoracic Society) and ERS (European Respiratory Society) [19].

As yet there is limited data on whether the results of the different analyzers are comparable. For clinical practice it is important to know whether the instruments used to measure exhaled NO can be used interchangeably [20].

Consequently, the aim of this study was to compare the newly available handheld FeNO analyser Vivatmo pro (BV; Bosch Healthcare Solutions, Waiblingen, Germany) in clinical routine with 2 reference devices: another handheld FeNO analyser NIOX VERO® (CN; Circassia Pharmaceuticals plc, Oxford, United Kingdom) and the stationary Ecomedics analyser CLD 88 (EC; Eco Medics AG, Duernten, Switzerland).

Methods

Subjects

From November 28, 2017 to February 02, 2018 a total of 106 asthma patients were invited to participate in the study (Table 1), all of whom had to be non-smokers for at least 1 year. Patients with a respiratory tract infection were excluded. The study was performed according to Good Clinical Practice standards and the Declaration of Helsinki, and was approved by the local Ethics Committee. All participants gave a written informed consent.

Table 1.

Demographic-and asthma-relevant parameters of included patients

Parameter	n	%	Mean (SD)	Median (range)
Women	62	62.0
Age, years	100		51.3 (14.0)	53 (20–87)
Height, cm			170.8 (8.9)	170.5 (154–193)
Asthma diagnosis, years			21.9 (14.0)	20 (0–62)
Weight, kg			80.2 (18.6)	78.5 (43–147)
BMI, kg/m2			27.4 (6.0)	26.1 (16.8–53.3)
FEV1, L			2.2 (0.9)	2.2 (0.7–5.5)
FEV1, %			71.4 (23.1)	75.0 (23–131)
FVC, L			3.2 (1.0)	3.1 (1.2–6.5)
FVC, %			86.7 (20.7)	86.5 (39–139)
ACT			16 (5.7)	15 (5–25)
Allergy sufferers	68	68.0
Former smokers	45	45.0
Pack-years			8.4 (9.5)	6.5 (1–60)
Medication
ICS dose, µg BDP/day	86	86.0	1,314 (842)	1,000 (200–4,000)
LABA	75	75.0
LAMA	41	41.0
Biologicals*	35	35.0

^*Omalizumab n = 8, Mepolizumab n = 25, Reslizumab n = 2.

BDP, beclomethasone dipropionate equivalent; BMI, body mass index; ICS, inhaled corticosteroids; ACT, Asthma Control Test; FVC, forced vital capacity.

Nitric Oxide Analysers

According to the manufacturers, both handheld devices Vivatmo pro (BV) and NIOX VERO (CN) were precalibrated during the production and are service- and calibration-free systems. The BV device is further maintenance-free and does not require any exchange of the internal Chemical Field Effect (ChemFET) sensor. The CLD 88 (EC) was calibrated using calibration gas containing 3 parts per million (ppm) of NO in N₂ monthly (NO calibration), and each day the flow measurement (flow calibration) and NO zero was calibrated as recommended by the manufacturer.

Exhaled Nitric Oxide Measurements

Exhaled NO measurements were performed according to the consensus guideline of the ERS and the ATS [18]. As exhaled NO values are highly flow dependent, all measurements were performed at a standardized exhalation flow rate of 50 mL/s. Patients had to exhale against a positive counter pressure of 10 cm H₂O to avoid cross-contamination with nasal NO. In order to guide the subjects in performing a valid exhalation manoeuvre, the flow parameters were controlled by visible and/or audible feedback.

Patients performed 2 consecutive measurements on each analyser with a random pre-defined measuring sequence concerning device order. In line with current guidelines 2 min relaxed tidal breathing between measurements were observed to avoid the potential confounding effect of serial inhalation/exhalation manoeuvres, which may lead to a decline in measured exhaled NO values [15, 18]. Exhaled NO levels were automatically calculated by the analysers and represented the mean of 2 technically adequate and reproducible measurements [14]. According to the manufacturers' instrument manuals the measurement ranges of the devices were 5–300 ppb for the BV device and the CN device, and 0.1–5,000 ppb for the EC device.

All measurements were completed within 1 h on the same day. After the measurements patients evaluated the different devices. They were asked to rate the analysers overall according to comfort and ease of performing the measurement. After measurement of exhaled NO, spirometry was performed in accordance with the ATS guidelines [19].

Ambient NO values were recorded for each patient prior to exhaled NO measurements using the value automatically obtained by the CLD 88 analyser.

Statistical Analyses

Descriptive data included means (±SD) or medians and quartiles for continuous end-points (e.g., NO values), and frequencies for categorical end-points (NO values </> = 40 ppb, patient characteristics and patient evaluation of the analysers).

The primary objective of the study was to estimate the bias and to show equivalence between the analysers BV, EC and CN. According CLSI-EP09-A3 guidance difference plots as well as a specific regression method (Passing-Bablok regression) were applied to estimate the difference (primary endpoint) and the slope/intercept, respectively [21, 22, 23]. The analyses focussed on the main therapeutical relevant level up to 70 ppb, and on the area in which the cut-points for a positive test or the presence of significant Type-2 inflammation are located [6]. Using the equivalence testing framework equivalence is demonstrated when the CI (here 90%) of the difference between the methods is within predefined limits (±5 ppb in the range 0–70 ppb, ±10% in the range >70 ppb). The corresponding p values were obtained with the two-one sided t test (TOST) [24]. The limits of agreement were calculated as the (95%, 95%)-tolerance interval of the mean of the differences [25]. Using the currently proposed clinical decision limit, binary measures (</≥ 40 ppb) were introduced and agreement of the methods was evaluated (positive and negative percent agreement, overall agreement) [7]. Repeatability was calculated as intraserial precision estimated by random effects ANOVA following the CLSI EP05-A3 guidance [26].

Further endpoints were evaluation of success rates and number of attempts needed by each of the devices.

All patients with valid measurement results of all 3 methods were included in the analysis.

A sample size calculation applying TOST within difference plot used the following assumptions: alpha level of 5%, power of 80%, a real difference of 3 ppb (for NO-values >50 ppb 2%) with a SD of 5 ppb (15%), and a proportion of patients with missing results of 5%. It resulted in 106 cases to be included.

Data analysis was performed using software SAS 9.4 (SAS Institute, Cary, NC, USA).

Results

In the study valid results for 100 patients from all 3 devices were obtained. Six patients did not accomplish 2 valid measurement values from all 3 devices, due to a FeNO level above the measurement range of the handheld devices (n= 1), technical problems (n = 2) or because they were not able to perform measurements with all the devices (n = 3).

In 68% of patients FeNO values were below 50 ppb, 89% had FeNO values up to 70 ppb as measured by EC. Median NO concentrations were 31 ppb (range 6–194) for BV, 33 ppb (9–147) for CN and 31 ppb (7–167) for EC.

Bland-Altman plots showed a high agreement (and equivalence within the statistical equivalence testing framework) in absolute FeNO levels for individual patients as well as an agreement within the predefined maximum bias of ±5 ppb obtained with all analysers within the therapeutically relevant range of 0–70 ppb between BV vs. EC and BV vs. CN (Fig. 1).

Fig. 1.

Bland-Altman difference plot for absolute differences (range 0–70 ppb) of the mean of 2 valid FeNO measurements using the BV vs. EC (a) and the BV vs. CN (b).

Agreement in absolute FeNO levels for individual patients was evaluated for pairwise analysers' comparisons, with mean differences of −0.26 ppb (90% CI −1.48 to 0.95; TOST p value <0.0001) between BV and EC analysers, −1.78 (90% CI −3.11 to −0.45; TOST p value <0.0001) for BV vs. CN and 1.52 (90% CI 0.43–2.60; TOST p value <0.0001) between CN and EC. The results suggested that FeNO values obtained with the CN were generally slightly higher than the FeNO measurements obtained with BV and EC.

For patients with a higher intensity of airway inflammation and for those who needed >3 attempts per device, the variation of the FeNO measurements both within a single device and between different analysers was higher. Thus, for patients with FeNO values above 70 ppb the mean inter-device bias was 8.6% (90% CI −3.1 to 20.2%; TOST p value 0.4159) for BV vs. EC, 9.6% (90% CI −2.9 to 22.1%; TOST p value 0.4782) for BV vs. CN, and −1.1% (90% CI −6.3 to 4.2%; TOST p value 0.0047) for CN vs. EC (Fig. 1).

Difference plot analysis for the differences between 2 devices (range <70 ppb) showed a mean inter-device bias of 0.21 ppb (90% CI −1.16 to 1.58; TOST p value <0.0001) between BV and EC compared to 1.15 ppb (90% CI −0.14 to 2.45; TOST p value <0.0001) for CN and EC in the case that only the first measurements were considered.

The calculated Passing-Bablok-regression coefficients indicate a good agreement of FeNO levels between BV vs. EC and BV vs. CN (Fig. 2), with slopes of 1.054 (95% CI 0.96–1.15) and 1.045 (95% CI 0.96–1.15), and intercepts of −1.12 (95% CI −3.09 to 0.66) and −3.09 (95% CI −5.93 to −1.23) respectively. Results indicate a statistically significant constant offset for BV vs. CN (as the CI for intercept does not include zero); however, as the visual inspection of the scatterplot indicates, this might be primarily related to observed differences between analysers in the >70 ppb range.

Fig. 2.

Device agreement: Passing-Bablok regression with scatter plots of the mean of 2 valid FeNO measurements using the BV vs. EC (a) and the BV vs. CN (b; Dashed line: Identity, Solid line: Passing-Bablok regression line). FeNO, fractional exhaled nitric oxide.

Repeatability coefficients within the therapeutically relevant range (ICCs, random effects ANOVA) were 0.944 (SD 4.2 ppb) for BV and 0.946 (SD 4.1 ppb) for CN. Thus, both handheld devices showed a comparable high repeatability.

Based on the NICE classification, a FeNO measurement of ≥40 ppb or higher is rated as a positive FeNO testing [14]. The overall percentage agreement (OPA) between the different analyzers within this predefined clinical cutoff was high: 96% of patients with the BV and 94% with the CN were classified equally as FeNO positive or negative compared to the measurement with the EC (Tables 2, 3, 4). Positive and negative percentage agreement (PPA/NPA) were 98.4/92.1% for BV vs. EC and 93.5/94.7% for CN vs. EC respectively. For BV compared to CN the result was 92% in terms of OPA and 96.7/85.0% in terms of PPA/NPA.

Table 2.

Agreement of FeNO results with the tested devices based on NICE criteria for a positive FeNO test (≥40 ppb assumed as a positive test result); overall agreement for BV vs. EC

FeNO	EC		Total
	<40 ppb	≥40 ppb
BV
<40 ppb	61	3	64
≥40 ppb	1	35	36
Total	62	38	100

Overall agreement for BV vs. EC: 96/100 = 96% (95% CI 90.1–98.9%). Positive percent agreement: 35/38 = 92.1% (95% CI 78.6–98.3%). Negative percent agreement: 61/62 = 98.4% (95% CI 91.3–100.0%).

FeNO, fractional exhaled nitric oxide.

Table 3.

Agreement of FeNO results with the tested devices based on NICE criteria for a positive FeNO test (≥40 ppb assumed as a positive test result); overall agreement for BV vs. CN

FeNO	CN		Total
	<40 ppb	≥40 ppb
BV
<40 ppb	58	6	64
≥40 ppb	2	34	36
Total	60	40	100

Overall agreement for BV vs. CN: 92/100 = 92% (95% CI 84.8–96.5%). Positive percent agreement: 34/40 = 85.0% (95% CI 70.2–94.3%). Negative percent agreement: 58/60 = 96.7% (95% CI 88.5–99.6%).

FeNO, fractional exhaled nitric oxide.

Table 4.

Agreement of FeNO results with the tested devices based on NICE criteria for a positive FeNO test (≥40 ppb assumed as a positive test result); overall agreement for CN vs. EC

FeNO	EC		Total
	<40 ppb	≥40 ppb
CN
<40 ppb	58	2	60
≥40 ppb	4	36	40
Total	62	38	100

Overall agreement for CN vs. EC: 94/100 = 94% (95% CI 87.4–97.8%). Positive percent agreement: 36/38 = 94.7% (95% CI 82.3–99.4%). Negative percent agreement: 58/62 = 93.5% (95% CI 84.3–98.2%).

FeNO, fractional exhaled nitric oxide.

According to the ATS classification, FeNO levels above 50 ppb likely indicate significant airway inflammation [6]. Considering this cutoff for clinically relevant inflammation, the OPA of BV and CN compared to EC were 92% (PPA 81.3%; NPA 97.1%) and 95% (PPA 84.4%; NPA 100%), respectively, and 93% (PPA 88.9%; NPA 94.5%) for BV vs. CN.

The ambient NO levels recorded in this study ranged between 4 and 139 ppb (mean 47 ± 27 ppb).

To evaluate a potential impact of ambient NO on the measurements with the handheld devices, the interfering ambient NO was correlated with the absolute differences of the FeNO readings of the handheld devices (BV and CN) toward the EC measurements. Ambient NO levels showed a significant linear correlation with higher FeNO readings for both handheld devices (BV and CN) compared to the EC measurements (p < 0.05); however, the correlation was only moderate (Pearson r < 0.5). The mean of absolute differences for both handheld devices was within the specific accuracy of ±5 ppb. Accordingly, no clinically relevant effect of ambient NO on FeNO was detected.

In this study set-up, no other asthma-related co-factor such as years since diagnosis, asthma control, asthma severity, FEV1, medication, allergic status, or BMI had a significant influence on FeNO differences between the analyzers.

In general, the acceptance of FeNO measurements was high among the patients independently of the device. Among all devices, 97% of all patients rated the FeNO measurement as pleasant to very pleasant compared to spirometry and approximately 90% of all patients valued the measurement as easy to very easy. Seventy percent of the patients already had experience with FeNO measurements and 65% already knew the CN analyzer.

The highest success rate to achieve 2 valid NO values without failed attempts was achieved with the BV analyser for 73% of patients vs. 62% for the CN analyser and 46% for the EC analyser. However, within up to 3 attempts in over 80% of the participants, 2 successful measured values were achieved for all devices (BV = 86%, CN = 85%, EC = 81%).

Discussion

The present study evaluated the equivalence to measure FeNO in asthma patients between the recently introduced BV and CN as another common handheld device and the stationary device EC as the reference method. It confirmed equivalence of the BV device with the other devices within the predefined limit (±5 ppb) in the range of 0–70 ppb.

Previous comparisons between different devices for measuring exhaled NO gave inconsistent results [27, 28, 29, 30]. The Bland-Altman-plots of the present study indicate an acceptable degree of agreement between the 3 analysers for patients with a FeNO in the range of 0–70 ppb. However, FeNO values obtained with the CN were slightly higher than with the EC or BV. In addition, in patients with higher FeNO values (>70 ppb) the variation of FeNO between all analyser was increased and the BV device seems to result in higher FeNO readings than both the EC and the CN devices. However, in the area above 70 ppb, only a very limited number of measurement results have been determined.

The clinical implication of these findings is that in patients with FeNO values <70 ppb, the exhaled NO concentrations measured with all 3 devices were statistically equivalent within predefined acceptance criteria and, thus, are compatible. This information is of relevance, since quite often clinical decisions have to be based on FeNO values obtained with different devices. This is true even though in patients with high FeNO concentrations (>70 ppb) where the spread increases with repeated measurements and with the use of different analyzers. Consequently, measurements of individual patients with high values should be assessed with caution. On the other hand, the clinical significance of very high FeNO values and differences between high values is limited. In the case of FeNO concentrations above 50 ppb, only a change above 20% is considered a clinically relevant trend [6]. Data on the course of therapy for this patient population with FeNO values above 70 ppb are hardly available, and therefore long-term studies with a higher FeNO measurement frequency (daily to weekly) would be useful to better evaluate the clinical relevance of changes in in this high range.

The results from the present study confirm those from earlier smaller trials. First, there was a good correlation between results obtained with the CN and EC devices [28, 29, 30], even though there was a consistent bias toward slightly higher readings in the order of 1–2 ppb when the EC as compared with the CN [29, 30]. Alving et al. [29] suggested that between-analyser differences were due to the use of different calibration gases and/or measurement procedures. In the present study, the calibration of the EC was performed according to the manufacturer's instructions and the CN and BV were precalibrated by the manufacturer.

Previous studies addressed the impact of ambient NO on FeNO measurements for chemiluminescent- and electrochemistry-based measurement techniques. Some of these studies reported that exhaled NO was influenced by ambient NO [31, 32], whereas others reported no effect of ambient NO on FeNO values [33, 34]. Elevated ambient NO concentrations lead to an increased NO peak during the initial phase of the breath maneuver; however, the height of the resulting NO plateau is unaffected of the ambient NO concentration [33, 35]. Some of the commercially available NO analysers, use a 2-step breath manoeuvre to reduce this effect, which consists of an inhalation step with NO-free air prior to the exhalation step. Alternatively, an interference with ambient NO can also be avoided by a one-step breath manoeuvre based on an optimized dead space in the measuring system to allow an effective bypass of the increased NO peak. This is the underlying approach of the gas flow technology within the BV analyser in combination with the corresponding mouthpiece.

The results show that even high levels of ambient NO in the environment have no clinical relevant interference on the FeNO measurements in the devices ­tested.

Additionally, in this study set-up, no other factors introducing measurement differences between the devices were detected. The majority of patients had already used different devices for FeNO measurement in the past and 65% of the patients previously used CN. However, the highest success rate was achieved with the newly launched BV. This indicates its easy and intuitive usage for patients with asthma. Because of its small size and portability, the BV can be expected to be used more generally and effectively, both in clinical practice and in research studies.

The ATS/ERS guidelines recommend a dual measurement of FeNO to obtain a valid value. The results of the present study indicate that a single measurement using the BV also led to valid FeNO values. This is in line with findings by Horváth et al. [36] suggesting the use of handheld devices in the case that just one measurement can be performed due to financial or other constraints.

Many easy-to-use NO analysers are now available. This may allow information regarding the underlying airway inflammation to be increasingly used in asthma management, particularly in the primary care setting. Further studies will have to answer the important question whether all the instruments used to measure exhaled NO accurately reflect disease activity, and when measurements change, whether this is because a different device has been used.

Potential limitations of this study include the necessity to perform serial inhalation/exhalation manoeuvres, which may lead to a decline in measured exhaled NO values [15]. Consequently, and in line with current guidelines, 2 min relaxed tidal breathing between measurements avoided the potential confounding effect [18]. This is true even though in patients with high FeNO concentrations (>70 ppb), the spread of repeated measurements and within the different analyzers increased. The number of patients with high FeNO concentrations was too small to evaluate the equivalence of the different devices. A further limitation is that this study only involved adult patients and no children.

In summary, the present study confirms that absolute exhaled NO measurements do not vary in a clinically relevant way between the BV, EC, and CN devices in patients with FeNO values in the range of 0–70 ppb. Consequently, the BV may be as effective as stationary chemiluminescence NO analyzers when used in clinical practice.

Statement of Ethics

This study was approved by the Research Ethical Committee of Universitätsmedizin Mainz. This study was supported by Bosch Healthcare Solutions GmbH.

Disclosure Statement

S.K. has received a grant by Bosch Healthcare Solutions GmbH for the participation at a pulmonary congress. M.W., S.V., and R.B. have no conflicts of interest.

Author Contributions

All authors contributed to the conception and design of the study, drafted the article, revised it critically, and finally approved of this submitted version.