Do clinimetric properties of LCI change after correction of signal processing?

Abstract

Background

The recently described sensor‐crosstalk error in the multiple‐breath washout (MBW) device Exhalyzer D (Eco Medics AG) could highly influence clinimetric properties and the current interpretation of MBW results. This study reanalyzes MBW data from clinical routine in the corrected software version Spiroware® 3.3.1 and evaluates the effect on outcomes.

Methods

We included nitrogen‐MBW data from healthy children and children with cystic fibrosis (CF) from previously published trials and ongoing cohort studies. We specifically compared lung clearance index (LCI) analyzed in Spiroware 3.2.1 and 3.3.1 with regard to (i) feasibility, (ii) repeatability, and (iii) validity as outcome parameters in children with CF.

Results

(i) All previously collected measurements could be reanalyzed and resulted in unchanged feasibility in Spiroware 3.3.1. (ii) Short‐ and midterm repeatability of LCI was similar in both software versions. (iii) Clinical validity of LCI remained similar in Spiroware 3.3.1; however, this resulted in lower values. Discrimination between health and disease was comparable between both software versions. The increase in LCI over time was less pronounced with 0.16 LCI units/year (95% confidence interval [CI] 0.08; 0.24) versus 0.30 LCI units/year (95% CI 0.21; 0.38) in 3.2.1. Response to intervention in children receiving CF transmembrane conductance‐modulator therapy resulted in a comparable improvement in LCI, in both Spiroware versions.

Conclusion

Our study confirms that clinimetric properties of LCI remain unaffected after correction for the cross‐sensitivity error in Spiroware software.

1. INTRODUCTION

We recently described and characterized a substantial sensor‐crosstalk error in a commercially available and widely used multiple‐breath washout (MBW) device (Exhalyzer D, Eco Medics AG) and suggested a possible correction that is now available in an updated version of Spiroware analysis software (Spiroware 3.3.1, Eco Medics AG). ¹ There is an obvious sense of concern in the MBW community, as the potential impact of these findings on existing and ongoing studies could highly influence clinimetric properties of MBW results and their current clinical interpretation. ²

MBW has become an important diagnostic tool in cystic fibrosis (CF) for both clinical follow‐up of the patients and as an endpoint for clinical trials of new therapies. ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ The lung clearance index (LCI) is a feasible, repeatable, and sensitive marker of ventilation inhomogeneity that correlates well with structural lung disease and tracks disease progression in children with CF. ⁵ , ⁶ , ⁹ , ¹⁰ LCI is calculated as the ratio of the cumulative expired volume divided by the functional residual capacity (FRC), based on indirect calculation of nitrogen (N₂) from oxygen (O₂) and carbon dioxide (CO₂) signals. This results in inherent sensitivity to measurement errors in these signals and the error described above previously resulted in a prolongation of the washout with falsely elevated LCI values. ¹

To evaluate the potential clinical impact of these findings, this study aims to reanalyze published and collected MBW data from clinical routine by applying the corrected signal‐processing algorithm and evaluating the effect on outcomes. We assessed whether previously described clinimetric properties of LCI hold for corrected results. We specifically examined corrected LCI with regard to (i) feasibility, (ii) repeatability, and (iii) validity as outcome parameters in children with CF.

2. METHODS

2.1. Study design and population

In this observational study combining several existing trials, we included N₂‐MBW data from healthy children and children with CF from previously published trials and two ongoing cohort studies (Basel‐Bern Infant Lung Development cohort ¹¹ and Swiss Cystic Fibrosis Infant Lung Development cohort ¹² ). The Ethics Committee of the Canton of Bern, Switzerland, approved the study protocols (B2019‐01072, PB_2017‐02139, 2017‐00279, and 2017‐00088) and parents gave written consent. We included MBW data from 275 study participants that were reported in previous studies and 28 participants from an ongoing trial (Table 1). Using the corrected results of these measurements, we evaluated the effect of the correction on MBW outcomes and consequently if clinimetric properties of LCI were affected.

Table 1.

Demographic characteristics of the study populations for included data sets

	Repeatability		Validity			Response to intervention
	Short‐term (15 min.)	Midterm (24 h)	Discrimination between health and disease	Longitudinal evolution of LCIa	Correlation with functional MRI
Patients (n)	16	35	138	72	14	28
Healthy	–	12	75	–	–	–
CF	16	23	63	72	14	28
Age (years)	12.4 (6.6; 17.2)	12.5 (5.6; 18.1)	10.0 (5.6; 18.1)	9.6 (4.0; 17.3)	14.0 (6.1; 18.9)	12.4 (5.7; 16.4)
Weight (kg)	40.0 (13.1)	41.7 (15.2)	41.7 (15.2)	31.1 (12.9)	45.4 (13.9)	39.1 (10.8)
Length (cm)	145.1 (14.4)	147.0 (18.8)	147.0 (18.8)	132.7 (20.2)	154.9 (16.2)	147.6 (14.3)

Note: Data are presented as mean (SD). Age is shown as mean (minimum to maximum).

Abbreviations: CF, cystic fibrosis; LCI, lung clearance index; MRI, magnetic resonance imaging.

^a Values for the longitudinal evolution of LCI are baseline values.

2.2. MBW measurements

N₂‐MBW tests were performed using the Exhalyzer D device and Spiroware analysis software (v3.2.1, Eco Medics AG) with weight‐adjusted dead spaces and settings according to current consensus guidelines. ¹³ , ¹⁴ Testing was performed with patients sitting upright and breathing through a mouthpiece. Quality control was performed according to current guidelines and tests with at least two acceptable MBW trials were included in our analysis. ¹⁴ , ¹⁵ , ¹⁶

2.3. Correction of signal processing in MBW

In brief, the sensor‐crosstalk error in the Exhalyzer D device underestimates O₂ and CO₂ gas concentrations, and thus overestimates end‐expiratory concentrations of N₂. Elevated N₂ concentrations cause a significant overestimation of the main MBW outcomes FRC and LCI, as well as of the N₂ back diffusion from the lungs. ¹ An updated Spiroware analysis software (v3.3.1, Eco Medics AG) will correct automatically for this error. Previously reported results, however, need to be reanalyzed. In this study, we reanalyzed raw data (A‐Files) using LungSim 1.01, a custom Python script developed by our group, replicating the signal processing and outcome calculation used in Spiroware analysis software (3.2.1 and 3.3.1). ¹

2.4. Reassessment of clinimetric properties and statistical analysis

We present nonparametric summaries for skewed data and parametric summaries for normally distributed characteristics. Statistical analyses were performed using Stata 16.1 (StataCorp 2019). Figures were created using Stata 16.1 or Graph Pad Prism 8 (Prism G 2018). The following clinimetric properties of LCI were examined and compared between the Spiroware software versions 3.2.1 and 3.3.1.

2.4.1. Feasibility (success rates of measurements)

Feasibility was defined by the number of study participants with acceptable MBW measurements, defined as at least two acceptable trials according to guidelines. ¹³ , ¹⁴

2.4.2. Short‐ and midterm repeatability (within 15 min and 24 h)

As previously described, for short‐ and midterm repeatability of LCI, MBW measurements were performed in triplicates 15 min and 24 h apart, respectively, with unchanged measurement conditions in school‐aged children with CF. ¹⁷ , ¹⁸ We used mixed‐effects linear regression models to calculate the variability in LCI between measurements allowing for correlation of repeated measurements within individuals. We used mean difference, intercept, and residual SD to calculate the coefficient of variation (CV%), intraclass correlation coefficient, and coefficient of repeatability. Based on a log‐linear model, we calculated upper limit of normal (ULN) for relative differences (95% quantile of a normal distribution) indicating a threshold below which 95% of the relative differences are expected to fall.

2.4.3. Validity

(A) Discrimination between health and disease

For discrimination between health and disease, we used the ULN of LCI, calculated as mean LCI of healthy study participants + 1.96 × SD from a previously described data set. ¹

(B) Correlation with functional magnetic resonance imaging (MRI) outcomes

Functional MRI data were acquired using matrix pencil decomposition MRI as previously described. ¹⁸ Main outcomes are fractional ventilation defect percentage (VDP) and relative perfusion defect percentage (QDP) expressed as the impaired fraction relative to whole lung volume. VDP and QDP were recalculated due to updated postprocessing. ¹⁹ Correlations between VDP or QDP and LCI were assessed by Spearman's rank correlation. ²⁰

(C) Longitudinal changes

Longitudinal changes in LCI were evaluated from a subset of the previously reported cohort including three monthly clinical routine MBW measurements in children with CF aged 3–18 years attending routine care between 2014 and 2018 in our center along with matching clinical information. ⁵ We used a mixed‐effects linear regression model to assess the mean rate of change in LCI with age included as a linear term, a participant‐specific random intercept and random slope to account for between‐participant variability, different observation periods for each participant, and unequal numbers of study visits. ²¹ As previously described, ⁵ we used a baseline model and a final model adjusted for predefined clinically most relevant covariates. Next, we assessed all potentially influencing covariates on LCI course first in a univariate analysis and second in the fully adjusted model.

(D) Response to intervention

The response to intervention by modulator therapy was assessed by evaluating the change in LCI between baseline, that is, before treatment start and after at least 2 weeks of treatment in three different treatment groups: In (i) either Orkambi® (Lumacaftor/Ivacaftor) or Symdeko® (Tezacaftor/Ivacaftor)‐treated patients (double modulator therapy), in (ii) Trikafta® (Elexacaftor/Tezacaftor/Ivacaftor)‐treated patients (triple modulator therapy), and (iii) patients who received first Orkambi® or Symdeko® followed by Trikafta® (combined modulator therapy). Response to intervention was defined as within‐group changes from baseline to under treatment and analyzed by paired t tests.

3. RESULTS

3.1. Study population

Summaries for demographical characteristics and number of measurements included are presented in Table 1.

3.1.1. Feasibility (success rates of measurements)

Due to the cross‐sensitivity error, N₂ concentration was overestimated rather than underestimated, thus allowing the reanalysis of all previously collected measurements and resulting in unchanged feasibility. Following reanalysis, we found a reduction in cumulative expired volume of 19.6%, which results in a comparably shorter washout duration for the participant.

3.1.2. Short‐ and midterm repeatability

Short‐term repeatability of LCI and FRC was assessed in 16 children with CF and is summarized in Table S1. Average LCI in children with CF was lower when analyzed in Spiroware 3.3.1 (LCI 9.4 [SD 1.8] vs. 10.8 [2.2]). In general, repeatability of MBW measurements was similar when analyzed in Spiroware 3.2.1 and 3.3.1. Variability between measurements expressed as CV% increased slightly in Spiroware 3.3.1 (LCI: 4.2 vs. 3.7; FRC 3.7 vs. 3.6). Also when expressing relative differences between measurements, the ULN (95% quantile) increased marginally (9.1% vs. 8.4%).

Midterm repeatability of LCI measurements was assessed in 12 healthy children and 23 children with CF; the results are summarized in Table S1. Mean (SD) LCI in healthy children was 6.8 (0.4) when assessed in Spiroware 3.2.1 and 6.1 (0.3) in Spiroware 3.3.1; in children with CF, mean (SD) LCI was 11.6 (2.6) and 9.7 (2.2) in Spiroware 3.2.1 and 3.3.1, respectively. Repeatability indices were similar when assessed in Spiroware 3.2.1 and 3.3.1. Although variability tended to be lower in healthy children when analyzed in Spiroware 3.3.1 (CV% 3.4 vs. 4.4), this was the opposite in children with CF (CV% 9.6 [3.3.1] vs. 8.1 [3.2.1]). Similarly, when assessing relative changes between visits, the ULN (95% quantile) was lower in healthy children in Spiroware 3.3.1 compared with that in 3.2.1 but higher in children with CF in 3.3.1 (Table S1). Although we found within‐subject between‐test SD to be independent of the magnitude of LCI for both Spiroware settings (Figure S1), within‐subject within‐test SD was associated with the magnitude of LCI for both settings (Spiroware 3.2.1: R ² = 0.3 (p < 0.001); 3.3.1: R ² = 0.2 (p < 0.001).

3.1.3. Validity

(A) Discrimination between health and disease

ULN based on 75 healthy controls from a retrospective data set ¹ was lower in Spiroware 3.3.1 with LCI 7.1 compared with 8.1 in 3.2.1. Sensitivity was comparable (77.8% [3.2.1] and 76.2% [3.3.1]) and specificity identical (98.7%) between 3.2.1 and 3.3.1 with some scatter in the critical area (Figures 1 and S2) based on natural variability. In Spiroware 3.3.1, two children with CF had abnormal LCI values compared with normal values in 3.2.1, whereas one healthy child was falsely categorized as abnormal based on slightly elevated LCI. In both Spiroware versions, LCI values of all the other healthy controls were within normal range and 15 children with CF had values below the ULN.

Figure 1.

Discrimination between health and disease holds after correction for cross‐sensitivity error. Scatter plot of LCI values for healthy controls (hollow circles) and participants with CF (solid black) obtained in Spiroware 3.2.1 and 3.3.1. The dashed lines represent upper limits of normality (95% quantile). Abbreviations: CF, cystic fibrosis; HC, healthy controls; LCI, lung clearance index; TO, turnover; ULN, upper limit of normal (95% quantile)

(B) Correlation with functional MRI outcomes

We reanalyzed the correlation of functional MRI and LCI in 14 children with CF. The extent of QDP ranged between 15% and 35% (Figure 2) The correlation between QDP and LCI remained consistently strong in both Spiroware versions (3.2.1: r _S = 0.66, p = 0.03; 3.3.1: r _S = 0.69 p = 0.02). The increase in LCI per increase in ventilation defect was comparable for both Spiroware versions (Spiroware 3.2.1: 1.28 LCI units/percent perfusion defect [95% confidence interval {CI} 0.67; 1.89]; Spiroware 3.3.1: 1.32 LCI units/percent perfusion defect [95% CI 0.55; 2.10]; Figure S3).

Figure 2.

Association of perfusion defect from functional MRI and LCI remains similar after correction for cross‐sensitivity error. Shown are the associations between the perfusion defect [%] from functional MRI and the LCI [TO] in Spiroware 3.2.1 (A) and Spiroware 3.3.1 (B) for n = 14 CF patients. Abbreviations: CF, cystic fibrosis; LCI, lung clearance index; TO, turnover

(C) Longitudinal tracking

To assess the differences in the longitudinal course of LCI between Spiroware 3.2.1 and 3.3.1, we reanalyzed 796 measurements from 72 children with CF (Table 1). Without adjustment for risk factors, LCI increase was comparable in both settings; however, less pronounced in Spiroware 3.3.1 (0.16 LCI units/year [95% CI 0.08; 0.24] vs. 0.30 LCI units/year [95% CI 0.21; 0.38] in Spiroware 3.2.1). The pattern of increase in LCI was similar in both settings, remaining stable during preschool years and school age, and then starting to increase in adolescence (Figure 3 and Table S2). Similar to our previous findings, Aspergillus and Pseudomonas aeruginosa colonization, severe exacerbations, and experiencing allergic bronchopulmonary aspergillosis during the study period remained individually associated with a steeper increase in LCI also in Spiroware 3.3.1, even though with a smaller magnitude (Table S3). The effect on covariates associated with acute changes in LCI (acute exacerbations, CF‐related diabetes, body mass index [BMI] z score) remained similar for both Spiroware settings (Table S4). With adjustment for previously defined risk factors (sex, BMI, pseudomonas aeruginosa and Aspergillus colonization, CF‐related diabetes, and acute and severe exacerbations), the pattern of increase in LCI was similar for both Spiroware settings, again less pronounced for 3.3.1 (0.08 LCI units/year [95% CI 0.01; 0.14; Spiroware 3.3.1] versus 0.19 LCI units/year [95% CI 0.12; 0.27; Spiroware 3.2.1]; Figure S4).

Figure 3.

LCI increases over time without adjustments in Spiroware 3.2.1 and 3.3.1. In Spiroware 3.2.1 (A), the increase in LCI over age was 0.30 LCI units/year (95% CI 0.21; 0.38). Increase in preschool age was −0.26 LCI units/year (95% CI −0.95; 0.44), in school age 0.22 LCI units/year (95% CI 0.08; 0.35), in adolescence 0.40 LCI units/year (95% CI 0.26; 0.54; p  for interaction 0.02), with a steeper increase in females during adolescence. In Spiroware 3.3.1 (B), the increase in LCI over age was 0.16 LCI units/year (95% CI 0.08; 0.24). Increase in preschool age was −0.27 LCI units/year (95% CI −0.87; 0.33), in school age 0.11 LCI units/year (95% CI −0.01; 0.22), in adolescence 0.30 LCI units/year (95% CI 0.18; 0.43; p for interaction 0.02), also with a steeper increase in females during adolescence. On the y axis, LCI raw values are given. The solid line represents mean LCI values across all participants with available data at a given age. Shaded areas represent point‐wise upper and lower 95% CIs, the dotted line refers to a ULN for LCI of 8 TO. Abbreviations: CI, confidence interval; LCI, lung clearance index; TO, turnover; ULN, upper limit of normal

(D) Response to intervention

To characterize differences in the response to intervention with double or triple modulator therapy between Spiroware 3.2.1 and 3.3.1, we reanalyzed 212 visits from 28 patients (Table 1) and compared mean LCI values at baseline, under double, under triple, and combined therapy. There was a statistically significant improvement (reduction) in LCI in all three treatment groups when compared with baseline in both Spiroware algorithms (Figure 4). In Spiroware 3.2.1, within‐group mean (95% CI) absolute change from baseline was −1.7 LCI units (−2.8 to −0.5, p = 0.012) under double therapy, −1.7 LCI units (−2.5 to −0.9; p ≤ 0.001) under triple therapy, and −2.5 LCI units (−4.1 to −1.0; p = 0.007) under combined modulator therapy (Figure 4 and Table S5). In Spiroware 3.3.1, LCI was substantially lower but the change from baseline remained statistically significant in all groups (within‐group mean [95% CI] absolute change from baseline was −1.5 LCI units [−2.5 to −0.4, p = 0.013] under double therapy, −1.3 LCI units [−1.9 to −0.7; p ≤ 0.001] under triple therapy, and −1.7 LCI units [−2.9 to −0.5; p = 0.014] under combined modulator therapy). Overall, within‐group mean values differed substantially between Spiroware 3.2.1 and 3.3.1 (mean [95% CI] difference between software versions over all groups was −1.5 [−2.0 to −1.1] LCI units).

Figure 4.

Responsiveness to intervention with modulator therapy holds after correction for cross‐sensitivity error. Data are presented as mean (95% CI) and before–after plots. Shown are mean (95%) LCI values from 212 visits of 28 patients at baseline, under double (Orkambi® [n = 8]), under triple (Trikafta® [n = 13]), and under combined therapy (first double followed by triple) modulator therapy (Orkambi® [n = 2] or Symdeko® [n = 5] followed by Trikafta®). Statistical analysis: paired t test (baseline vs. endpoint). Abbreviations: LCI, lung clearance index; TO, turnover; 95% CI, 95% confidence interval

4. DISCUSSION

4.1. Summary

In this study, we can confirm that clinimetric properties of LCI are still valid after correction for the recently described measurement error in the widely used MBW Spiroware software. ¹ As expected, the correction led to lower LCI values and thus lower thresholds, implicating that we need to redefine existing thresholds and accordingly adjust clinical interpretation of LCI and its changes.

4.2. Comparison to literature

4.2.1. Clinical impact of measurement error

The initial concerns that the findings of Wyler et al. ¹ about the measurement error in the widely used MBW software Spiroware 3.2.1 could tremendously change utility of LCI in clinical routine, can be rejected by our findings. We could show, that clinimetric properties of LCI still hold, even though with a slightly smaller magnitude based on overall lower LCI values. This is due to the measurement principle of the test, the indirect N₂‐calculation was found to be overestimated by the cross‐sensitivity error in the O₂ and CO₂− signals, leading to a prolongation of the washout with falsely high LCI values. Our results suggest that clinical interpretation of LCI change is not affected by the correction of this error. As such, we can support recently published results from clinical trials and extent the interpretation into the clinical application. ²

4.2.2. Impact on feasibility (success rates of measurements)

As the correction of the gas measurement error results in shorter washouts, we were able to reanalyze all our trials with the corrected software version. ²² Thus, also in clinical practice, overall N₂‐MBW will become shorter with the corrected software, which might increase the feasibility of the test, especially in the younger and possibly less cooperative age range, but also in patients with more advanced disease and longer washout times. ¹³ , ²²

4.2.3. Impact on repeatability

Overall, we found short‐ and midterm repeatability ¹⁷ , ¹⁸ to be similar after cross‐sensitivity correction, and thus limits for intervisit changes and variability to be unaffected. Within‐subject between‐test SD remained independent of the magnitude of LCI, which preserves the property of LCI to guide individual clinical decisions.

4.2.4. Validity

The discriminatory ability of LCI holds after correction with a linear trend to lower LCI values for healthy but also children with CF. Thereby, the higher the LCI value, the higher the downward correction, which needs closer reevaluation of patients with highly elevated LCI. Potentially, the overestimation of back‐diffusion of N₂ might have overrated LCI values of patients with more severe disease and longer washouts. However, correlation with functional defects detected by MRI scans ¹⁷ , ¹⁸ , ²⁰ remained very similar before and after measurement correction, reassuring that the pathophysiological understanding of LCI is still valid. Correspondingly, the pattern of increase in LCI with age remained similar with a smaller magnitude after measurement correction (Figure 3 and S4). A very similar effect was seen by Robinson et al. ² reanalyzing change in LCI in preschool CF children over 12 months. In line with Robinson et al., ² treatment effects of CF transmembrane conductance modulators remained statistically significant, whereas the magnitude of the change was smaller.

4.3. Strengths and weaknesses of this study

We performed a thorough reanalysis of clinical and research MBW data using the corrected algorithm incorporated in Spiroware 3.3.1, with data originally collected or reloaded in Spiroware 3.2.1. ²³ We performed rigorous quality control with only perfect data being used, ¹⁵ which minimizes bias due to sampling variation. Further, we were able to use a wide range of different data sets obtained in a clinical routine setting or from clinical studies to assess the impact of measurement correction on various clinimetric properties of LCI. By nature, the main limitation lies in the retrospective application of the correction. Besides, we had two different data sets from healthy controls available, one consisting of 12 subjects providing repeated LCI measurements 24 h apart to assess repeatability and the other consisting of 75 subjects providing 1 LCI measurement to calculate normative values. The inherently wider spread of the second data set along with the different statistical approach in the first data set accounting for repeated measures led to a small difference in the ULN for LCI (first data set 6.7 vs. 7.1 in the second data set, Spiroware 3.3.1).

4.4. Clinical relevance and outlook

Although there is a significant change with the corrected algorithm, we show that the clinimetric properties of LCI still hold. This means that clinical utility and our understanding of LCI and its changes remain valid. This also suggests that previously published papers do not need to be withdrawn. However, this implies also that for absolute correct estimation of effect sizes, previously published data sets need to be reanalyzed to enable comparison with results from Spiroware 3.3.1. Besides, we will need to closely evaluate changes within individuals applying the new software, to adjust individual therapeutic goals but also to capture individual disease progression at this new level. The option of retrospective correction has the great advantage that all previously collected data in Spiroware can be reanalyzed and corrected.

5. CONCLUSION

Our study confirms that clinimetric properties of LCI remain unaffected after correction for the recently detected measurement error in the widely used Spiroware software. However, the correction results in lower LCI values, with the concurrent need to redefine existing thresholds and adjust clinical understanding of LCI and its changes on this new level.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Bettina S. Frauchiger: Conceptualization, methodology, formal analysis, investigation, resources, writing—original draft, review and editing, visualization. Marc‐Alexander Oestreich: Conceptualization, data curation, methodology, formal analysis, investigation, resources, writing—original draft, review and editing, visualization. Florian Wyler: Software. Nathalie Monney: Resources. Corin Willers: Resources. Sophie Yammine: Writing—original draft, review and editing. Philipp Latzin: Conceptualization, supervision, project administration, funding acquisition.

Supporting information

Supporting information.

Supporting information.

Supporting information.

Supporting information.

Supporting information.

Supporting information.

Frauchiger BS, Oestreich M‐A, Wyler F, et al. Do clinimetric properties of LCI change after correction of signal processing? Pediatric Pulmonology. 2022;57:1180‐1187. 10.1002/ppul.25865

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.