Within-session variability as quality control for oscillometry in health and disease

Abstract

Oscillometry is increasingly adopted in respiratory clinics, but many recommendations regarding measurement settings and quality control remain subjective. The aim of this study was to investigate the optimal number of measurements and acceptable within-session coefficient of variation (CoV) in health, asthma and COPD.

15 healthy, 15 asthma and 15 COPD adult participants were recruited. Eight consecutive 30-s measurements were made using an oscillometry device, from which resistance at 5 Hz (R_rs5) was examined. The effect of progressively including a greater number of measurements on R_rs5 and its within-session CoV was investigated. Data were analysed using one-way repeated-measures ANOVA with Bonferroni post hoc test.

The CoV(R_rs5) of the first three measurements was 6.7±4.7%, 9.7±5.7% and 12.6±11.2% in healthy, asthma and COPD participants, respectively. Both mean R_rs5 and CoV(R_rs5) were not statistically different when progressively including four to eight measurements. Selecting the three closest R_rs5 values over an increasing number of measurements progressively decreased the CoV(R_rs5). In order for ≥95% of participants to fall within a target CoV(R_rs5) of 10%, four or more, five and six measurements were needed in health, asthma and COPD, respectively.

Within-session variability of oscillometry is increased in disease. Furthermore, the higher number of measurements required to achieve a set target for asthma and COPD patients may not be practical in a clinical setting. Provided technical acceptability of measurements is established, i.e. by removing artefacts and outliers, then a CoV of 10% is a marker of quality in most patients, but we suggest higher CoVs up to 15–20% should still be reportable.

Short abstract

Within-session variability of oscillometry indices is intrinsically higher in disease. Quality control should first focus on technical acceptability of measurements, i.e. by removing artefacts and outliers over reducing variability, and CoVs up to 15−20% should still be reportable. https://bit.ly/3w7qbIR

Introduction

The forced oscillation technique, also known as oscillometry, is a method of measuring respiratory system impedance that is noninvasive, noneffort dependent, simple to administer and reproducible. Oscillometry provides detailed respiratory mechanics and sensitivity measures, especially of the small airways. Its utility in a research setting is well established [1, 2], and its clinical use is increasingly recognised [3, 4]. The technique involves superimposing pressure oscillations at the mouth onto resting tidal breathing. Respiratory impedance is then calculated as the ratio between pressure and flow, and can be broken down to resistance (R_rs), which is a measure of airway calibre, and reactance (X_rs), which is a measure of the elastic properties of the respiratory system. Both measurements are sensitive to heterogeneous airway narrowing and closure, which typically occurs in airway diseases.

As oscillometry matures as an emerging clinical test, there is a greater need to standardise testing protocols. Expert recommendations on the nature of the testing sessions have been made [1, 5, 6]; current standards [4] recommend acquiring at least three replicates within a single testing session that are deemed acceptable quality control criteria (visual inspection, within-session coefficient of variation (CoV) and automated signal processing). Minimising the within-session CoV is desirable, because this will improve the between-session reproducibility of the test.

A target cut-off CoV of ≤10% is recommended for adults and 15% for children [4], but there is limited empirical evidence supporting these cut-offs. Watts et al. [7] and Robinson et al. [8] both found that increasing measurement duration improved CoV for a fixed number of three measurements, probably due to increased chances of obtaining artefact-free recordings. However, the impact of number of measurements on the ability to achieve a specific target CoV cut-off is unknown. There is a practical constraint on how many repeated measurements can be obtained within a single session within a clinical setting because of potential time constraints and subject fatigue, which may ultimately affect the measurements themselves. Furthermore, these factors may also depend on disease state; Timmins et al. [9] had previously shown that within-session variability is typically higher in asthma and COPD compared to health.

Therefore, the aim of this study was to investigate the optimal number of measurements and acceptable within-session CoV in health, asthma and COPD, within a single testing session. In addition, we analysed the number of measurements required to achieve a set target for within-session CoV.

Methods

Study design

Three groups of participants were recruited for this study (15 healthy controls, 15 with asthma and 15 with COPD), from the Woolcock Institute of Medical Research and the Royal North Shore Hospital (Sydney, Australia). Subjects attended a single laboratory session during which their demographic data were collected and they undertook standard spirometry, as well as eight consecutive oscillometry measurements. This study was approved by the Human Ethics Review Committee of the Northern Sydney Local Health District (ethics no. LNR/16/HAWKE/11).

Study subjects

All healthy controls were either nonsmokers or had a smoking history of ≤10 pack-years, no reported history of cardiac or pulmonary disease and no history of regular respiratory or cardiac medication use. Participants with asthma had a respiratory physician diagnosis of asthma, were either nonsmokers or had a smoking history of ≤10 pack-years, as well as an absence of any respiratory disease other than asthma. COPD was defined as a respiratory physician diagnosis of COPD and the absence of any respiratory disease other than COPD, a smoking history of ≥10 pack-years and no exacerbations within the previous 6 weeks; obstruction was confirmed by an FEV₁/FVC (forced expiratory volume in 1 s/forced vital capacity) ratio less than the lower limit of normal [10].

Oscillometry

Participants were instructed to breathe in a relaxed manner on a tremoFlo C-100 oscillometry device (THORASYS Thoracic Medical Systems, Montreal, QC, Canada). Patients sat upright, wearing a nose clip, with their hands firmly pressed against and supporting their cheeks, and thumbs positioned below the chin. After establishing a stable tidal breathing pattern, eight consecutive 30-s measurements were collected. The airwave oscillometry (AOS) perturbation signal was used, which is a pseudorandom noise waveform spanning 5–37 Hz. For this study, we report the resistance (R_rs) measured at 5 Hz (R_rs5) and reactance (X_rs) measured at 5 Hz (X_rs5).

Data analysis

We investigated the effect of number of measurements on oscillometry parameters and within-session CoV. This was carried out by calculating mean and CoV in three ways: 1) from all measurements; 2) from only the first three measurements; and 3) from only the closest three measurements available. From this, the mean and CoV of R_rs5 was assessed progressively by increasing the number of measurements available for evaluation from four to eight measurements, and comparing against the average of the first three measurements.

We also investigated what constitutes an acceptable within-session CoV. Using target cut-offs for CoV of 5%, 10%, 15% and 20%, we successively determined the number of measurements required for at least 95% of the population to fall within these cut-offs when the closest three measurements were selected. This was evaluated for the health and disease groups.

Effect of quality control settings

Given that there remains no general consensus for the appropriate protocol for quality control and cleaning of oscillometry data, we also examined the sensitivity of our results to the effect of different quality control schemes. Four different quality control methods were used. 1) The “sd-based” method: this method excluded any R_rs values that fell beyond a ±5 sd range of the mean. This is the default method in the tremoFlo software (version 1.0.36; THORASYS Thoracic Medical Systems), and is the method used in the main findings presented. 2) The “manual” method: following data collection, whole breaths that contained data artefacts (cough, swallow, vocalisation or breath hold), as apparent on the volume–time trace, were manually excluded from the analysis using the tremoFlo software. 3) The “combined” method: outlier R_rs5 values (±5 sd) were automatically excluded by the tremoFlo software and whole breaths containing data artefacts were additionally and manually removed. 4) The “none” method: no automatic or manual exclusions were applied to the data. In all schemes, negative R_rs values were automatically excluded and within-breath analysis was performed to obtain total, inspiratory and expiratory R_rs and X_rs at 5 Hz and 19 Hz. Only whole breaths were included for the calculation of inspiratory and expiratory oscillometry parameters.

Statistical analyses

For all comparisons, repeated-measures one-way ANOVA with post hoc Bonferroni test was used for normally distributed data and Friedman test with Dunn's post hoc test where data were not normally distributed. Post-bronchodilator spirometry of asthma and COPD patients were compared using unpaired t-test. Data are presented as mean±sd. Within-session repeatability was assessed using intra-class correlation coefficient (ICC; SPSS version 26, IBM SPSS Inc., Armonk, NY, USA, mixed-effects model, absolute agreement, mean of three raters).

Results

Participant characteristics

Participant demographics and lung function are shown in tables 1 and 2. Participants with asthma or COPD were older and had a reduced FEV₁ and FEV₁/FVC ratio compared to healthy controls, with COPD subjects having the lowest FEV₁/FVC ratio.

TABLE 1.

Subject demographics and baseline spirometry

	Healthy	Asthma	COPD
N	15 (4 male)	15 (6 male)	15 (10 male)
Age (years)	30.3±8.5	57.2±21.2**	71.4±9.2****
BMI (kg·m−2)	23.1±2.5	27.1±4.7*	26.2±6.3
Smoking history (never/current/ex)	13/1/1	12/1/2	0/3/12
Smoking history (pack-years)#	0 (0, 0)	0 (0, 0.25)	40 (28, 77)****,####
GOLD stages (I/II/III/IV)	–	–	6/5/4/0
Pre-BD FEV1 (%)	96.6±12.1	90.2±20.0	59.5±21.1****,###
Pre-BD FVC (%)	99.3±9.9	110.9±21.3	90.5±17.8##
Pre-BD FEV1/FVC	82.3±7.2	65.4±8.1***	49.6±13.8****,###
Post-BD FEV1 (%)	–	88.7±20.0	67.0±24.1#
Post-BD FVC (%)	–	105.0±18.0	94.0±18.5
Post-BD FEV1/FVC	–	66.3±8.6	52.1±15.1##

Data are presented as mean±sd or median (interquartile range), unless otherwise stated. BMI: body mass index; GOLD: Global Initiative for Chronic Obstructive Lung Disease; BD: bronchodilator; FEV₁: forced expiratory volume in 1 s; FVC: forced vital capacity. *: p<0.05; **: p<0.01; ***: p<0.001; ****: p<0.0001 compared with healthy controls; ^#: p<0.05; ^##: p<0.01; ^###: p<0.001; ^####: p<0.0001 compared with asthma.

TABLE 2.

Baseline mean and variability of oscillometry measurements for study subjects

	Healthy	Asthma	COPD
Rrs5 (cmH2O·s·L–1)	3.1±1.0	4.9±2.0*	5.0±1.7**
z-score Rrs5	−0.5±3.0	1.2±1.2	1.8±1.2**
CoV (Rrs5) (%)	6.7±4.7	9.7±5.7	12.6±11.2
ICC Rrs5	0.97	0.98	0.93
Xrs5 (cmH2O·s·L–1)	−1.3±0.5	−2.8±2.2	−3.9±3.0***
z-score Xrs5	−0.3±1.8	−2.7±3.8	−4.7±4.9***
ICC Xrs5	0.98	0.95	0.97
Xrs5.in–Xrs5.ex (cmH2O·s·L–1)	−0.7±0.3	1.0±3.1	1.7±3.0**
Rrs5–Rrs19 (cmH2O·s·L–1)	0.2±0.4	1.4±1.2**	1.7±0.8***
AX (cmH2O·s·L–1)	5.2±4.2	28.1±29.6*	38.7±32.4***
VT (L)	0.7±0.3	1.0±1.2	0.8±0.2

Data are presented as mean±sd unless otherwise stated. Variables were calculated from the first three consecutive 30-s measurements using the sd-based quality control method, with no attempt to reduce within-session coefficient of variation (CoV). AX: reactance area; BD: bronchodilator; ex: expiratory; FEV₁: forced expiratory volume in 1 s; FVC: forced vital capacity; in: inspiratory; R_rs5: resistance of the respiratory system at 5 Hz, R_rs19: resistance of the respiratory system at 19 Hz, VT: tidal volume; X_rs5: reactance at 5 Hz. **: p<0.01; ***: p<0.001 compared with healthy controls.

Table 2 also shows oscillometry parameters for the healthy, asthma and COPD groups when considering only the first three consecutive 30-s measurements, using the sd-based quality control method (see supplementary material). Participants with asthma or COPD had a higher R_rs5 than healthy controls and COPD patients had a more negative reactance than in healthy controls with increased expiratory flow limitation (X_rs5.in–X_rs5.ex). There were no significant differences in R_rs5 or X_rs5 between asthma and COPD. In both asthma and COPD there was an increased R_rs5–R_rs19 and area under the reactance curve (A_Xrs) compared with healthy controls, although no difference between disease groups.

Effect of measurement number on oscillometry parameters

When considering all available measurements, there was no overall effect of measurement number on oscillometry parameters across all subject groups. Specifically, when including an increasing number of measurements from four to eight, neither the mean R_rs5 nor the CoV(R_rs5) (%) were significantly different from including the first three measurements (figure 1a and b, respectively).

FIGURE 1.

The mean and within-session variability of total R_rs at 5 Hz does not change with an increasing number of measurements. The mean (a) and CoV % (b) of R_rs5 was calculated after three to eight 30-s oscillometry measurements were carried out on healthy individuals (black squares) and patients with asthma (orange triangles) or COPD (green circles; N=15 for all groups). CoV: coefficient of variation; R_rs5: total resistance at 5 Hz.

When considering only the three closest R_rs5 values from the available measurements, the mean R_rs5 did not change with increasing the number of measurements from four to eight (figure 2a). However, taking five or more measurements resulted in a significant reduction of CoV(R_rs5) for all subject groups (figure 2b). In health, the CoV(R_rs5) decreased from 6.7±4.7% at three measurements to 1.0±0.5% by the eighth measurement (p<0.001). In asthma and COPD this decrease was from 9.7±5.7% to 1.7±1.4%, and 12.6±11.2% to 2.0±1.6%, respectively (p<0.0001 for both).

FIGURE 2.

Selecting the closest three R_rs5 values that were taken over four to eight measurements significantly reduced the CoV compared to when only three measurements were taken. The mean (a) and CoV (b) of total R_rs5 was calculated when the three closest measurements were selected from four to eight 30-s oscillometry measurements carried out on healthy individuals (black squares) and people with asthma (orange triangles) or COPD (green circles; N=15 for all groups). **: p<0.01; ***: p<0.001; ***: p<0.0001 compared with measurement 3 of the respective patient group. CoV: coefficient of variation; R_rs5: resistance of the respiratory system at 5 Hz.

The number of measurements required to achieve target cut-off

In order for at least 95% of subjects to fall within a target CoV(R_rs5) of ≤5%, at least five measurements were needed for health, and at least eight for asthma (figure 3). The COPD group, on the other hand, were unable to obtain this threshold even when taking eight measurements. When the threshold was set at ≤10%, at least four, five and six measurements were needed in health, asthma and COPD, respectively. When the threshold was set at ≤15%, at least four measurements were needed for health and asthma, and five in COPD. At ≤20% threshold, at least three measurements were needed for health and asthma, and four in COPD.

FIGURE 3.

Asthmatic and COPD patients require a greater number of measurements to achieve a CoV R_rs at 5 Hz of ≤5%, ≤10%, ≤15% or ≤20%. The number of measurements needed for 95% of the healthy (black squares), asthmatic (orange triangles) or COPD (green circles) populations to obtain a CoV(R_rs5) of ≤5% (a), ≤10% (b), ≤15% (c), ≤20% (d), when the closest three measurements were selected from four to eight measurements (N=15 for all groups). CoV: coefficient of variation; R_rs5: resistance of the respiratory system at 5 Hz.

The effect of quality control method on oscillometry parameters

The effect of quality control method of oscillometry parameters on the above findings was examined by comparing results generated by the sd-based method, to that produced with the “manual”, “combined” and “none” quality control methods (see supplementary material, Tables S1–S4, and Figures S1–S4). Our observations were consistent regardless of the quality control methods employed.

Discussion

Summary of findings

In this study we examined the within-session variability of oscillometry in a group of healthy controls and patients with asthma and COPD, in order to determine the optimal number of measurements required to achieve an acceptable CoV, within a single testing session. We demonstrated that increasing the number of measurements increases the chances of obtaining at least three measurements within a set target for within-session CoV. Furthermore, in order for the majority of participants to achieve a target CoV(R_rs5) of ≤10%, we required at least four, five and six measurements in the health, asthma and COPD groups, respectively.

Effect of number of measurements on within-session CoV

Increasing the number of oscillometry measurements (up to eight) did not reduce the within-session variability in healthy individuals or patients with obstructive airway diseases, when all measurements were included. However, it did allow for the best three measurements to be selected, thus increasing the probability of decreasing the CoV from when only the first three or all eight consecutive measurements were chosen. The lack of statistical differences in R_rs5 in all these cases suggests that increasing the number of measurements in a clinical session does not provide any additional information in terms of the properties of the airways. Although this latter finding should be interpreted with caution given the small sample sizes in our study, it is supported by a previous study [7].

The within-session CoV values reported in this study (table 2, i.e. 6.7%, 9.7% and 12.6% for health, asthma and COPD, respectively) were generally higher than those demonstrated in a previous study (4.2%, 6.6% and 5.8%, respectively), where 60-s rather than 30-s measurements were used [9]. Measurement duration has been shown to reduce within-session CoV [7]. Our results are closer to those obtained from 30-s measurements in another study, which used a manual exclusion quality control approach (9%, 8% and 6%, respectively) [7]; however, in contrast to our findings, both those studies reported lower within-session CoV for COPD compared to asthma. In a more recent study [11], we were able to calculate within-session CoV from a larger clinical dataset using manually quality-controlled measurements in triplicate, which confirms higher values in disease and more similar values between asthma and COPD (Table S5); the 95th centiles for health, asthma and COPD in that dataset were 12.5%, 13.3% and 17.8%, respectively. It is perhaps not surprising that here we report higher mean CoV values, given that unlike in previous studies, no attempt was made in our study to reduce the CoV during data collection, due to the stated aims of the study.

Within-session variability expressed as ICCs (0.97, 0.98 and 0.93, respectively) were more comparable to other studies in the literature [12, 13]. These results emphasise that ICC is a more reliable group-based measure of within-session variability than CoV, as the latter is more susceptible to outliers and values close to zero, and is a poor indication of quality for X_rs.

Validity of setting CoV cut-offs

ERS standards currently recommend a cut-off of 10% in adults and 15% in children as a quality control target, as is common practice. We have shown that selecting the closest three of four to eight measurements allowed the subject groups to reach target thresholds of CoV(R_rs5) of ≤5%, ≤10% and ≤15%, which was not generally obtainable when using only the first three consecutive measurements. Furthermore, although a target CoV(R_rs5) of ≤10% was achievable across health, asthma and COPD, a greater number of measurements was required in disease. When the CoV cut-off was set at ≤10%, at least four measurements were needed in healthy subjects and at least five and six measurements in asthma and COPD, respectively. The extended testing and repeated coaching involved may not be practical in a busy clinical laboratory setting, where the patient may also have to undergo multiple other tests.

In addition, our results provide additional evidence that increased variability is probably itself an intrinsic marker of obstructive airway diseases, and not just of measurement quality per se. This is supported by the finding that CoV was higher in the asthma and COPD groups, coupled with the observation that within-session variability was correlated with degree of airway disease (data not shown), compared to in health. Higher variability of respiratory impedance in disease and worsening disease status has also been observed in multiple studies in adults [9, 14–17] as well as children [18, 19], and may reflect increased instability in the airways or heterogeneity of accessible lung units [20, 21].

For these reasons, we propose that rather than making repeated measurements in an attempt to reduce CoV, quality control efforts should first and foremost focus on excluding artefacts and outlier breaths, which has previously been shown to impact within-session CoV [8]; where the R_rs frequency and X_rs frequency spectra are available from some software platforms, these could further be used to determine outlier recordings. A target within-session CoV of 10% is achievable within three measurements for the majority of the population, and can be an indicator of a high-quality test. However, for some individuals, particularly patients with respiratory disease, a higher CoV is not necessarily an indicator of poor quality. Based on this study and the upper limits of CoV observed in disease, we suggest that a more relaxed threshold of, for example, 15% or 20% may be classified as “reportable” quality, perhaps within the context of a grading system.

Sensitivity analysis using different quality control criteria

We investigated how our main findings were altered by the use of four different methods of post hoc quality control, aimed at removing points within measurements that were outliers and/or patient-derived artefacts, present during data collection. It would also have presented an opportunity to determine which quality control method provided the most replicable within-session CoV. However, we saw that the quality control method chosen had minimal effect on how CoV(R_rs5) varied with number of measurements. In particular, when comparing the automatic quality control (sd method) with the manual method (a more stringent quality control method), the results did not vary significantly, with only a slightly higher chance of getting more acceptable results in asthma but not COPD at CoV(R_rs5) of ≤15%. This lack of dependence on quality control method is observed despite the fact that the default sd method is relatively permissive, allowing values within up to 5 sds to be included, whereas the manual method would have excluded whole breaths (including high values) that appeared aberrant. It is also in contrast to our previous findings in children comparing quality control measures based on the 5-sd method, 3-sd method, and manual exclusion of whole breaths – only the latter had a significant impact on within-session CoV [8]. It may be that in our study, sufficient artefact-free breaths were captured within a 30-s recording to provide an accurate and robust estimate of R_rs5, and consequently of within-session CoV. It is also worth noting that our findings showed it is possible to achieve excellent reliability (in terms of ICC) from just three measurements.

Limitations

The sample sizes were relatively small, and the asthma and COPD groups in our study were older and contained a range of disease severities. However, the sample size is comparable to previous studies examining within-session variability [7, 9, 19], and the age and heterogenous nature of the disease groups was an accurate representation of the populations attending respiratory clinics. Despite these limitations the results provide valuable insight for further development of oscillometry standard operating procedures. We also did not investigate effects on the CoV of X_rs parameters, as the high susceptibility of X_rs to outliers (due to its proximity to zero) limits its practical utility as a quality control measure in the first place.

Conclusion

In conclusion, we demonstrated that increasing the number of measurements does not alter oscillometric measures of airway resistance, although it does increase the chances of obtaining at least three measurements within a set target. However, and more importantly, within-session variability is greater in disease, and while the recommended target cut-off of CoV(R_rs5) ≤10% is generally achievable, the higher number of measurements required to achieve this target, particularly in disease, may not be practical in a clinical setting. Hence, quality control should be focused first on removing artefacts and outliers, and a within-session CoV ≤10% viewed as a marker of high quality as recommended by current ERS standards, but here we provide evidence that a within-session CoV of up to 15–20% (particularly in disease) is not necessarily a marker of poor quality and should be reportable. Our findings can be used in conjunction with current oscillometry guidelines and recommendations and may help in the development of future recommendations on methodology.

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material 00074-2021.supplement ^{(317.2KB, pdf)}