Calibration of bronchial methacholine challenge: addressing dose accuracy and practicality

ABSTRACT

Methacholine bronchial provocation (BMP) is a valuable tool in supporting the diagnosis of asthma, but the BMP must be validated regarding dosing, since the BMP basically is a dose response study.

Historically, the dose delivered by a nebulizer has been calibrated gravimetrically, by weighing the nebulizer before and after dosing. However, this method is no longer recommended, since it has been recognized that a large fraction of the weight loss was due to evaporation. Unfortunately, practical alternatives are not available, forcing clinicians to rely on the manufacturer's specified dose output. We studied the validity of the dose claimed to be delivered by the Vyaire APS-Pro.

Methods

To determine the dose output, we applied a radioactive method, considered the gold standard, and we validated a commercially available chemical analysis of chloride.

Results

The output from the APS-Pro was found to be highly correlated (R² = 0.94) with the dose specified by the APS-Pro software but was consistently 1.8 times higher. The new chemical method demonstrated accuracy comparable to the radiometric approach. Notably, we observed significant variations in output across different nebulizers.

Discussion

The methacholine dose delivered to the mouth via the APS-Pro does not align with company specifications due to variability between nebulizers and a non-linear relationship between nebulization time and output, leading to higher output rates during shorter nebulization periods.

Conclusion

Individual output calibration of specific BMP systems remains necessary, as current systems still do not consistently meet manufacturer’s specifications. Clinicians must therefore have access to practical calibration methods.

Introduction

Bronchial methacholine provocation (BMP) is important for diagnosing asthma [1] and requires standardization to ensure the accuracy of the aerosol dose delivered during testing. The interpretation of BMP results depends primarily on an accurate and precise measurement of the dose inhaled but also of the quality of spirometry performed during the methacholine challenge. However, unlike spirometry [2], there is no established quality grading system for BMP. The response to the methacholine challenge is dose-dependent, but obtaining accurate information on inhaled doses from commercial bronchoprovocation systems is challenging. The lack of valid dose output information from manufacturers necessitates traceable calibration by the end-user.

Historically, the ‘gravimetric method’ was used to estimate the nebulizer output [3,4], but it overestimates due to evaporation [4,5]. The current gold standard for dose determination involves depositing aerosols on filters and measuring the solute output chemically or with radioactivity, though these methods are limited to specialized laboratories [6]. Therefore, there is a need for more accessible methods to determine dosimeter-nebulizer output or development of dosimeter-nebulizer systems that comply with manufacturers' dose specifications.

Our research focused on validating the dose delivered by the APS-Pro dosimeter (Vyaire, Höchberg ,Germany), quantifying dose variability between nebulizers, examining the impact of nebulizer fill volume and assessing dosimeter driving pressure and dosing timing compliance with specifications. We also aimed to evaluate a commercially available chemical analysis method for methacholine chloride, with the goal of improving feasibility of dosimeter output calibration methods.

Methods

Design

This laboratory study did not involve patients or require ethics committee approval. The study was conducted in two phases, interrupted by the Covid-19 pandemic. In the first phase, dosimeter calibration using the gravimetric method was performed on two APS-Pro systems. In the second phase, calibration was performed with ^99mTc-DTPA (diethylenetriaminepentaacetic acid).

Settings and materials

The radiometric study was performed at the Department of Clinical Physiology and Nuclear Medicine, Rigshospitalet, Copenhagen. The gravimetric study was performed at the Allergy and Lung Clinic Helsingør. The study of nebulizer driving pressures was performed at Bispebjerg Hospital, Copenhagen.

The study was conducted under ambient conditions. Calibration of dosimeter output with ^99mTc-DTPA and chemical analyses of deposited methacholine chloride were performed on a single APS-Pro. Measurements of nebulizer driving pressures and nebulization times were conducted at different locations using four APS-Pro and one MEC Challenger dosimeter (MEC, Bruxelles, Belgium).

Measurement methods

Gravimetric method

One Sidestream single patient reusable nebulizer was weighed before and after nebulization of methacholine chloride. Doses were triggered by simulated tidal volume breathing using a calibration syringe (for details, see supplementary material).

Radiometric method

A HEPA filter (Westmed 6216, Timik, Odense, Denmark) was used to capture the aerosol, with radioactivity and methacholine measured to determine the dose deposited on the filter. The experimental setup is shown in Figure 1.

Figure 1.

Experimental setup for aerosol generation and collection on filters.

Chemical analyses

Filters were analyzed by an external accredited laboratory to determine deposited methacholine chloride content (supplementary material).

The accuracy and precision of both the radioactive and chemical methods were validated by applying doses of ^99mTc-DTPA and methacholine defined by a pipette to filters and micro tubes and then analyzed (ISO 10,304-1:2009 [7]) (supplementary material).

Programming of the dosimeter

The APS-Pro was programmed by the local Vyaire representative using SentrySuite software to follow three specific one or two concentration challenge protocols (supplementary material table S1).

Ventilation

Dosing was triggered by simulated tidal volume breathing with a manually operated 2 L calibration syringe [8] (Jaeger, Hochberg, Germany). Mean tidal volume (V_T) was 0.765 L (mean) (SD = 0.007) and frequency (RF) 14.8 min⁻¹ (SD = 0.30) (supplementary material: ‘Ventilation’).

Pressure measurements

Nebulizer driving pressures were measured with a Wika CPG 1500 differential storage manometer (Wika Denmark) (calibrated by the manufacturer) via a T-piece inserted before the nebulizer. Sample rate is 50 s⁻¹.

Results

The variation in dose resulting from variation in methacholine mass between Provocholine 100 mg vails was negligeable since variation between vials was less than 10 mg (5%) resulting in a maximal variation in methacholine concentration between 19 and 21 mg mL⁻¹.

Output from the nebulizer was not related to fill volumes in the range 1–4 mL (Table 1) or RF (Table 2). A smaller V_T resulted in a higher deposition of ^99mTc-DTPA (Table 2).

Table 1.

Nebulizer solute (dose) and solution output and effect of filling volume.

Variables	Nebulizer no. 1	Nebulizer no. 1	Nebulizer no. 1
Filling volume mL	1	2	4
Output ꙡL	0.62	0.65	0.60
Output ꙡL s−1	1.23	1.29	1.21
Output mg methacholine	0.012	0.013	0.012

Sidestream no. 1. Single patient re-usable nebulizer. MEC dosimeter. Pressure 1.5 B and nebulization time 0.5 s. Ventilation RF 15 min⁻¹ V_T 750 mL. Mean of 10 doses of methacholine chloride 20 mg mL⁻¹ and ^99mTc-DTPA. Output = output from single activation of dosimeter. Output (dose) in mg is calculated from the radioactivity of the deposited aerosol.

Table 2.

Nebulizer solute (dose) and solution output and effect of the ventilation pattern.

Tidal volume ml	500	500	500	750	750	750
RF min−1	10	15	20	10	15	20
Output ꙡL	2.72	2.59	2.60	1.06	0.89	0.92
Output ꙡL s−1	5.45	5.17	5.20	2.13	1.79	1.85
Output mg methacholine	0.05	0.05	0.05	0.02	0.02	0.02

Sidestream no. 1 single patient re-usable nebulizer. Fill volume 2 mL MEC dosimeter. Pressure 150 kPa and nebulization time 0.5 s.  Mean of 10 doses of methacholine chloride 20 mg mL-1 and 99mTc-DTPA. Output = output from single activation of dosimeter. Output (dose) in mg is calculated from the radioactivity of the deposited aerosol.

Nebulizer no. 1 was tested 5 times, and the results show a high dose repeatability [9] (test same day) and slightly lower dose reproducibility [9] (test on different days) (Table 3) but there was up to a factor of two differences in output between nebulizers not previously used.

Table 3.

Nebulizer solute (dose) and solution output. Methacholine 20 mg mL⁻¹ from 6 different nebulizers.

Test day	1	1	1	3	2	2	2	2	2	2
Nebulizer no.	1	1	1	1	1	2	3	4	5	6
Filter dose 10 actuations mg	0.12	0.13	0.12	0.18	0.16	0.21	0.28	0.30	0.20	0.27
Filter dose one actuation mg	0.01	0.01	0.01	0.02	0.02	0.02	0.03	0.03	0.02	0.03
Filter dose one actuation mg s−1	0.02	0.01	0.02	0.04	0.03	0.04	0.06	0.06	0.04	0.05
Output, µL s−1	1.23	1.29	1.21	1.79	1.55	2.1	2.76	3.00	2.04	2.72
Output µL	0.62	0.65	0.60	0.89	0.77	1.10	1.38	1.50	1.02	1.36

Output from five new Sidestream reusable nebulizers (1–5) and one used (6). Fill volume 2 ml. M.E.C. dosimeter with nebulization time 0.5 s and RF 15 min⁻¹, V_T 750 ml. Nebulizer 1 was tested thrice in one day (test 1 repeatability) and days later (test 2 + 3) reproducibility. Mean of 10 doses of methacholine chloride 20 mg mL⁻¹ and ^99mTc-DTPA. Output = output from single activation of dosimeter. Output (dose) in mg is calculated from the radioactivity of the deposited aerosol.

The solute output (dose) from the APS pro was highly correlated (R² = 0.94) to the specified dose in a wide testing range and the difference between specified and measured dose was increasing with increasing doses up to 10,000 µg (Figure 2). However, in the more clinically relevant testing range (methacholine dose < 500 µg) the difference between measurements was not correlated to the testing range (figure S1).

Figure 2.

Correlation between nominal solute output (calculated) and radiometric measured dose of methacholine deposited on filter.

The solution delivered per nebulization time (nebulization rate) from the dosimeter-controlled nebulizer was inversely proportional to the absolute nebulization time (figure S2).

The gravimetric study showed the same pattern as the radiometric method. The measured dose was highly correlated to the specified dose, but the measured dose was approximately a factor two higher than the APS-Pro specified dose (figure S3).

The deposited methacholine dose determined with chemical analyses and the dose determined with ^99mTc-DTPA was correlated (Figure 3) (R² = 0.97 y = 0.06 + 1.82x).

Figure 3.

Correlation between solute output of methacholine measured with radiometric method and with chemical analyses.

The validation study of the chemical and radiometric method showed a high accuracy and precision of both methods when methacholine chloride was deposited into micro tubes and filters, respectively (figure S4).

Driving pressure: A typical example of an APS-pro nebulizer driving pressure-time relationship with a rapid increase and a slower decrease in pressure is shown in supplementary material (figure S5 and S6). Four APS-pro systems pressure profiles were compared (Figure 4). The measured pressures show the same pattern for all systems except ‘Room 4’ which shows a markedly and repeatedly higher pressure at step P1.

Figure 4.

Nebulizer driving pressure profiles during bronchial challenge with 4 different APS-Pro systems.

Nebulization time is not well defined (supplementary material figure S4 and S5) but can be estimated from the pressure-time graph. It is possible to calculate the proportion of time a specified pressure is reached (figure S5). App. 50 percent of the pressure measurements were below the pressure specified by the manufacturer [10] (1.4 B ± 0.2 B).

Discussion

In this study, we questioned the validity of specifications provided by the manufacturer of a commonly widespread bronchial challenge system, the APS-Pro marketed by large international respiratory equipment companies for more than 20 years. We were also searching for a practical method for nebulizer calibration applicable in clinical practice.

We found a high overall agreement of the methacholine dose calculated by the SentrySuite software and the radiometric calibration (Figure 2). However, both the accuracy and precision of the APS-Pro equipment seems to be overestimated as judged from doses reported with a precision down to a single ꙡg which might be misleading. Agreement of ‘software calculated methacholine dose’ and measured dose in the clinically relevant range (< 400 µg) was within ±150 µg (Figure 2) but only for a specific nebulizer. Since between nebulizer output variation was up to 2-fold, we must accept that uncertainty of dose roughly is at least ±300 µg) methacholine unless the specific setup is calibrated. Therefore, it is not possible to determine a precise PD₂₀ based on APS-Pro calculated dose as suggested by Schulze [11] and instead output calibration of the specific systems is still necessary to obtain a precision better than ± 450 µg. Large differences in nebulizer output have been known for many years [12] and are acknowledged by the manufacturer [13].

Another source of variation in methacholine doses is the ventilatory pattern during dosing and we choose to comply with the ventilation mode recommended by the ERS Technical standard for bronchial challenge [14]. This resulted in a repeatable output from a specific nebulizer (Table 1).

Driving pressure and nebulization time are two other determinants of nebulizer dose and thus sources of variation in PD₂₀. We found that nebulization time was not well defined since the pressure-time relationship was not a square function (supplementary material figure S4 and S5), and we found that the most recent claimed driving pressure (1.4 B) was reached for approximately half of the nebulization time only (figure S5). We also found that the output per nebulization time (rate) (mL s⁻¹) was related to the absolute nebulization time showing relatively higher output at short nebulization times which also makes calculation of dose from dosimeters based on constant (without dosimeter) output calibration data invalid (supplementary material figure S2).

Our primary results on solute output from the APS-Pro are in line with previous findings but only few studies have reported the solute output using rapid single or two concentration protocols for the APS-Pro. Praml [15] found a solute output per time (1.27 mikL s⁻¹ Sidestream nebulizer) surprisingly close to our findings (1.29 mikL s⁻¹) (Table 1) considering the many sources of variation especially the nebulizers. Gatnash [16] found a 1.9 mikL dose per actuation (Sidestream) compared to our findings (0.6–1.5 mikL) but the differences between the studies might be too large to make comparison of doses relevant.

Large differences between output from nebulizers of the same brand have been reported for years [12,17] but the problem has not yet been solved for jet-nebulizers even though recognised by the manufacturer 20 years ago (Care Fusion Appl_Note_Provocation_E Page 2 Update: 21.04.04-HE).

Besides output differences between individual nebulizers, we also found differences in the nebulizer operating conditions (driving pressure) between different APS-Pro systems and difficulties in defining nebulization time. To our knowledge, details of the time-pressure relationship and comparison of this between provocation systems have not been reported previously even though highly relevant for standardization of the nebulized dose. These factors also indicate that dose calibration of individual systems is still necessary and therefore much more practical traceable calibration methods are strongly needed.

We tested commercially available chemical analyses of NaCl which can be converted to methacholine chloride content and compared it to the radiometric method (Figure 3). The chemical analysis was validated with traceable calibration instruments, and we found that the accuracy and precision are sufficient (supplementary material figure S4). The difference in accuracy of the chemical analyses between the results from the validation study (micro tubes and filters) and the study of the dosimeter might be due to the extraction process which must be focused on when the method is explored further.

Other chemical analyses [18,19] have been applied and recommended in the ERS standard [14] but they are not readily accessible to clinicians to our knowledge like the chemical analyses recommended in the ERS technical standard [14].

Strength and limitations of the study

We applied the radiometric method which is optimal for the quantification of deposited solutes in aerosol studies. Our instruments were carefully calibrated with traceable methods, and the procedures were performed by a trained staff only.

One weakness might be that we have examined only two dosimeter-nebulizer systems, but we have chosen the most widespread used dosimeter in our community where more than 70 systems has been delivered covering more than 90% of the departments performing bronchial challenges according to the company representative and National hospital statistics. Our findings are probably relevant for most dosimeters.

It might also be considered a weakness that we did not apply a breathing simulator but for general usability reasons we preferred using manually operated syringes. Considering the wide variation in spontaneous ventilation during bronchial challenge [20–22], it should be considered how clinically relevant a tighter expensive control of ventilation is.

We used the ventilation mode recommended by the ERS standard [14] and tested the accuracy and precision of our ventilatory performance, which was remarkably good.

Our findings show that we cannot rely on the specifications for the APS-Pro concerning dose output, and we are confirming old but may be neglected evidence that output from nebulizers show large variations [12]. Our findings indicate that it is still necessary to calibrate bronchial challenge systems individually to obtain satisfactory accuracy and precision of dose.

In search of a traceable methacholine-dose calibration method that is more practical for clinicians than the radiometric method, we tested a commercial chemical analysis of methacholine (chloride content). We found acceptable agreement with the radiometric method (Figure 3), and our validation study showed a high accuracy and precision of the chemical method.

The gravimetric method, which has been used routinely for many years [3], is well known to overestimate output due to evaporation, but it is reasonable to assume that it can be useful to secure a ‘constant output’ over time in a specific setting. Furthermore, the gravimetric method might be a highly relevant alternative to no calibration at all, and the gravimetric method has previously been considered acceptable by the ERS [3,4].

The clinical implication of our study is primarily related to the diagnostic process of asthma where clinicians must integrate test uncertainty, as with other tests and accept that the PD₂₀ (provocative dose inducing 20% fall in FEV₁) measured with the APS-pro has a lower accuracy than indicated by the specifications although they are not stated precisely in the documentation. A PD₂₀ is considered repeatable within 1.6 doubling dose [23–25] but that is not the same as reproducibility (long term intervals or different settings and equipment) which may be much lower making interlaboratory comparison difficult as long as traceable calibration methods are not applied. Therefore, it is too simplistic to use terms such as a positive or negative bronchial challenge, as used by in GINA [1] and instead we should be careful and calculate probabilities for asthma based on PD_20, determined with known uncertainty, as suggested by Fagan [26,27].

To compensate for the variation in dose between nebulizers and low or unknown dose accuracy, a challenge protocol with a high total dose of methacholine could be chosen, making it possible to rule out asthma with a high probability [14,24].

The problem with large output differences between nebulizers has been accentuated since the COVID-19 pandemic, which increased the safety procedures and demanded single patient nebulizers or sterilizing the nebulizer between patients.

Clinicians strongly need access to much more practical traceable calibration methods for BMP or equipment which comply with specifications [6] as stated in the recent ERS standard [14]. Unfortunately, traceable calibration equipment or services have not yet been made available to clinicians or researchers.

Conclusion

Bronchial challenge dosimeters and jet nebulizers unfortunately still need laborious individual calibration after more than 40 years knowledge of the problem with inter nebulizer variability [12]. Clinicians must have access to practical and valid calibration methods until the manufacturers guarantee accurate dosing [6]. Until that happens, we need to take the uncertainty of BMP methods into consideration, when using the results for diagnostic purpose, as with all other diagnostic tests and therefore the uncertainty of the bronchial challenge must be declared and documented to clinicians.

Funding Statement

This work was supported by the FAS foundation for the development of quality in specialist practice.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Grants

FAS foundation for the development of quality in specialist practice

Supplementary material

Supplemental data for this article can be accessed online at https://doi.org/10.1080/20018525.2025.2546678.