Exercise-induced Bronchoconstriction: Reproducibility of Hyperpolarized ³He MR Imaging

Ventilated volume appears to be a robust metric for evaluating ventilation in patients with exercise-induced bronchoconstriction, and the results of this study corroborate those of studies of other obstructive lung diseases.

Abstract

Purpose

To quantitatively evaluate interday, interreader, and intersite agreement of readers of hyperpolarized helium 3 (HPHe) MR images in patients with exercise-induced bronchoconstriction.

Materials and Methods

This HIPAA-compliant, institutional review board approved study included 13 patients with exercise-induced bronchoconstriction. On two separate days, HPHe MR imaging of the lungs was performed at baseline, immediately after a 10-minute exercise challenge (postchallenge), and 45 minutes after exercise (recovery). Patients were imaged at two sites, six at site A and seven at site B. Images were analyzed independently by multiple readers at each site. Lung volume, ventilation defect volume, ventilated volume, and the number of defects were measured quantitatively, and the location of defects was evaluated qualitatively at site A. Interday and interreader agreement were evaluated by using the intraclass correlation coefficient (ICC), and intersite agreement was evaluated by using a modified Bland-Altman analysis.

Results

The ICC between days for ventilation defect volume, ventilated volume, and number of defects was at least 0.74 at both sites. The ICC for lung volume was greater at site B (0.83–0.86) than at site A (0.60–0.65). Defects seen in the same location in the lung on both days included 19.7% of those seen on baseline images and 29.2% and 18.6% of defects on postchallenge and recovery images, respectively. Interreader ICC for each measurement was at least 0.82 for each site. Analysis of intersite agreement showed biases of 612 mL for lung volume, −60.7 mL for ventilation defect volume, 2.91% for ventilated volume, and −6.56 for number of defects.

Conclusion

The reported measures of reproducibility of HPHe MR imaging may help in the design and interpretation of single- and multicenter studies of patients with exercise-induced bronchoconstriction.

© RSNA, 2012

Introduction

Exercise-induced bronchoconstriction (EIB) is the obstruction of lung airways after strenuous exercise. It is often, but not always, associated with chronic asthma (1,2). The bronchoconstriction is regionally heterogeneous and causes the development of spatially variable areas of low ventilation (ventilation defects) distal to affected airways (3). Symptoms of EIB are transient and self-limiting, reaching a maximum during the first 10 minutes after exercise and dissipating in 30–60 minutes (4,5). Testing for EIB typically involves spirometric measurements before and after a standard exercise protocol, and a decrease in forced expiratory volume in 1 second (FEV₁) of 10% or more is considered to be diagnostic of EIB (6).

Spirometric testing is a simple, low-cost method for diagnosis of EIB. However, spirometry results reflect FEV₁ for the whole lung and do not provide information on regional bronchoconstriction or ventilation. The size and location of defects may have important implications in determining disease severity or the efficacy of drug treatment. Several lung imaging techniques to measure regional ventilation are available, including high-resolution computed tomography (CT) (7), nitrogen 13 positron emission tomography (PET) (8), and hyperpolarized helium 3 (HPHe) magnetic resonance (MR) imaging. CT and PET are not ideal for longitudinal studies that involve multiple visits or imaging before and after exercise, particularly in younger patients, because they involve the use ionizing radiation. HPHe MR imaging involves the use of a biologically inert nonradioactive contrast agent that relaxes from the hyperpolarized state slowly relative to the length of the imaging experiment (T1 = 20–30 sec at 1.5 T). Therefore, HPHe MR imaging is useful for longitudinal imaging of lung ventilation in disorders such as EIB.

Although HPHe MR imaging is a promising approach for assessing lung function, the reproducibility of semiquantitative and quantitative HPHe MR imaging biomarkers has not been thoroughly evaluated. Images must be analyzed for metrics such as defect number, location, size, and severity, and this analysis is often based on the subjective judgment of a human reader, who can introduce an unknown degree of bias and variability. In addition, different research groups may use different methods or criteria for identifying defects. Three aspects of reproducibility that must be evaluated include (a) the agreement of a single reader’s evaluations of images for ventilated volume–associated measures and defect locations in the same patient imaged on separate days; (b) the agreement among multiple independent readers who used the same visualization and measurement tools; and (c) the agreement among multiple independent readers who used different visualization and measurement tools at different imaging centers. This study is a quantitative evaluation of interday, interreader, and intersite agreement of HPHe MR imaging in patients with EIB.

Materials and Methods

Patients

We received support for this study from Merck and equipment from GE Healthcare. Authors who had no affiliation with these companies had full control over the study and the data at all times. Merck required a 30-day review of the manuscript in advance of publication, but the scientific integrity and results were entirely under the purview of the principal investigators (S.B.F., N.N.J.). The study was conducted in accordance with Health Insurance Portability and Accountability Act regulations and was approved by the internal human patients review boards of both institutions. Thirteen adult patients with mild asthma and a history of EIB were included in this study (mean age ± standard deviation for the entire group, 24.4 years ± 7.0; men, 24.8 years ± 2.6; women, 24.2 years ± 8.4). All patients had received spirometric test results showing a decline of at least 15% in FEV₁ after performance of a 10-minute exercise challenge. The purpose of this study was to establish the robustness of HPHe MR imaging results as biomarkers of pulmonary function in patients with obstructive lung disease at two different imaging sites with different equipment platforms. Six patients were imaged at the University of Wisconsin (site A), and seven patients were imaged at the Robarts Research Institute, University of Western Ontario (site B).

MR Imaging before and after Exercise Challenge

Patients performed a 10-minute treadmill exercise challenge during two visits separated by 7–14 days. HPHe MR images were acquired 3.5 hours before exercise (baseline), immediately after exercise (postchallenge) and 45 minutes after exercise (recovery) (Fig 1). Spirometric measurement of FEV₁ and forced vital capacity was performed within minutes of each MR imaging examination to verify changes in pulmonary function. To control for diurnal variation in lung function, patients were imaged at approximately the same time of day for both visits.

Figure 1:

Timeline of exercise challenge and HPHe MR imaging examinations during each visit.

Imaging was performed at site A by using a 1.5-T Signa HDx MR imaging system (GE Healthcare, Milwaukee, Wis) with a flexible-wrap single-channel volume coil (IGC Medical Advances, Milwaukee, Wis). At site B, a 3-T Signa HDx MR imaging system (GE Healthcare) was used with a rigid-body single-channel volume coil (Rapid Biomedical, Rimpar, Germany). Helium 3 (³He) was hyperpolarized by using a Heli-Spin prototype commercial polarizer (GE Healthcare Biosciences, Durham, NC) and the spin-exchange optical pumping method described by Möller et al (9).

HPHe was prepared as a 4.5 mmol per liter dose mixed with nitrogen to a total volume of 1L in a polyvinyl fluoride bag (Jensen Inert Products, Coral Springs, Fla). The bag was purged of oxygen to slow T1 relaxation. While patients were in the imager, they inhaled the gas from functional residual capacity through a small plastic tube attached to the bag and were imaged during a 16-20–second end-inspiration breath hold. Blood oxygen saturation was monitored by means of pulse oximetry, and 100% oxygen was administered through a nasal cannula between imaging sequences to promote respiratory recovery. No adverse events were observed from gas inhalation or imaging procedures.

HPHe MR imaging at both sites used a fast two-dimensional gradient-echo sequence with the following parameters: repetition time, 3.6 msec; echo time, 1.1 msec; flip angle, seven degrees; number of sections, 12–16; field of view, 40 cm; acquisition matrix, 128 × 128; section thickness, 15 mm; and centric encoding. The acquisition matrix was reconstructed to a 256 square matrix, yielding a voxel size of 1.56 × 1.56 × 15 mm. At site B, proton images were acquired after patients inhaled a 1-L bag of ⁴He from functional residual capacity with a three-dimensional spoiled gradient-echo sequence with a repetition time of 3.0; echo time, 1.1 msec; flip angle, 10 degrees; field of view, 40 cm; acquisition matrix, 256 × 224 (80% phase field of view); and section thickness, 10 mm.

Image Analysis

Images from all 13 patients were analyzed by two readers at site A (S.B.F. and D.J.N., with 10 and 2 years of experience, respectively) and three readers at site B (A.W., with 2 years and H.A. and S.C., both with 1 year of experience). Readers used the imaging approach of their respective sites and customized visualization and measurement tools developed in MATLAB (Mathworks, Natick, Mass). At site A, total lung volume (V_L) and the total volume of defects (V_D) were measured by means of manual segmentation of lung and defect boundaries of HPHe images (Fig 2). The number of defects (N_D) was automatically calculated in MATLAB on the basis of the connectivity of the segmented regions of defect. Defective regions that were not connected horizontally, vertically, or diagonally (eight connectivity) were counted as separate defects. At site B, defects were identified by consensus of the three readers, after which each reader independently measured V_L by manually segmenting registered proton images (Fig 2a) taken at baseline and V_D by manually segmenting HPHe images taken at each time point. Ventilated volume (V_V), a common imaging biomarker of ventilation defects (10), was calculated at both sites as . Readers performed all image analysis while blinded to the randomized patient identification, day of imaging, and time point. With the exception of measurement of N_D at site B, each reader’s measurements were independent.

Figure 2a:

Coronal images of lungs show examples of segmentation steps performed at each site. (a) Segmentation of V_L from proton image acquired in patient at site B. (b) Segmentation of V_L from HPHe image of patient at site A. Ventilation defects appear as dark regions. (c) Segmentation of V_D from HPHe image.

Figure 2b:

Coronal images of lungs show examples of segmentation steps performed at each site. (a) Segmentation of V_L from proton image acquired in patient at site B. (b) Segmentation of V_L from HPHe image of patient at site A. Ventilation defects appear as dark regions. (c) Segmentation of V_D from HPHe image.

Figure 2c:

Coronal images of lungs show examples of segmentation steps performed at each site. (a) Segmentation of V_L from proton image acquired in patient at site B. (b) Segmentation of V_L from HPHe image of patient at site A. Ventilation defects appear as dark regions. (c) Segmentation of V_D from HPHe image.

Interday Agreement

Interday agreement refers to the similarity of measurements obtained at HPHe MR imaging examinations in the same patient on separate days and evaluated by a single reader, or a single set of readers in the case of analysis by consensus. We evaluated the agreement of the location of defects and the values of V_L, V_D, V_V, and N_D obtained from image analysis. Two radiologists at site A (S.K.N. and C.J.N., with 7 and 9 years of experience, respectively) identified the location of individual defects by consensus and determined whether each defect appeared on images from both days. The reproducibility of defect location was then calculated for each patient and time point as the number of defects seen on images from both days divided by the total number of defects seen on images from either day. Images were excluded if no defects were seen on those of either day at a given time point (four of 78 examinations). Agreement of V_L, V_D, V_V, and N_D between the two days was evaluated separately for each of the five readers by using the intraclass correlation coefficient (ICC) and repeatability, with the latter defined as 2.77 times the standard deviation of the results of between the two imaging days in each patient (11). All ICC calculations in this study were tested for absolute agreement rather than for consistency.

Interreader Agreement

Interreader agreement refers to the similarity of measurements among all readers at a single site. Interreader V_L, V_D, V_V, and N_D at a single site were evaluated by using the ICC.

Intersite Agreement

Intersite agreement on V_L, V_D, V_V, and N_D values was evaluated by using a modified Bland-Altman analysis with the measurements by each reader treated as repeated measurements. An in-depth description of this method has been previously published (12,13). The presence of repeated measurements results in increased variance of measurement differences and a wider confidence interval (CI) than the traditional Bland-Altman analysis. If measured values at site A and site B are denoted as X and Y, respectively, the total variance of each method can be described as

where is the variance of the true values, and are the variances of each method by patient interaction, and and are variances of measurements in an individual patient for each method. The goal is to find the variance of the difference between single measurements by each method, Var (X – Y), to determine the CI. For m_x and m_y repeated measurements at site A and B, respectively, this quantity becomes

where and are the average observed values among readers. The Bland-Altman bias,, is equal to and the CI is calculated as .

Results

Interday Agreement

Figure 3 shows typical images from examinations 7 days apart at site A (Fig 3, A, B) and site B (Fig 3, C, D). Defects appear with similar size and location in the lungs in each examination, as indicated by the arrows. On average, defects were seen at the same location on images from both days with different frequencies across the time points (Fig 4). Of the defects seen on baseline images, 19.7% ± 16.8% (range, 0%–40.7%) were present at the same location on the second day of imaging, compared with 29.2% ± 14.6 (range, 0%–56.5%) and 18.6% ± 16.9 (range, 0%–56.5%) of defects seen on postchallenge and recovery images, respectively. The frequency of repeated defects was significantly higher on postchallenge images compared with those obtained at baseline and recovery (P < .05, Wilcoxon signed-rank test).

Figure 3:

Typical coronal HPHe MR images acquired at baseline (left), postchallenge (middle), and recovery (right) time points for one patient on the, A, first and, B, second visits at site A and one patient on the, C, first and, D, second visits at site B. Defects (arrows) appear with similar size and location during each visit.

Figure 4:

Box-and-whisker plots show percentage of defects observed in the same location in lung at both visits. Whiskers indicate largest and smallest values within 1.5 times interquartile range, and median values are indicated by horizontal lines in boxes. Significant differences (*) were observed between baseline and postchallenge values and between postchallenge and recovery values.

Interday ICC and repeatability values of V_L, V_D, V_V, and N_D for each of the five readers are shown in Table 1. The ICC values for V_D, V_V, and N_D were at least 0.74 in all readers and showed little difference between sites. However, the ICC of V_L showed a large difference between sites (0.60–0.65 at site A compared with 0.83–0.86 at site B), likely due to the use of different image sets for V_L measurement: HPHe images at site A and proton images at site B. As might be expected, interday repeatability varied inversely with interday ICC.

Table 1.

Interday ICC and Repeatability for Each Reader

Note.—Percentages for V_V were calculated as .

^*(Units in parentheses apply only to repeatability columns. Other data are ICCs.

Interreader Agreement

Interreader ICC values for each site are shown in Table 2 and indicate good agreement for all independently assessed measures. Note that the N_D ICC values for site B were excluded because defects were identified by consensus at this site.

Table 2.

Interreader ICC at Sites A and B

Note—N_D was not analyzed for site B because these defects were identified by consensus among readers.

Intersite Agreement

Table 3 shows the average values for V_L, V_D, V_V, and N_D measured at each site for baseline, postchallenge, and recovery HPHe MR imaging examinations. V_L at site B was measured only at baseline and then replicated throughout examination time points. V_L (site A only), V_D, and N_D all increased after exercise, while V_V decreased, indicating symptoms of EIB. Although examination results from both sites show similar trends in measurements, there were biases between sites, as shown in the Bland-Altman analysis in Table 4. V_D and N_D were consistently higher at site B than at site A, while V_L and V_V were higher at site A. The measurement with the least bias was the V_V (−2.91%), which had an intersite CI of less than ±10.3%.

Table 3.

Mean Values for All Examinations for Each Site

Note.—Values are mean ± standard deviation. Percentages for V_V were calculated as .

Table 4.

Bland-Altman Analysis Comparing Methods across Sites

Note.—Measurement differences were calculated as A minus B. Percentages for V_V were calculated as .

Discussion

Reliable estimates of reproducibility are necessary to compare HPHe MR imaging results between studies and to evaluate changes in ventilation after intervention such as drug treatment. Values for agreement and repeatability can be interpreted the minimum expected for the uncertainty of HPHe MR imaging measurements. To our knowledge, this study presents the first analysis of reproducibility of HPHe MR imaging of patients with EIB that compares results of image analysis at different sites.

The reproducibility of defects appearing at individual locations in the lungs throughout visits varied according to the extent of bronchoconstriction and/or the exercise state. Only 19.7% ± 16.8, and 18.6% ± 16.9 of defects seen at baseline and recovery, respectively, appeared during both visits. This suggests that the location of defects that occur without exercise is highly variable. By comparison, 29.2% ± 14.6 of defects were visible in the same location at both visits after airway challenge with exercise. This observation is consistent with other work (14) showing recurrence of defects at the same locations in patients with asthma before and after methacholine challenge. Repeated bronchoconstriction in particular airways after exercise may reflect long-term remodeling, persistent injury, or sensitization of these airways.

The interday ICC of V_D, V_V, and N_D were all at least 0.74 for all readers, suggesting agreement among the five readers. The difference in the interday reproducibility of V_L between sites (ICC at site A, 0.60–0.65; at site B, 0.83–0.86; repeatability at site A, 725–793 mL; at site B, 489–520 mL) is likely attributed to the different source images readers at each site used to segment the lung boundary and is a limitation of the study design. Site B used proton images in which the lung boundary was clearly visible and not affected by ventilation. Site A, however, used HPHe images in which the lung boundary was less crisp and may have been completely obscured at the location of severe defects. This would require some degree of subjective estimation of the boundary location by the readers. The difference in methods between sites introduces variability in V_L, especially when defects occur at the base or the apex of the lung. Interestingly, the interreader ICC for V_L was high for both sites (Table 2). This suggests that readers at both sites arrived at similar estimates by using either type of source images. The repeatability of the V_V measurement at both sites is comparatively low in light of the higher repeatability values in V_L, perhaps because the V_D values are lower than those of V_L, which results in a relatively small dynamic range for V_V values compared with those of V_L.

Another revealing limitation of the analysis is that lung volume was noted to vary at different time points. Specifically, site B used proton images acquired only at the baseline time point to determine V_L and for normalization to calculate V_V at all subsequent time points. However, the values of V_L measured at site A (Table 3) indicated that patients have systematically different lung inflation volumes before and after the challenge (P < .05, between baseline and postchallenge), which is possibly due to obstruction of expiration and air trapping. This change in V_L would not be reflected by the single set of proton images captured at baseline in our study, which highlights the importance of acquiring proton images in conjunction with HPHe images at each examination, perhaps concurrently during a single breath hold (15), to determine V_L most accurately.

Perhaps not surprisingly, the Bland-Altman analysis revealed intersite differences in V_L, V_D, V_V, and N_D of varying magnitude depending on the measure. The largest difference was in the V_L measure, which was lower at site B by an average of 612 mL. The large V_L bias when we used proton images might be attributed to partial volume effects, particularly near the diaphragm, combined with perceived differences between positive and negative image contrast. The negative contrast of the lungs on proton images may cause the lungs to appear less inflated, and the positive contrast on HPHe images may cause an appearance of greater inflation, especially in areas of partial volume mixing. Thus, much of the bias in V_L likely depends on the inherent properties of imaging nuclei rather than site-specific effects such as the method of segmentation. Although the effects of positive and negative image contrast might be mitigated by inverting the grayscale of either proton or HPHe images, this bias may be important to note when comparing results from multiple studies and underscores the need for consistency in lung volume interpretation.

Another potential limitation of this study was the relatively narrow selection criteria, which focused on patients with both asthma and EIB. Reproducibility and robustness of the measures studied may not be as high in more heterogeneous disease processes in the general asthma population or in general obstructive lung disease. However, our results corroborate those of recently published studies (16,17) involving HPHe MR imaging of chronic obstructive pulmonary disease and cystic fibrosis. Mathew et al (16) reported an interday ICC of 0.74 for the V_V of HPHe MR imaging examinations performed 7 days apart. Kirby et al (17) reported an interreader ICC of 0.93 for V_V measured by two readers. In our results, both the intersite bias (2.91%) and CI (−4.52%, 10.3%) in V_V were relatively modest despite the differences in analysis methods, imager field strength, and coil hardware used. These recent results and our values suggest that V_V is a robust measure for multicenter studies of obstructive lung disease.

Finally, the image analysis methods used were semiautomated. Automated approaches such as k–means clustering (18) or statistical shape descriptor–based methods (19) would potentially mitigate the subjective elements of the analysis used in this study while providing substantial time savings. However, it is likely that a human observer would innately correct for image contrast and signal intensity differences between images from the two participating sites, such as those caused by different radiofrequency coils and, potentially, field strengths. Although authors of another study (20) have experimentally verified that the signal-to-noise ratio is independent of field strength in the 1.5-T and 3-T systems used in this study, differences in T2* effects were observed. A 3.3%–7.7% signal loss is expected at 3 T compared with that at 1.5 T at our acquisition parameters on the basis of published values of T2* (20,21). This small signal loss would most likely not affect a human reader, but it may be sufficiently large that a fully automated approach would require some level of intersite calibration for consistent results. Because images were both acquired and analyzed at two sites in this study, our data set would be suitable for validating and investigating the robustness of an automated approach.

In conclusion, in this study we investigated the reproducibility of HPHe MR imaging in patients with EIB. Interday and interreader agreement by using a single method were generally high, but methodologic differences between the research groups produced differences in results for measures of V_L. Notably, the choice of image used to segment the lung boundary differed at the two sites and should be controlled for in future studies. Nonetheless, despite methodologic differences, V_V appeared to be a robust metric for evaluating ventilation in patients with EIB at both sites, and our results corroborated those obtained in studies of other obstructive lung diseases. The measurement of reproducibility in this study may guide the planning and interpretation of future HPHe MR imaging studies.

Advances in Knowledge.

• • The intraclass correlation coefficients for hyperpolarized helium 3 (³He) images collected on two separate days were 0.60–0.86 for lung volume, 0.77–0.89 for ventilation defects, 0.74–0.89 for ventilated volume, and 0.77–0.82 for number of defects, indicating high agreement among readers between days.

• • The intraclass correlation coefficients for multiple readers of hyperpolarized ³He images at a single site were 0.82–0.91 for lung volume; 0.91–0.95 for ventilation defect, 0.92–0.96 for ventilated volume, and 0.91 for number of defects, indicating high agreement among readers.

• • Ventilation defects that were seen in the same location in the lung on images from both days included 19.7% of defects on baseline, 29.2% on postchallenge, and 18.6% on recovery images, respectively.

• • The average bias for ventilated volume was 2.91% for readers of hyperpolarized ³He images at the two imaging sites.

Implications for Patient Care.

• • The ventilated volume measurement derived from hyperpolarized ³He MR imaging can help quantify reproducible changes in regional lung ventilation before and after exercise challenge, as reflected by an intraclass correlation coefficient of 0.74–0.89 for ventilated volume measurements on multiple days.

• • The ventilated volume measurement can potentially serve as an imaging marker to help diagnose pulmonary disorders, make decisions about therapy intervention, and determine longitudinal progression of obstructive lung disease.