Correlation and Agreement of Handheld Spirometry with Laboratory Spirometry in Allogeneic Hematopoietic Cell Transplant Recipients

Abstract

BACKGROUND

Early detection of subclinical lung function decline may help identify allogeneic hematopoietic cell transplantation (HCT) recipients who are at increased risk for late non-infectious pulmonary complications including bronchiolitis obliterans syndrome (BOS). We evaluated the use of handheld spirometry in this population.

METHODS

Allogeneic HCT recipients enrolled in a single center observational trial performed weekly spirometry with a handheld spirometer for one year after transplantation. Participants performed pulmonary function tests in an outpatient laboratory setting at 3 time points: pre-transplant, day 80 and 1 year post-transplant. Correlation between the two methods was assessed by Pearson and Spearman correlations; agreement was assessed using Bland-Altman plots.

RESULTS

A total of 437 subjects had evaluable pulmonary function tests. Correlation for FEV₁ was r=0.954 (p<.0001) at day 80 and r=0.931 (p<.0001) at 1 year when the handheld and laboratory tests were performed within one day of each other. Correlation for handheld FEV₆ with laboratory FVC was r=0.914 (p<.0001) at day 80 and r=0.826 (p<.0001) at 1 year. The bias, or the mean difference (handheld minus laboratory) for FEV₁ at day 80 and 1 year was −0.13L (−0.63, 0.37) and −0.10L (−0.77, 0.56), respectively. FEV₆ showed greater bias at day 80 [−0.51L (−1.44, 0.42)] and 1 year [−0.40L (−1.81, 1.01)].

CONCLUSIONS

Handheld spirometry correlated well with laboratory spirometry after allogeneic HCT and may be useful for self-monitoring of patients for early identification of airflow obstruction.

INTRODUCTION

More patients are surviving after allogeneic hematopoietic cell transplantation (HCT) for the treatment of hematologic malignancies and associated disorders,¹ however late onset non-infectious pulmonary complications such as cryptogenic organizing pneumonia, interstitial pneumonitis, and bronchiolitis obliterans syndrome (BOS), a manifestation of chronic graft-versus-host disease (cGVHD) that affects up to 14% of those with extrapulmonary cGVHD, remain a significant cause of morbidity and mortality.^2-6 These complications often present at an advanced stage of disease when the underlying process is irreversible.⁷

Early detection of pulmonary function changes after HCT may identify a subset of patients at highest risk for late pulmonary complications. For instance, lung function decline at day 80 post-transplant is associated with development of cGVHD within one year and a higher risk of non-relapse mortality within five years.⁸ Detection of at risk populations may facilitate pre-emptive treatment or participation in clinical trials at a stage when the disease process may be reversible. However, frequent spirometric monitoring through a pulmonary function laboratory or an office setting can be expensive and inconvenient for the patient.

An alternative approach is self-monitoring with a handheld spirometer, which is portable, inexpensive, and easy to use, thereby permitting more frequent testing.^9-11 In recipients of lung transplantation, weekly self-monitoring with a handheld spirometer has been shown to detect BOS at an earlier stage than clinic spirometry, enabling initiation of treatment earlier in the disease course.¹² We evaluated whether this approach can be applied to an HCT population, including pediatric patients, by comparing handheld spirometry with laboratory spirometry in a cohort enrolled in a prospective study performed during the first year after allogeneic HCT.

METHODS

Study Design

Patients were enrolled in a prospective, observational, longitudinal study conducted at Fred Hutchinson Cancer Research Center (FHCRC) and the Seattle Cancer Care Alliance (SCCA) designed to follow the natural history of community-acquired respiratory virus infection and airflow decline in allogeneic HCT recipients as previously described.¹³ The institutional review board of FHCRC approved the study protocol (1587.00). Allogeneic HCT candidates undergoing transplant between December 2005 and February 2010 were eligible. Enrolled patients >/= 6 years of age performed pulmonary function and laboratory studies just prior to HCT and subsequently on a regular basis for 12 months. As part of routine transplant care, patients underwent extensive pre-transplant evaluation and were required to reside in Seattle during and after their transplantation procedure until discharge around 80 days post-transplant. Clinical follow-up after discharge was arranged at the discretion of the primary providers. Patients were offered a long-term follow-up evaluation at the SCCA at 1 year after transplant.

Clinical Data

Demographic and clinical data were prospectively collected in a research database. Conditioning regimens were categorized as myeloablative plus total body irradiation (TBI), myeloablative non-TBI, or nonmyeloablative. Underlying disease risk was categorized as low, intermediate, and high risk according to previously published criterion.¹⁴ Smoking status was categorized as current, former, never, or unknown at the time of transplant. Acute GVHD severity was scored according to an established symptoms-based scoring system.¹⁵ Chronic GVHD was categorized as present or absent according to NIH consensus guidelines.¹⁶

Spirometry Protocol

At enrollment, participants were instructed on the use of a portable electronic handheld spirometer commonly used in monitoring patients with asthma (KoKo Peak Pro-6, Ferraris Respiratory, or PiKo-6, Pulmonary Data Services, Inc.), chosen for affordability, ease of use, and small size. Patients were asked to perform handheld spirometry once prior to HCT and then on a weekly basis for one year after HCT. Before discharge at day 80, patients performed handheld spirometry in the presence of a study coordinator. Patients made 3 consecutive attempts for each test. Handheld spirometry results performed at home were recorded on a weekly symptom log and relayed to the study coordinator by mail, phone, or online survey. All standard PFTs were performed at the SCCA outpatient PFT laboratory (Sensormedics) per American Thoracic Society/European Respiratory Society guidelines¹⁷ at 3 time points as part of standard clinical management at SCCA: baseline (pre-transplant), day 80 post-HCT and 1 year post-HCT.

Analysis of Spirometry

Laboratory tests performed closest to the date of transplant at baseline, day 80 +/− 40, and 365 days +/− 120 were used for analysis. The handheld spirometry performed closest to the laboratory test and within these time windows was selected for comparison. Patients were excluded from the spirometry analysis if handheld FEV₁ or FEV₆ values were not available. Because the handheld spirometer only measured FEV1 and FEV₆, FEV₆ was used as a surrogate for FVC in the analysis. The best of 3 handheld FEV₁ and FEV₆ values for each time point was used. To reduce the impact of a time interval on the comparison of handheld with laboratory spirometry, subset analyses were performed on subjects with data that fell within the lower quartile of time interval between handheld and laboratory values at baseline, day 80, and 1 year (“Subset 1”). To simulate a screening situation in which handheld spirometry precedes laboratory spirometry for the post-transplant time period, analysis was performed on a second subset (“Subset 2”) where only post-transplant handheld data greater than the lower quartile of time interval but less than 14 days prior to the laboratory spirometry at day 80 and 1 year were considered. Subsets of pediatric subjects (<20 years old) ages 6-13 and 14-19 were also analyzed. Percent predicted values were calculated using NHANES III¹⁸ formulas as recommended by American Thoracic Society guidelines.¹⁹

Statistical Analysis

Pearson and Spearman correlation coefficients were calculated and linear regression plots were used to compare handheld with laboratory values. Agreement between the two techniques was illustrated using Bland-Altman plots, which display the difference between the handheld and laboratory values plotted against the mean value of the two test values. Bias, the degree to which the methods differ from one another, was defined as the mean difference between measures and associated 95% confidence intervals (CI) of the bias (mean difference +/− 1.96 multiplied by the standard error of the mean). Bland-Altman plots are accompanied by upper and lower limits of agreement, which reflect the variation of bias within a sample and are defined by the 95% CI for the differences (mean difference +/−1.96 multiplied by the standard deviation of the difference).²⁰ All statistical analyses were performed using SAS 9.3 for Windows (SAS Institute, Inc., Cary, NC).

RESULTS

Study Cohort

The total number of patients enrolled was 571 of whom 437 had PFTs available for analysis (Figure 1). Baseline characteristics are shown in Table 1. Pediatric subjects represented 10% (43/437) of the total cohort. Of the 437 subjects, 38 (9%) had died by day 80 and 147 (34%) by 1 year. The mean duration of follow-up was 207 days (+/−136.6 days). Sixty percent (262/437) completed laboratory spirometry at all three specified time points. The mean number of handheld spirometry studies reported per subject was 20 (+/−16.8). The proportion of patients still alive and free of clinical relapse with complete laboratory and handheld spirometry tests at baseline was 80% (349/437), 98% (355/363) at day 80, and 67% (178/265) at 1 year.

Figure 1.

Study cohort and subsets at three time points in relation to allogeneic HCT.

Table 1.

Baseline characteristics of cohort, n=437

Characterstic	Category	N (%)
Age (IQR)	Median (IQR)	51.4 (38.8-60.5)
Age group	<20	43 (10%)
	20-39	77 (18%)
	40-59	204 (47%)
	>=60	113 (26%)
Race	Caucasian	355 (81%)
	Non-Caucasian	82 (19%)
Gender	Female	161 (37%)
	Male	276 (63%)
Underlying disease risk	High	163 (37%)
	Intermediate	36 (8%)
	Low	238 (54%)
Donor relationship	Related	181 (41%)
	Unrelated	256 (59%)
Cell source	Bone marrow	74 (17%)
	Cord blood	34 (8%)
	Peripheral blood	329 (75%)
Conditioning	Myeloablative+TBI	116 (27%)
	Myeloablative+non-TBI	134 (31%)
	Non-Myeloablative	187 (43%)
Acute GVHD (grade)	0-1	116 (27%)
	2	264 (60%)
	3-4	57 (13%)
Chronic GVHD	No	227 (52%)
	Yes	210 (48%)
Smoking status	Current	28 (6%)
	Former	122 (28%)
	Never	270 (62%)
	Unknown+Missing	17 (4%)
Pre-transplant Lung Function (Baseline)	Median FEV1, % predicted (IQR)	84 (77-93), n=419
	Median FEV1/FVC (IQR)	0.76 (0.72-0.81)

Comparison of Handheld Spirometry with Laboratory PFT

Comparison of handheld with laboratory spirometry parameters showed a linear relationship at all three time points which was closest to the line of equality at day 80 and 1 year for most of the spirometric parameters analyzed (FEV₁, FEV₆ and FEV₁/FEV₆, Figure 2 and Figure S1). Overall correlation for the total cohort was highest for FEV₁: at baseline, day 80 and 1 year, r=0.722, r=0.923, and r=0.748 respectively. Coefficients for FEV₆ were r=0.679, r=0.887, and r=0.783, respectively, (p<.0001 for all). Spearman coefficients were higher than Pearson coefficients for all parameters except FEV₁ at day 80 (r=0.921, p<.0001) (Table 2). The bias, or the mean difference (handheld minus laboratory) in FEV₁ in liters for baseline, day 80 and 1 year was −0.24 (95% confidence interval [CI] −0.32, −0.17), −0.19 [−0.23, −0.15], and −0.08[−0.19, 0.03].

Figure 2.

Handheld FEV₁ versus Laboratory FEV₁ at baseline, day 80 and 1 year for the total cohort and Subset 1. Relationship of FEV₁ measurements between handheld spirometry (Y-axis) and laboratory PFT (X-axis), with line of regression (___), 95% confidence intervals (---), and line of equality (…) which indicates perfect agreement.

Table 2.

Pearson correlation, Spearman correlation, bias, and limits of agreement of handheld spirometry compared with laboratory spirometry.

Parameter	Time Point	Cohort	Pearson coefficient	Spearman coefficient	Bias (95% CI), L	Limits of agreement, L
FEV1	Baseline	Total	0.722	0.820	−0.24(−0.32, −0.17)	−1.64, 1.15
		Subset 1	0.592	0.822	−0.16(−0.34, 0.02)	−1.91, 1.59
	d80	Total	0.923	0.921	−0.19(−0.23, −0.15)	−0.9, 0.5
		Subset 1	0.954	0.928	−0.13(−0.19, −0.08)	−0.63, 0.37
		Subset 2	0.916	0.917	−0.21(−0.26, 0.17)	−0.95, 0.52
	1yr	Total	0.748	0.895	−0.08(−0.19, 0.03)	−1.53, 1.37
		Subset 1	0.931	0.932	−0.10(−0.19, −0.02)	−0.77, 0.56
		Subset 2	0.566	0.840	−0.03(−0.29, 0.24)	−2.2, 2.15
FEV6 vs FVC	Baseline	Total	0.679	0.784	−0.41(−0.52, −0.31)	−2.33, 1.5
		Subset 1	0.612	0.740	−0.34( -0.55, −0.13)	−2.38, 1.7
	d80	Total	0.887	0.901	−0.57(−0.63,-0.51)	−1.65, 0.51
		Subset 1	0.914	0.911	−0.51(−0.61, −0.42)	−1.44, 0.42
		Subset 2	0.879	0.897	−0.6(−0.67, −0.52)	−1.7, 0.51
	1yr	Total	0.783	0.853	−0.40(−0.52, −0.28)	−1.97, 1.17
		Subset 1	0.826	0.917	−0.40(−0.58, −0.22)	−1.81, 1.01
		Subset 2	0.750	0.780	−0.40(−0.61, −0.19)	−2.15, 1.35
FEV1/FEV6 vs FEV1/FVC	Baseline	Total	0.282	0.473	0.02(0.01, 0.03)	−0.22, 0.26
		Subset 1	0.320	0.393	0.03(0.01, 0.05)	−0.18, 0.23
	d80	Total	0.568	0.636	0.06(0.06, 0.07)	−0.09,0.21
		Subset 1	0.710	0.669	0.07(0.06, 0.08)	−0.05, 0.18
		Subset 2	0.512	0.619	0.06(0.05, 0.07)	−0.1, 0.22
	1yr	Total	0.329	0.480	0.05(0.03, 0.07)	−0.22, 0.31
		Subset 1	0.724	0.709	0.05(0.04. 0.07)	−0.07, 0.18
		Subset 2	0.175*	0.373**	0.06(0.02, 0.1)	−0.25, 0.36

“Total” refers to the cohort which had handheld tests performed closest to the date of transplant at baseline, +/− 40 days at day 80, and +/− 120 days at 1 year were used for comparison with laboratory PFT values. Subset 1 includes subjects with spirometry data that fell within the lower quartile of time interval between handheld and laboratory values at baseline (pre-transplant), day 80, and 1 year. Subset 2 includes subjects with handheld spirometry performed >1 day and </=14 days prior to the laboratory spirometry. P-values are <.0001 for all Pearson and Spearman coefficients reported except as indicated.

^*P=0.156.

^**P=0.0019.

Subset Analysis

The time interval for Subset 1 was 15 days at pre-transplant baseline (n=95), 1 day for day 80 (n=93), and 1 day for 1 year (n=62) (Table 3). Pearson and Spearman coefficients for all parameters were higher for Subset 1 compared with the total cohort at day 80 and 1 year except FEV₁ and FEV₆ at pre-transplant baseline (Figure 2, Table 2). At day 80 and 1 year, Pearson coefficients for FEV₁ for Subset 1 were 0.954 and 0.931, respectively.

Table 3.

Cohort and subsets by time period in relation to HCT

Time period in relation to HCT	Total cohort, n	Median Interval in days (IQR)	Subset 1, n	Subset 2, n
Baseline	349	20 (15,29)	95	NA
Day 80 post	355	4 (1,7)	93	243
1 year post	178	6 (1, 16)	62	123

Subjects with handheld and laboratory spirometry performed pre-transplant (baseline) +/− 40 days at day 80, and +/− 120 days at 1 year were included. Subset 1 includes subjects with spirometry data that fell within the lower quartile of time interval between handheld and laboratory values at baseline, day 80, and 1 year. Subset 2 includes subjects with handheld spirometry that was performed >1 day and <14 days prior to laboratory spirometry. NA=not applicable.

Agreement for FEV₁ and FEV₆ was closer for Subset 1 compared with the total cohort at baseline and day 80 and slightly wider or unchanged at 1 year. At day 80 and 1 year, the bias in FEV₁ was −0.13L [CI: −0.19, −0.08] and −0.1L [−0.19, −0.2], respectively. For FEV₆, the bias at day 80 and 1 year was −0.51L [−0.61, −0.42] and −0.4L [−0.58, −0.22], respectively. Bland-Altman analysis revealed that the handheld spirometric measurements were consistently lower than laboratory measurements (Figure 3).

Figure 3.

Bland-Altman plots of Subset 1 comparing handheld measurements of FEV₁ and FEV₆ with laboratory measurements of FEV₁ and FVC at day 80 and 1 year, showing the mean difference (handheld minus laboratory) between techniques (center –) and upper and lower limits of agreement (mean difference +/− SD, upper and lower ---).

For Subset 2, subjects with handheld spirometry performed >1 day and <14 days prior to the laboratory spirometry, the Pearson and Spearman correlations for FEV₁ and FEV₆ were similar to those of Subset 1 at day 80 and 1 year (Figure S2). The degree of agreement for FEV₁ was less at day 80 compared with Subset 1, but similar at 1 year. Agreement of handheld FEV₆ to laboratory FVC was similar to Subset 1. The limits of agreement for Subset 2 were consistently wider than limits of agreement for Subset 1 for all parameters analyzed (Table 2).

Pediatric subjects were divided into a younger (ages 6-13, n=18) and older subset (ages 14-19, n=25). The number of subjects with spirometry available for analysis at baseline and 1 year was small and correlation for all parameters at these time points was low. At day 80, correlation of FEV₁ was r=0.969 (p<.0001) in the younger cohort and 0.882 (p<0.0001) in the older cohort; FEV₆ was 0.985 (p<.0001) and 0.792 (p=.0003), respectively (Table S1). Analysis of agreement was not performed due to the low number of subjects.

DISCUSSION

In this large prospective cohort of allogeneic HCT recipients, spirometric parameters from an inexpensive and commercially available portable handheld spirometer correlated well with standard laboratory spirometry with a high degree of agreement for FEV₁. These results suggest that frequent monitoring of lung function with a handheld spirometer may be a feasible alternative to laboratory-based testing for detecting clinically significant airflow decline after allogeneic HCT.

Early detection of pulmonary dysfunction remains a challenge in managing late-onset pulmonary complications after HCT. There is increasing recognition that the risk of mortality has already risen by the time patients become symptomatic,²¹ and the “window of opportunity” to intervene and improve outcomes is during the pre-clinical phase of disease.⁷ Now that there is prospective evidence that medical intervention early after diagnosis of BOS may improve lung function, early detection is actionable.^{22, 23} Pulmonary function testing is the primary diagnostic tool for identification of patients with BOS.²⁴ However, because NIH consensus recommendations to perform PFT testing every 3 months after HCT up to 2 years^{25, 26} are difficult for many HCT programs to implement due to cost and inconvenience to the patient, many months may elapse between tests, thereby missing pre-clinical changes that represent early lung function decline. Therefore, self-monitoring with an inexpensive handheld device represents a potentially attractive means of improving early detection.

Previous comparisons between handheld devices, desktop office spirometers, and standard laboratory-based spirometers have yielded mixed results.^{27, 28} Additionally, comparative studies between portable handheld devices and laboratory equipment have been limited to small numbers of patients¹² or healthy volunteers.^{29, 30} Our study is the first analysis of the performance of portable handheld devices in a large cohort of immunocompromised patients, including pediatric subjects, outside of lung transplant recipients. Our data indicate that the handheld FEV₁ met the American Thoracic Society recommended limits of acceptable bias for spirometric measurements of +/− 100 mL³¹ at 1 year and approached these limits at day 80.

The most reliable measure of airflow decline, FEV₁, appears to be representative of the patient’s lung function at that point in time when measured using the handheld device. As one might expect, correlation was strongest when there were fewer days between tests, as shown for FEV₁, followed by FEV₆, at day 80 and 1 year for Subset 1 compared with Subset 2. The weaker correlation at baseline was likely due to the longer time interval between pre-transplant evaluation PFT and the first spirometry performed on the day of enrollment into the trial. There may also be a training effect as reflected by the tighter correlation at day 80 and 1 year.

While FEV₁ (and to a lesser extent FEV₆) measured by both methods are closely related, the negative bias shows that handheld spirometry tended to underestimate the laboratory value. The bias toward lower FEV₁ may result in overdiagnosis, but this is acceptable and preferable to missed diagnoses given the significant morbidity and mortality of post-transplant complications. Some basic inexpensive handheld spirometers, such as the ones used in this study, measure only FEV₆, which is the maximal volume exhaled in 6 seconds, whereas FVC is the maximal volume exhaled to the end of a forced expiratory effort which is usually greater than 6 seconds. Although not routinely used for interpretation by most practitioners, FEV₆ is recorded by standard laboratory spirometers provided that the patient exhales to 6 seconds. FEV₆ is considered is an acceptable surrogate for FVC³² and has been validated in patients with obstructive lung disease.^33-35 Recording FEV₆ is considered to be more reproducible and less physically demanding than FVC,¹⁷ although these were not the specific reasons that we used FEV₆ in our study. In our study FEV₆ agreed with FVC less closely than FEV₁ measurements, which might be expected since the expiratory time of FEV₆ is shorter than FVC.

The limits of agreement for all parameters tested were wider than the 350 cc for FEV₁ and 500 cc for FVC as suggested by Liistro et al,²⁸ which is likely a reflection of the time interval between the studies. The limits of agreement for Subset 1 were narrower than Subset 2 for every parameter and time point analyzed, suggesting that the limits of agreement would improve if the handheld and laboratory tests had been performed on the same day. Because the study protocol was not designed to compare the two methods directly, which ideally would be performed sequentially in the same setting, significant bias and variation in agreement is inherent to this analysis.

Thus, handheld spirometry can be used as a screening tool to monitor for subclinical changes that would require further evaluation, rather than as a substitute for laboratory PFTs. Serial decline in FEV₁ by handheld spirometry should be followed promptly by standard PFT testing. This strategy was tested in a small prospective trial of post-HCT recipients in which 11 of 37 patients were diagnosed with late-onset noninfectious pulmonary complications, prompted by changes detected by home spirometry.¹⁰

As mentioned previously, this study was limited by the trial protocol, which was not designed to prospectively evaluate spirometric performance. The analysis of spirometry was also limited by decreased compliance with handheld spirometry after day 80, which may have been due to other study requirements, transfer of patients to home, and lack of motivation. However, the conditions of this analysis reflected those of a real-world clinical setting, in which HCT recipients are discharged from the transplant service approximately 3 months after HCT and subsequently follow-up only as needed until the first year follow-up appointment. Given the multi-system nature of HCT complications, performing spirometry consistently on a weekly basis when free of respiratory symptoms may remain a challenge for some patients. If screening handheld spirometry is implemented in this population, strategies to improve compliance including patient education and monitoring will be required.

In conclusion, this study establishes handheld spirometry as a valid method of longitudinal monitoring of lung function after HCT. These findings represent a critical step towards resolving the unmet need of a reliable screening methodology for BOS and other late pulmonary complications in the HCT population,⁷ and support the use of a convenient and inexpensive handheld spirometer to identify HCT recipients eligible for early intervention of airflow decline in a prospective clinical trial.

Supplementary Material

1

Figure S1. Handheld FEV₆ and FEV₁/FEV₆ versus laboratory FVC and FEV₁/FVC at three time points for Subset 1. Relationship of FEV₆ measurements between handheld spirometry (Y-axis) and laboratory PFT (X-axis), with line of regression (___), 95% confidence intervals (---), and line of equality (…) which indicates perfect agreement.

2

Figure S2. Handheld FEV₁ versus laboratory FEV₁ at day 80 and 1 year for Subset 2 which includes subjects with handheld spirometry performed >1 day <14 days from laboratory spirometry.

Highlights.

Ms. No. YBBMT-D-15-00566

Correlation and Agreement of Handheld Spirometry with Laboratory Spirometry in Allogeneic Hematopoietic Cell Transplant

• Frequent lung function monitoring after HCT is expensive and inconvenient

• FEV₁ from a portable handheld device correlated and agreed with laboratory FEV₁

• Handheld spirometers can be used to screen for lung function decline after HCT

ABBREVIATIONS LIST

BOS

bronchiolitis obliterans syndrome

cGVHD

chronic graft-versus-host-disease

CI

confidence interval

HCT

hematopoietic cell transplantation

PFT

pulmonary function test

TBI

total body irradiation