Noncontact spirometry with a webcam

Abstract.

We present an imaging-based method for noncontact spirometry. The method tracks the subtle respiratory-induced shoulder movement of a subject, builds a calibration curve, and determines the flow-volume spirometry curve and vital respiratory parameters, including forced expiratory volume in the first second, forced vital capacity, and peak expiratory flow rate. We validate the accuracy of the method by comparing the data with those simultaneously recorded with a gold standard reference method and examine the reliability of the noncontact spirometry with a pilot study including 16 subjects. This work demonstrates that the noncontact method can provide accurate and reliable spirometry tests with a webcam. Compared to the traditional spirometers, the present noncontact spirometry does not require using a spirometer, breathing into a mouthpiece, or wearing a nose clip, thus making spirometry test more easily accessible for the growing population of asthma and chronic obstructive pulmonary diseases.

1. Introduction

Asthma and chronic obstructive pulmonary disease (COPD) are the most prevalent obstructive airway diseases, affecting tens of millions of people in the United States alone.¹ The most common way to diagnose the diseases and reassess the progression of the diseases is spirometry, which measures how much a patient inhales and how fast he/she exhales. In a spirometry test, the patient is instructed to exhale rapidly and forcefully into a mouthpiece connected to a spirometer that measures breath flow rate and volume. To ensure that all the air is inhaled into the spirometer for accurate flow measurement, the patient is also instructed to wear a nose clip, which leads to complaints of discomfort. For good hygiene, a disposal mouthpiece is used for each spirometry test. The needs of the spirometer, nose clip, and mouthpiece contribute to factors that prevent widespread use of spirometry at home. To overcome this barrier, here we describe an imaging-based noncontact spirometry using a webcam.

There are increasing efforts in developing noncontact respiratory monitoring methods. Depending on the monitoring principles, these methods can be divided into three categories: thermal, photoplethysmography (PPG), and body movement detections. The first category measures the air temperature change associated with exhaled breath near the mouth and nose regions of a subject using an infrared imaging system.^2,3 The temperature change can be also detected via the pyroelectric effect.⁴

The second category extracts the respiratory signal embedded in the PPG signals. PPG measures the change of light absorption or reflection induced by the change of blood volume with each pulse. The movement of the thoracic cavity affects the blood flow during breathing, which leads to a modulation in the PPG signal by the respiratory activity.⁵ Several PPG signal processing methods, including independent component analysis,⁶ principal component analysis,⁵ digital filtering,⁷ and variable frequency complex demodulation⁸ have been proposed to remove noise in PPG and extract respiration-induced modulation in PPG.⁹

The third category detects the subtle chest movement induced by breathing with different technologies, such as frequency-modulated radar wave^10,11 and ultrawideband impulse radio radar.¹² Several optical imaging-based methods have been introduced to monitor respiratory activities. For example, our group¹³ developed a differential method to track the shoulder movement associated with breathing. Reyes et al.¹⁴ measured the intensity change of the chest wall. Lin et al. applied an optical flow algorithm to detect respiratory activities.¹⁵ These methods used low-cost cameras. More sophisticated three-dimensional imaging with multicamera and projector-camera setups were used to track chest surface deformation during breathing.^16,17

The works summarized above demonstrated noncontact monitoring of respiratory activities, but to date a noncontact spirometry has not been reported. To achieve this capability, one must be able to accurately and quickly measure respiratory cycles over an extremely wide flow rate range and validate the results with real subjects. In this paper, we describe an imaging-based method to perform spirometry, which can track the flow-volume (spirometry) curve and extract important parameters, including forced expiratory volume in the first second (FEV1), forced vital capacity (FVC), and peak expiratory flow rate (PEF). These parameters are critical for the diagnosis and management of asthma and COPD. We compare and calibrate the imaging-based noncontact spirometry with the simultaneously performed traditional spirometry and validate the results with a pilot study including 16 subjects.

2. Method

The imaging-based spirometry first captures the image of a subject’s upper body, including face and shoulders, during inhalation and exhalation with a webcam (Fig. 1). Then, it analyzes the shoulder movement and determines the spirometry exhalation rate versus exhalation volume curve, FEV1, FVC, and PEF. To validate and calibrate the method, this work includes a simultaneous spirometry test of each subject with a traditional spirometer (gold standard reference) [Fig. 1(a)]. The data from the imaging-based and traditional spirometry tests are compared and used to construct a calibration curve for the subject. From the calibration curve, the exhalation volume is determined from the shoulder movement, and the corresponding exhalation rate is obtained from the time derivative of the exhalation volume.

Fig. 1.

Principle of noncontact spirometry. (a) Experiment setup. (b) Video recording. (c) and (d) Video analysis. (e) Signal analysis. (f) Flow-volume (spirometry) curve.

2.1. Data Acquisition

A web camera (Logitech C905) was used to capture the video of a subject’s upper body under a typical indoor ambient light condition. The subject was instructed to sit on a backrest chair at the distance of 90 cm from the camera and to perform a forced spirometry test using a gold standard commercial spirometer (MicroLoop, Carefusion), during which both the video and spirometry data were recorded synchronously with a laptop computer [Fig. 1(a)]. The frame rate of the camera was set at 30 frames per second (fps), and the spatial resolution of frame was $960\times 720\text{\hspace{0.17em}\hspace{0.17em}pixels}$. The commercial spirometer complies with ATS/ERS 2005 standards, and its sampling rate is $>100\mathrm{Hz}$.

Sixteen subjects were enrolled in an Institutional Review Board (IRB) study approved by Arizona State University (No. STUDY00004548). The subjects included different genders (nine males, seven females), ages ($28.1\pm 3.2$ years old, $\text{mean}\pm \mathrm{SD}$), body mass index ($22.5\pm 3.6$, $\text{mean}\pm \mathrm{SD}$), and heights ($1.71\pm 0.09\mathrm{m}$, $\text{mean}\pm \mathrm{SD}$).

Following the standard forced spirometry test procedure, the subject was asked to wear a nose clip, inhale as deeply as they can, and then exhale into the mouthpiece attached to the spirometer as hard as possible and for as long as possible. In each test, the subject performed six forced breathing cycles continuously, in which three forced cycles were used to build calibration curves while the others were used for validation.

2.2. Video Processing

Two shoulder regions consisting of $50\times 50\text{\hspace{0.17em}\hspace{0.17em}pixels}$ each were selected for detecting respiratory related movement. The regions included the middle portions of the shoulders with clear boundaries that separated the body and background. The upper body movement of the subject was tracked with the Kanade–Lucas–Tomasi (KLT) tracker in the defined region of interest (ROI)^18–20 during spirometry test. A Harris corner detector was used to detect feature points within the ROI of the shoulders.¹⁹ The detector computes the spatial variation ($E$) of image intensity in all directions, with

$$E(u,v)=[\begin{array}{cc}u& v\end{array}]\left[\begin{array}{cc}⟨I_x^2⟩& ⟨I_x{I}_y⟩\\ ⟨I_x{I}_y⟩& ⟨I_y^2⟩\end{array}\right]\left[\begin{array}{c}u\\ v\end{array}\right],$$ (1)

where $I_x$, $I_y$ are the gradients of the image intensity of the feature point in the $x$- and $y$-directions, $u$, $v$ are the numbers of pixels shifted from each point in the image in the $x$- and $y$-directions, and the angle brackets $⟨⟩$ denote averaging (over $u$, $v$). The matrix in Eq. (1) is the Harris matrix. The points, which have large eigenvalues for both $x$- and $y$-directions in the Harris matrix, were defined as feature points.

To track the feature points frame by frame, affine transformation was used in the adjacent frames for ROI registration. In general, an affine transformation is composed of rotation, translation, scaling, and skewing. Considering two patches of an image in adjacent frames, an affine map $f$ acting on patch $x$ is represented as

$$f(x)=\mathbf{A}x+\mathbf{b},$$ (2)

where $\mathbf{A}$ is the deformation matrix and $\mathbf{b}$ is the translation vector. The transformation parameters can be determined in a closed form when minimizing the dissimilarity measure $ϵ$. An example of the dissimilarity measure is the sum of squared difference, given by

$$ϵ={\iint }_w{[J(\mathbf{A}x+\mathbf{b})-I(x)]}^{2}w(x)\mathrm{d}x,$$ (3)

where $w$ is a weighting function.

2.3. Signal Analysis

To calibrate the shoulder movement, we counted the number of pixels of a certain feature in the image frame and related it to the actual physical length of the feature. The conversion factor $\alpha$ is defined as

$$\alpha =\frac{\text{feature}\mathrm{\backslash \_}\text{length}(\mathrm{mm})}{\text{feature}\mathrm{\backslash \_}\text{pixel}\mathrm{\backslash \_}\text{number}(\text{pixel})}.$$ (4)

In the $i$’th frame, the vertical component of the feature points, representing the shoulder position pos, due to respiration, is given by

$$\mathrm{pos}(i)=\alpha \frac{1}{n}\sum _{j=1}^n{y}_j(i),$$ (5)

where $y_j(i)$ is the vertical component of the $j$’th point in the $i$’th frame, $n$ is the total frame number of the video.

To correlate the shoulder displacement (change in position) with the spirometer reading, a calibration curve was built for each subject, which converted the shoulder displacement to breathing volume in forced spirometry (Fig. 2). The exhaled volume data from three randomly selected forced breathing cycles of each subject were fitted with a fifth-order polynomial. Note that the sampling rate of spirometer ($>100\mathrm{Hz}$) was higher than the web camera (30 Hz). To match the spirometer reading and camera output, we down sampled volume data from the spirometer for close comparison with the measurement of our optical system. The flow rate was determined from time derivative of the breathing volume.

Fig. 2.

Calibration curves (a) to (p) for different subjects. The dotted lines are obtained from three forced breathing cycles, and the blue lines are fifth-order polynomial fitting curves. The sum of squares due to error (SSE) indicators are of the fitting quality (see text for more details).

3. Results

3.1. Calibration Curves

In each test, the subject performed six forced breathing cycles continuously. We randomly selected three forced cycles to build a calibration curve, and the other three cycles were used for validation for each subject. Figure 2 shows these calibration curves for different subjects. Despite the variability in the calibration curves for different subjects, each could be fitted with a fifth-order polynomial using summed square of residuals (${\mathrm{SS}}_{\mathrm{res}}$) and $R$-square ($R^2$) to evaluate the goodness of fit. ${\mathrm{SS}}_{\mathrm{res}}$ is defined as

$${\mathrm{SS}}_{\mathrm{res}}=\sum _{i=1}^n{(y_i-f_i)}^2,$$ (6)

where $y_i$ is the exhaled volume from the images, $f_i$ is the fitting function (fifth-order polynomial), and $R$-squared is

$$R^2=1-\sum _{i=1}^n{(y_i-f_i)}^2/\sum _{i=1}^n{(y_i-\overline{y})}^2,$$ (7)

where $\overline{y}$ is the average volume and $\sum _{i=1}^n{(y_i-\overline{y})}^2$ is the total variance of the data. ${\mathrm{SS}}_{\mathrm{res}}$ measures the variance of the fitting model, and $R^2$ describes how close the data are to the fitted curve. The ${\mathrm{SS}}_{\mathrm{res}}$ ranges from 0.11 to 7.30, with the median value of 1.10, indicating small errors between the exhaled volume using our proposed method and the fitting model in most cases. $R^2$ ranges from 0.90 to 0.99, indicating good fitting quality.

3.2. Flow-Volume Curves and Vital Respiratory Parameters

Using the calibration curves obtained above, we converted the remaining three breathing cycles into flow-volume curves (blue lines) for each subject. This flow-volume is one of the most important spirometry test results that is provided by most commercial spirometers (Fig. 3). For comparison, the simultaneously recorded curves from the commercial spirometer are also shown in Fig. 3 (red lines). Frechet distance (FD) and Pearson product-moment correlation coefficient (Pearson’s $r$) were used to measure the similarity between the imaging-based spirometry curves and the gold standard spirometry curves. Given two curves $f:[a,b]\to V$ and $g:[a,b]\to V$, FD is defined as

$$F(f,g)=\underset{\alpha ,\beta t\in [\mathrm{0,1}]}{\inf \max }\{d(f[\alpha (t)],g[\beta (t)])\},$$ (8)

where $\alpha$ (resp. $\beta$) is an arbitrary continuous nondecreasing function from [0,1] onto $[a,b]$ (resp. $[a^{\prime },b^{\prime }]$).

Fig. 3.

Flow-volume curves (a) to (p) for different subjects. The red curves are from the gold standard spirometer, and blue curves are from the imaging-based method. $r$ is a Pearson product-moment correlation coefficient, and “FD” measures the distance between the calibrated curve and spirometer result.

Pearson’s $r$ measures the linear dependence between the imaging-based results and the gold standard spirometry results. As shown in Fig. 3, the Pearson’s $r$ values ranged from 0.89 to 0.99, and FD was found to range from 0.76 to 3.32, indicating good agreements between the imaging-based and traditional spirometer curves for most subjects.

Vital parameters, FEV1, FVC, and PEF, were calculated from the imaging-based spirometry curves (see Table 1 and Fig. 4.). Figure 4 shows that the results measured using the imaging-based method are in excellent agreement with those by the gold standard spirometer. To quantify the agreement, the Pearson $r$, root-mean-square-error (RMSE), relative error, and paired sample $t$-test were analyzed. The Pearson $r$ values for FEV1, FVC, and PEF are 0.95, 0.98, and 0.97, respectively, indicating good linear correlation between the imaging-based and traditional spirometers. The RMSE values of FEV1, FVC, and PEF are 0.27, 0.18, and 0.56 respectively, indicating small differences between the two methods. The average errors of FEV1, FVC, and PEF are 8.5%, 6.9%, and 7.7%, respectively. In the paired $t$-test, FEV1 and FVC show the pairwise difference between the imaging-based spirometry and the gold standard spirometry with a mean of zero at the 5% significance level, which is consistent with the RMSE values.

Table 1.

FEV1, FVC, and PEF values for three forced breathing cycles $(\text{mean}\pm \sigma )$.

Subject	FEV1 (L)		FVC (L)		PEF (L/s)
	Estimated	Gold	Estimated	Gold	Estimated	Gold
1	$3.69\pm 0.06$	$3.43\pm 0.04$	$4.34\pm 0.21$	$4.63\pm 0.06$	$9.07\pm 0.32$	$8.42\pm 0.31$
2	$2.46\pm 0.12$	$2.50\pm 0.01$	$2.55\pm 0.12$	$2.59\pm 0.04$	$5.03\pm 0.65$	$4.86\pm 0.07$
3	$3.11\pm 0.12$	$3.09\pm 0.04$	$3.25\pm 0.08$	$3.12\pm 0.06$	$6.55\pm 0.57$	$6.86\pm 0.11$
4	$2.14\pm 0.33$	$2.38\pm 0.03$	$2.42\pm 0.07$	$2.69\pm 0.04$	$5.10\pm 0.57$	$4.80\pm 0.03$
5	$3.67\pm 0.14$	$3.84\pm 0.14$	$3.98\pm 0.07$	$4.03\pm 0.03$	$8.16\pm 0.40$	$7.80\pm 0.12$
6	$2.59\pm 0.19$	$2.83\pm 0.06$	$3.14\pm 0.22$	$3.14\pm 0.03$	$3.01\pm 0.20$	$3.07\pm 0.16$
7	$2.62\pm 0.28$	$3.04\pm 0.01$	$3.32\pm 0.12$	$3.31\pm 0.03$	$5.01\pm 0.13$	$5.14\pm 0.13$
8	$3.37\pm 0.09$	$3.73\pm 0.02$	$3.80\pm 0.09$	$3.95\pm 0.15$	$9.71\pm 0.36$	$9.40\pm 0.30$
9	$4.46\pm 0.10$	$4.47\pm 0.16$	$4.86\pm 0.22$	$4.85\pm 0.04$	$10.46\pm 0.34$	$10.16\pm 0.95$
10	$3.73\pm 0.12$	$3.30\pm 0.02$	$4.12\pm 0.16$	$4.11\pm 0.07$	$7.30\pm 1.07$	$7.02\pm 0.21$
11	$2.74\pm 0.05$	$2.75\pm 0.07$	$3.01\pm 0.09$	$2.85\pm 0.02$	$5.90\pm 0.55$	$5.81\pm 0.11$
12	$2.76\pm 0.09$	$2.86\pm 0.06$	$3.80\pm 0.26$	$3.71\pm 0.04$	$5.67\pm 0.49$	$6.00\pm 0.44$
13	$3.78\pm 0.08$	$3.81\pm 0.02$	$4.21\pm 0.18$	$4.14\pm 0.02$	$8.67\pm 0.32$	$8.40\pm 0.17$
14	$4.88\pm 0.12$	$4.46\pm 0.05$	$5.07\pm 0.10$	$5.24\pm 0.01$	$7.55\pm 0.49$	$7.60\pm 0.37$
15	$3.07\pm 0.13$	$3.28\pm 0.10$	$3.23\pm 0.10$	$3.57\pm 0.01$	$8.06\pm 0.18$	$7.49\pm 0.38$
16	$4.54\pm 0.24$	$4.52\pm 0.05$	$4.90\pm 0.07$	$5.10\pm 0.01$	$8.92\pm 0.08$	$8.93\pm 0.45$

Fig. 4.

Comparison of (a) FEV1, (b) FVC, and (c) PEF obtained with the imaging-based and gold standard spirometry for different subjects. The values are averaged over three breathing cycles, and the error bars are standard deviations of the measurements.

4. Discussion

Respiration induces craniocaudal, anteroposterior displacements, and cross-sectional area changes of rib cage and abdomen,²¹ which has been studied by analyzing the three-dimensional movements of the body.²² In this work, we have shown that accurate spirometry can be obtained from the shoulder displacement alone without complex three-dimensional chest movement measurement. This finding is consistent with a previous study that shows good correlation between vertical body movement measured by magnetic resonance imaging and spirometry.²³ To convert the displacement into volume, if we could simply model the lung as a cube or sphere, a third-degree polynomial fitting would suffice. However, the human lung is a complicated system, in which bronchi and alveoli have different shapes. Empirically, we found that a fifth-degree polynomial could fit the breathing volume well. Although a higher degree could be even more accurate, it brings up the over-fitting issue. In a future study, we could explore a general calibration to make the system easier for home use.

In the imaging-based spirometry, accurate tracking of the shoulder displacement is critical. In this work, we used the KLT tracking algorithm, which requires good contrast of the shoulder image. This means that the subject’s clothes must have a substantially different color from the background color. We found that this requirement was not difficult to meet for all the sixteen subjects who wore different clothes with different colors and patterns (Fig. 5). In addition to KLT, other algorithms^24–26 may also be used to track motions.

Fig. 5.

Snapshots of the videos in the dataset. The subjects wore clothes with various colors and patterns.

Subjects with different genders, ages, body mass indices, and heights were included in this small pilot test. We found that height is moderately correlated with FVC. The Pearson correlation between height and FVC is 0.69. The correlation between body mass index and FVC is 0.50, which showed weak correlation. For the two gender groups, we found a significant difference ($p<0.004$) between male ($4.63\pm 0.69\mathrm{L}$, $\text{mean}\pm \mathrm{SD}$) and female subjects ($3.39\pm 0.72\mathrm{L}$, $\text{mean}\pm \mathrm{SD}$) in FVC. More subjects with diverse profiles will be needed to validate these observations.

The spirometry curves and extracted respiratory parameters provide key information about asthma, COPD, and other respiratory conditions. For example, a concave upward pattern is often observed in asthma or COPD patients.²⁷ Certain distinct patterns are linked to the obstructing lesions in the upper airway and variable intrathoracic obstructions. However, all the subjects enrolled in our current study are healthy adults. In future study, we will include subjects with respiratory diseases. Another direction of further improvement of the current work is to use a faster camera. The current work used a camera with a frame rate of 30 fps, and the faster camera will improve the temporal resolution and is expected to lead to more accurate measurement of the exhalation rate.

5. Conclusion

We have demonstrated an imaging-based spirometer for accurate forced spirometry tests with a webcam. We validated the technology with a pilot study including sixteen subjects and compared the data with simultaneously performed spirometry tests with a gold standard commercial spirometer. The spirometry curves and key respiratory parameters, including FEV1, FVC, and PEF, from the imaging-based approach are in excellent agreement with those by the gold standard technology. The imaging-based spirometer does not require acquiring a spirometer, using a mouthpiece, or wearing a nose clip, which will lower the cost and improve the user experience, thus contributing to the diagnosis and management of the large and growing asthma and COPD populations.

Biographies

Chenbin Liu received his BA degree in electronic information engineering from Hunan University, Changsha, and his PhD in biomedical engineering from Zhejiang University, Hangzhou. He is a postdoc researcher in Biodesign Center for Bioelectronics and Biosensors, Arizona State University. His research interests include using image processing and machine learning method to develop mobile health and computer-aided diagnosis system.

Yuting Yang received her BA degree in biomedical engineering from Sichuan University, Chengdu, and her PhD in biomedical engineering from Zhejiang University, Hangzhou. She is a postdoc researcher in Biodesign Center for Bioelectronics and Biosensors, Arizona State University. Her research interests include developing workout energy expenditure and heart beat montioring system based on low-cost camera.

Francis Tsow received his PhD in electrical engineering from Arizona State University, Tempe. He is an assistant research professor, in Center for Bioelectronics and Biosensors, Biodesign Institute, Arizona State University. His research interests include sensors that are related to environmental monitoring, fitness/exercise and health monitoring, diagnosis, and disease management in a low cost, efficient, and high impact manner.

Dangdang Shao received her BA degree in mechanical engineering and automation from Shanghai Jiao Tong University, her master’s degree in electrical engineering from Shanghai Jiao Tong University, Shanghai, and her PhD in electrical engineering from Arizona State University, Tempe. Her research interests include monitoring vital physiological signals using optical system, image processing, and artificial intelligence.

Nongjian Tao received his BA degree in physics from Anhui University, Hefei, and his PhD in physics from Arizona State University, Tempe. He is the director of the Center for Bioelectronics and Biosensors at the Biodesign Institute and is a professor in the Ira A. Fulton Schools of Engineering at Arizona State University. His research interests include exploring fundamental properties of single molecules and nanostructured materials, developing new sensing and detection technologies, and building fully integrated devices for real-world applications.

Disclosures

No conflicts of interest, financial or otherwise, are declared by the authors.