Earbud-Based Sensor for the Assessment of Energy Expenditure, Heart Rate, and VO_2max

Abstract

Introduction/Purpose

The goal of this program was to determine the feasibility of a novel noninvasive, highly miniaturized optomechanical earbud sensor for accurately estimating total energy expenditure (TEE) and maximum oxygen consumption (VO₂max). The optomechanical sensor module, small enough to fit inside commercial audio earbuds, was previously developed to provide a seamless way to measure blood flow information during daily life activities. The sensor module was configured to continuously measure physiological information via photoplethysmography (PPG) and physical activity information via accelerometry. This information was digitized and sent to a microprocessor where digital signal processing (DSP) algorithms extract physiological metrics in real-time. These metrics were streamed wirelessly from the earbud to a computer.

Methods

In this study, 23 subjects of multiple physical habitus were divided into a training group of 14 subjects and a validation group of 9 subjects. Each subject underwent the same exercise measurement protocol consisting of treadmill-based cardiopulmonary exercise (CPX) testing to reach VO₂max. Benchmark sensors included a 12-lead electrocardiography (ECG) sensor for measuring heart rate, a calibrated treadmill for measuring distance and speed, and a gas-exchange analysis instrument for measuring TEE and VO₂max. The earbud sensor was the device under test (DUT). Benchmark and DUT data collected from the 14-person training dataset study were integrated into a preconceived statistical model for correlating benchmark data with earbud sensor data. Coefficients were optimized, and the optimized model was validated in the 9-person validation dataset.

Results

It was observed that the earbud sensor estimated TEE and VO₂max with mean ± SD percent estimation errors of −0.7 ± 7.4% and −3.2 ± 7.3% respectively.

Conclusion

The earbud sensor can accurately estimate TEE and VO₂max during CPX testing.

INTRODUCTION

Modifiable health risk factors, such as high stress, poor diet, and a sedentary lifestyle account for 25% of all medical expenses and millions of deaths per year worldwide [2]. The US population is becoming increasingly overweight and unhealthy, with an estimated 66% of adults categorized as obese or overweight by the CDC [25]. Nonetheless, more than half of American adults exercise on a regular basis [11], spending more than $55 billion in weight loss programs and over $17 billion on fitness products [31]. The disconnect between dollars spent on weight-loss and obesity levels may be explained by recent findings that traditional diets do not work [24] alone to prevent weight gain and promote weight loss. Dietary measure must be combined with energy expenditure to accomplish long term weight loss and maintenance.

Weight-loss programs aimed at promoting fitness through direct measurement of physical activity via pedometer feedback have shown promise. In particular, incorporating a pedometer in daily life activities has been shown to result in a significant reduction in body mass index (BMI) and blood pressure [7]. Furthermore, combining engaging feedback with an online user experience correlates with improved maintenance of weight loss in long-term diet/weight management studies [10]. These observations indicate that even better outcomes may result from more direct feedback about energy expenditure and aerobic fitness level, such as VO₂, calories burned, and VO₂max.

Indeed, there is a clear opportunity to encourage a broader population to embrace active lifestyles by integrating mobile fitness monitoring devices with compelling user experiences. However, compelling user experiences must be meaningful, and to be meaningful the fitness monitoring gadgets must provide information that is sufficiently accurate to be actionable. This goal is challenged by the fact that commercial pedometers are inaccurate by greater than ± 20% in reporting calories burned [8, 29].

Recent energy expenditure studies, utilizing a wearable ActiHealth chest strap monitor for measuring both physical activity (PA) and heart rate (HR), have demonstrated greater accuracy [5–6]. These researchers achieved such predictive accuracy through branched equation modeling employing heart rate information and accelerometry information as independent variables. While these findings are quite encouraging, researchers employing the ActiHealth monitor point out several shortcomings. First, despite the relatively high precision achievable through branched equation modeling, poorer accuracy is observed if individual calibrations are not employed [5–6]. This means that the wearable monitors must be calibrated for each user, in a process that is both time-consuming and burdensome. Furthermore, as audio earbuds are packaged with smartphones and digital media players that are sold in volumes of hundreds of millions of units a year [16], the audio earbud form-factor provides the opportunity to reach a larger consumer audience that that of a heart rate chest strap which is sold in volumes of less than 10 million/year.

The goal of this study was to determine the feasibility of a highly miniaturized, noninvasive optomechanical earbud sensor technology for accurately monitoring physiological metrics such as heart rate (HR), total energy expenditure (TEE), and maximum oxygen consumption (VO₂max), and this study is reported herein.

METHODS

To overcome these reported limitations, an earbud sensor module – as opposed to an ActiGraph wrist-, arm-, or leg-worn sensor – was selected in this study (Figure 1). Details of the mechanism of operation are described elsewhere [17–21], but in summary, the earbud was comprised a highly integrated sensor module capable of measuring subtle blood flow changes via reflective photoplethysmography (PPG) and changes in body motion via a 3-axis accelerometer. This sensor module was designed to: 1) capture and digitize the optical PPG signal and 2) send the digitized information to a digital signal processor (DSP) for removing motion artifacts and environmental noise from the PPG signal and to continuously generate estimates of heart rate and VO₂ metrics in real time based on a statistical model comprising PPG and accelerometry information. The DSP was in electrical communication with a Bluetooth chipset so that the real-time metrics could be called upon by a client device (such as a laptop or smartphone). A preliminary feasibility study of this PerformTek^® earbud sensor module had previously demonstrated accurate heart rate measurements during exercise, thus potentially eliminating the need for an electrocardiographic chest strap in many use cases. This was a critical finding for the issue of user compliance, as 58% of U.S. headphone owners listen to headphones while exercising and 34% wear headphones during everyday life activities (such as doing work around the house) [13], an order of magnitude greater than the number of Americans who exercise with chest straps.

Figure 1. The components and size of the device under testing (DUT).

Shown are the earbud and the medallion containing the majority of the computational components. Shown for scale is a US quarter. Note that the position of the sensor module at the bottom of the antitragus. The sensor module is configured to fit between the concha and antitragus of the ear.

Subjects

In this study, 23 subjects of good physical health were divided into a training group of 14 subjects and a validation group of 9 subjects. This sample size is justified by the high “effect size” observed for calibrated correlations of VO₂ and heart rate [22] and is further supported by the very high R² coefficient observed [23] when comparing the earbud-determined heart rate to 12-lead-ECG-measured heart rate during exercise. The training group (Table 1a) comprised twelve men and two women: Age – 39 ± 11.8 yr, Weight – 73.5 ± 12.2 kg,, Height – 69 ± 2.9 cm, BMI – 23.6 ± 2.1 kg•m⁻². The validation group (Table 1b) comprised five men and four women: Age – 36 ± 6.9 yr, Weight – 67.6 ± 15.7 kg, Height – 173 ± 7.4 cm, BMI – 22.3 +/− 4.0 kg•m⁻². Each subject underwent the same exercise measurement protocol including a treadmill-based cardiopulmonary exercise (CPX) test, at 0-degree incline, to reach VO₂max. Achievement of VO₂max was determined by reaching at least two of the three following criteria: plateau in VO₂ over the last minute of exercise, achievement of at least 1.10 in respiratory exchange ratio (RER), and achievement of at least 17 in perceived exertion on the Borg Scale. The mean ± SD VO₂max of the training group and validation group were 55.9 ± 6.5 mL/kg/min and 55.1 ± 5.5 mL/kg/min respectively. Benchmark sensors included 12-lead ECG for measuring heart rate, the calibrated treadmill for measuring distance traveled, and a gas-exchange analysis instrument for measuring TEE and VO₂max. The earbud sensor served as the device under test (DUT). All subjects provided informed consent as approved by Duke’s Investigational Review Board.

Table 1.

Descriptive characteristics (mean ± SD) of a) training group and b) validation group.

a) Training Group		b) Validation Group
Parameter	Value (mean ± SD)	Parameter	Value (mean ± SD)
Gender	2 females, 12 males	Gender	4 females, 5 males
Age	30 ± 11.8 years	Age	36 ± 6.9 years
Weight	73.5 ± 12.2 kg	Weight	67.6 ± 15.7 kg
Height	175 ± 7.4 cm	Height	173 ± 7.4 cm
Distance	2.95 ± 0.5 km	Distance	2.80 ± 0.3 km
Energy Expenditure	213 ± 47.8 kcals	Energy Expenditure	178 ± 51.5 kcals
Maximum VO2	55.9 ± 6.5 ml/kg/min	Maximum VO2	55.1 ± 5.5 ml/kg/min
BMI	23.6 ± 2.1 kg•m−2	BMI	22.3 ± 4.0 kg•m−2

CPX Testing

Subjects began the study by first being prepped for wearing the benchmark sensors. A Quinton12-lead ECG system was used as a benchmark for heart rate and a TrueMax 2400 ParvoMedics (ParvoMedics, Sandy, UT) gas exchange analysis mouthpiece was used as a benchmark for energy expenditure and continuous measures of VO₂. The benchmark sensors were calibrated according to standard maintenance guidelines of the manufacturers. The subjects were then fitted with an earbud sensor (Figure 1) powered by the aforementioned PerformTek physiological monitoring technology. Participants were then asked to sit at rest in the supine position, in reclining chair, for a few minutes while wearing the benchmark equipment and earbud sensor. Following the resting period, subjects were instructed to move from the chair to the calibrated treadmill and execute the CPX testing with graded intensity ranging from 0 – 9.1 mph speeds. The protocol used consisted of 2-min stages, increasing the workload by approximately one metabolic equivalent per stage. Measurements from the benchmark sensors and earbud sensor were collected continuously throughout the entire treadmill run. Participants were asked to continue running during each increasing speed until they were completely exhausted. The last 40s of benchmark gas exchange analysis data were averaged to determine measured peak VO₂.

Earbud Sensor

The novel noninvasive earbud sensor (Figure 1) used in this study was designed by Valencell, Inc. The earbud sensor comprised a sensor module, a microprocessor, and a wireless Bluetooth^® chipset. The optomechanical sensor module, comprising the sensor elements, was embedded within the right audio earbud as shown in Figure 1, such that the sensor module would rest between the concha and antitragus of each subject upon earbud placement. The right and left earbuds were designed to be pluggable to a wireless Bluetooth “medallion” via a detachable connector (as shown in Figure 1). The medallion housed the microprocessor and the Bluetooth chipset.

At the heart of this noninvasive earbud sensor is a highly miniaturized optomechanical module [17–21, 23] that measures optical and mechanical information from the area of a user’s ear between the antitragus and concha. This novel sensor module comprises an infrared light-emitting diode, a photodetector element, a three-axis accelerometer, and an optomechanical housing. Designed to fit flush with the body of a standard audio earbud, the earbud essentially maintains the form-factor of a typical audio earbud and does not require an ear clip or an in-ear-canal sensor to function.

The optical and mechanical information collected from the ear are sampled via methods akin to reflective photoplethysmography (PPG) and 3-axis accelerometry, and this sampled information is then processed by novel algorithms [17–18] coded on firmware within the microprocessor for extracting weak blood flow signals from excessive motion noise. It is well known that motion artifacts are the greatest limiting factor to accurate vital signs monitoring via PPG [3, 14, 27]. However, Valencell’s PerformTek biometric algorithms actively process noisy body signals and extract accurate biometrics even during aggressive running and physical activity [23]. These biometric signals are then combined with contextual accelerometry information within a statistical model to generate assessments of heart rate zone, calories burned, aerobic capacity (VO₂max), and other parameters [17–21]. A phone, computer, or other mobile device can communicate directly with the microprocessor via a Bluetooth link. In this particular study, the earbud sensor data was streamed directly to a laptop via Bluetooth.

Statistical Methodology

A multiple linear regression model had been developed previously by Valencell to provide a linear relation between estimated TEE, as predicted by the earbud sensor measurements, and the measured TEE, as recorded by the benchmark gas-exchange analysis device. This linear model comprised fixed and time-varying terms. The fixed terms included weight (W), age (A), and gender (G) having a binary value of 0/1 for women/men respectively. The time-varying terms included the earbud-estimated TEE (EB_TEE) and linear operations of real-time PPG and accelerometry (ACC). Though the details of the linear model are outside the scope of this paper, the formalism of the resulting linear equation may be described by EB_TEE = f(g(PPG), h(ACC), W, A, G), where “g” and “h” are functions of PPG and ACC respectively. It is important to note that this linear model was directed towards estimating TEE, and not the individual elements of resting energy expenditure (REE) or activity-related energy expenditure (AEE), as TEE is what is measured by gas-exchange analysis.

A separate model had been previously developed by Valencell to estimate VO₂max based on HR and accelerometry data collected during several prior rounds of CPX testing. The methodology behind this VO₂max estimation is described elsewhere [18], and the equation follows the formalism of EB_VO₂max = f(Max_HR, Min_HR, k(ACC)), where EB_VO₂max is the earbud-derived VO₂max, Max_HR is maximum reliable heart rate measured by the earbud sensor, Min_HR is the minimum reliable heart rate measured by the earbud sensor, and “k” is a function of the accelerometer readings measured throughout the CPX testing.

Following the fourteen-person training data study, the best-fitting coefficients for the TEE and VO₂max models were determined, and the resulting optimized equations were utilized in the nine-person validation data study to estimate TEE and VO₂max in real time. The resulting earbud-estimated values (EB_TEE and EB_VO₂max) were then compared with benchmark-measured values in accordance with Bland-Altman formalism [1, 4].

RESULTS

Heart Rate

As previously described, the earbud measurements of heart rate (HR) and physical activity are part of the foundational formulas for EB_TEE and EB_VO₂max. Therefore, it is important that these measurements are accurate. An exemplary characteristic plot of real-time ECG, PerformTek HR, and h(ACC) for a subject undergoing a CPX test is shown in Figure 2. Note that for this test, the benchmark ECG and earbud HR are nearly identical throughout the entire run, such that they completely overlap each other. Although complete overlap was not always observed, complete overlap was typically observed. Only rarely did the earbud or ECG diverge to a substantial degree, as exemplified by the tight correlation shown in Figure 3, and on the rare occasions when divergence was observed it was often attributable to either the earbud moving out of the ear or the ECG leads decoupling from the subject’s skin. For the sake of objectivity in this study, all HR data points measured by the earbud and ECG sensors are shown in Figure 3, even for the case where earbud or ECG failures are subjectively believed to have occurred.

Figure 2. CPX testing output from a characteristic test.

In this characteristic CPX test, shown on the abscissa is time of the progressive exercise test conducted on a standard treadmill; on the rightward ordinate is shown physical activity intensity using accelerometer counts in arbitrary units (a.u.; green line); on the leftward ordinate is shown heart rate (HR) in beats per minute (BPM) from either the PerformTek earbud device (red) or from the simultaneously measured ECG (blue). The earbud-determined heart rate and the ECG-measured heart rate show complete alignment in this exemplary characteristic test.

Figure 3. Heart Rate using the Earbud (DUT) and the simultaneously measured ECG benchmark.

a) Regression relation comparing estimated (Earbud) vs. measured (ECG) heart rate for all data points collected for each participant and b) Bland-Altman plot of same. All the data points were taken from the training data collected during the Duke CPX test. The mean difference (bias) was −0.2% and the standard deviation was 4.4%. The mean is shown by the green line, and the 1.96 SD (95% limits of agreement) boundaries are shown by the red lines.

A Bland-Altman plot comparing earbud-estimated HR (EB_HR) vs. the benchmark 12-lead ECG (ECG) measured from the fourteen-person training group is presented in Figure 3. This figure illustrates the excellent agreement between EB_HR and ECG throughout a full range of activity from rest to >200 beats per minute; the mean difference (bias) was −0.2%, the standard deviation was ±4.4%, and the coefficient of determination (R²) was 0.98. In contrast with other reported optical HR measurement devices reported in literature [3, 14, 27], the EB_HR measurement is quite robust throughout a full range of activity because the PerformTek biometric signal extraction algorithms are capable of characterizing motion noise during numerous activities and attenuating motion-artifacts from the optical signal in real-time.

In contrast with hip and pocket-worn pedometer-based approaches for calculating distance [8, 15, 29], the physical activity level measured by the earbud prototype provides a good reference for body displacement during walking, jogging, and running without requiring knowledge of the users’ gender, height, age, weight, or fitness. Furthermore, neither a calibration regimen nor GPS is required to tune parameters to the wearers’ gait. The earbud prototype distance measurement was highly accurate, with a bias of 0.3%, a standard deviation of 4.2%, and an R² of 0.93. This distance measurement was obtained through a novel transformation of three-axis accelerometer data and its accuracy is aided by the sensor location at the ear.

TEE

The EB_TEE closely estimated the benchmark TEE for the training group dataset, with a bias of 0.7% and a standard deviation of ±7.4% (Figure 4). The correlation between EB_TEE and the benchmark TEE for the validation group dataset was identical to that of the training dataset, with a bias of −0.7%, a standard deviation of ±7.4%, and an R² coefficient of 0.86 (Figure 4).

Figure 4. Bland-Altman plots comparing reference CPX data and Earbud (DUT) benchmark data collective simultaneously.

Bland-Altman plot of the difference between energy expenditure (EE; kilocalories) using the reference CPX test and the DUT benchmark EE measurements taken from the 14-person CPX training group. The mean is shown by the green line, and the 1.96 SD (95% limits of agreement) boundaries are shown by the red lines.

VO₂max

The EB_VO₂max closely estimated the benchmark VO₂max for the training group dataset, with a bias of −0.1%, a standard deviation of ±8.7%, and an R² coefficient of 0.36 (Figure 5). The correlation between EB_VO₂max and the benchmark TEE for the validation group dataset similar to that of the training dataset, with a bias of −3.2% and a standard deviation of ±7.3%.

Figure 5. Bland-Altman plots comparing reference CPX data and Earbud (DUT) benchmark data collective simultaneously.

a) distance traveled; b) VO₂max (ml/kg/min); EE (kilocalories). Nine subjects were studied in this CPX validation group study. The mean is shown by the green line, and the 1.96 SD (95% limits of agreement) boundaries are shown by the red lines.

DISCUSSION

To satisfy commercial, clinical, and research oriented markets for personal energy balance monitoring, a wearable sensor module must satisfy four key criteria. The sensor module must be: 1) seamless with daily living (comfortable, convenient, and socially acceptable), 2) sufficiently accurate for multiple life activities (indoors and outdoors), 3) able to provide understandable, actionable, and motivational feedback to the user, and 4) autonomous and user-friendly. Today, a variety of commercially available products offer step-counting and estimated calorie monitoring. Many of these solutions have provided value to researchers studying energy balance and to fitness professionals and clinicians monitoring the progress of exercise and diet plans. However, none of these products satisfy all the above criteria, limiting the effectiveness and scope of: 1) long-term clinical research on energy balance research and 2) health and fitness solutions for end users. In contrast, newly developed earbud sensor technology offers the promise of meeting these needs, enabling a truly seamless energy balance monitoring platform for use in clinical research, consumer fitness, clinical assessment of energy balance, and mobile health management.

The feasibility has been established for the highly miniaturized, noninvasive optical earbud sensor technology for accurately monitoring physiological metrics such as heart rate (HR), total energy expenditure (TEE), and maximum oxygen consumption (VO₂max) through the ear. The earbud sensor accurately predicted HR throughout all activity levels investigated, from rest to peak performance, with a mean difference (bias) of −0.2% and a standard deviation of ±4.4% when compared with an ECG benchmark device. Furthermore, real-time algorithms within the earbud sensor accurately predicted: a) TEE with a bias of −0.7% and a standard deviation of ±7.4% and b) VO₂max with a bias of −3.2% and a standard deviation of ±7.3%. This particular evaluation did not address user comfort or battery life, but a commercially available Bluetooth audio headset, the iriverON^™, incorporating the evaluated PerformTek^® biometric sensor technology advertises several hours of measurement time while also supporting music.

The excellent performance of the earbud sensor for accurately measuring heart rate throughout extreme physical activity is especially noteworthy. Motion artifacts have been the greatest limitation to continuous monitoring of vital signs during activity [3, 14, 27], and the ability to accurately monitor vital signs with a consumer-priced audio headset is particularly impactful to public health.

When compared with heart rate chest-strap approaches for estimating energy expenditure [5–6], the earbud sensor algorithms for estimating TEE and VO₂max are also noteworthy. It is likely that the fixed location of the earbud with respect to the spine increases the accuracy of activity measurements which feed the TEE and VO₂max models. However, some important limitations of these algorithms are of note. First, these algorithms have demonstrated substantial efficacy in estimating TEE and VO₂max under walking, jogging, and running conditions, conditions common for clinical CPX evaluations. However, it is yet to be determined how accurate these algorithms will be at estimating these parameters during everyday life activities and other exercising regimens, such as weight-lifting, swimming, contact sports, daily household activities, and the like. There are several studies emphasizing the importance of caution when applying VO₂ estimation models to universal physical activity [9, 12, 22, 25, 28, 30, 32, 33]. Moreover, it is not clear how well these algorithms will predict resting energy expenditure (REE) or energy expended during sedentary activity, where caloric expenditure is dominated by the metabolic rate of the individual as opposed to physical activity. Second, the VO₂max range of participants in the validation study was relatively small: from ~50 to 65 mL/kg/min (Figure 5). Indeed, the relatively low R² coefficient for estimated VO²max and the ostensible nonlinear bias of Figure 5b together suggest that the accuracy of the VO²max model cannot be affirmed with the current dataset. Last, to be clinically relevant, the predictions for TEE and VO₂max would ideally be even more accurate, with the goal of a standard deviation of less than ±5%.

To address these areas for improvement, future work should evaluate the efficacy of the earbud sensor at estimating TEE and VO₂max in a larger cohort group having a broader range of aerobic capacity, ranging from ~35 to 70 mL/kg/min. Furthermore, the earbud sensor should be put to the test of estimating TEE during a broader set of activities than simple treadmill testing, using the standard doubly labeled water (DLW) methodology as a benchmark. Additionally, the ability of the earbud sensor to estimate the resting metabolic rate (REE) of subjects should also be assessed.

Improving the accuracy of these assessments will rely on: 1) optimizing algorithms based on a larger study sample of subjects exercising in more diverse environments (such as daily life activities) and 2) adding additional biometrics to the predictive algorithms for energy expenditure and VO₂max. Valencell’s PerformTek earbud sensor is comprised mostly of novel optomechanics and signal extraction algorithms. The accuracy of the heart rate algorithms is so high, approaching machine error, such that it is unlikely that additional improvements can be made in the optomechanical sensor module for accuracy. Rather, advancements are likely to arise from the development of an optimized statistical model that incorporates personalized REE estimations into the model. The algorithms for estimating personalized REE can be developed by evaluating the PPG profile of subjects at rest with a gas-exchange analysis benchmark (REE testing) and identifying new blood flow profile features that correlate with the gas-exchange analysis data.