Reliability and Validation of the Hexoskin Wearable Bio-Collection Device During Walking Conditions

Abstract

To evaluate if the Hexoskin smart shirt (HxS) would produce valid and reliable measurements for heart rate (HR), respiratory rate (RR), minute ventilation (V_E), step count (SC), and energy expenditure (EE) when compared to a Polar T-31 heart rate monitor, an Applied Electrochemistry Moxus Metabolic System, and a manual step count. A two-day walking treadmill protocol with participants walking for 3 minutes at 3 speeds (1.5mph, 2.5mph, 3.5mph, 0% grade) was performed. Forty-nine volunteers participated the first day, forty-six on the second, thirty-one were used for reliability. Values calculated for the HxS data used Pearson’s product-moment correlation (p < 0.05; r ≥ 0.70) for validity and Cronbach’s α (≥ 0.70) for reliability. HxS HR (1.5mph; p<0.01, r=0.86, α=0.86. 2.5mph; p<0.01, r=0.81, α=0.88. 3.5mph; p<0.01, r=0.85, α=0.85), HxS RR (1.5mph; p<0.01, r=0.87, α=0.93. 2.5mph; p<0.01, r=0.86, α=0.92. 3.5mph; p<0.01, r=0.71, α=0.76), HxS V_E (1.5mph; p=0.66, r=0.11, α=0.70. 2.5mph; p=0.01, r=0.15, α=0.73. 3.5mph; p=0.31, r=0.08, α=0.14), HxS SC (1.5mph; p=0.90, r=−0.01, α=0.70. 2.5mph; p=0.22, r=0.13, α=0.86. 3.5mph; p<0.01, r=0.74, α=0.85), HxS EE (1.5mph; p<0.01, r=0.56, α=0.85. 2.5mph; p<0.01, r=0.50, α=0.83. 3.5mph; p<0.01, r=0.51, α=0.80). HxS HR and RR provided valid and reliable measures at all three speeds while V_E, SC, and EE had a mixture of results based on speed. These results are important in the use of the Hexoskin in an accurate manner for athletes, coaches, and for the potential medical applications being advocated in the field of telemedicine procedures.

INTRODUCTION

Currently, the fields of fitness and sport are experiencing a rapidly growing interest in wearable technology. The global wearables market is expected to reach a value of 19 billion U.S. dollars in 2018, more than ten times its value five years ago (22). Also, the Center for Disease Control (CDC) estimates that over 2/3 of the US population is either overweight or obese (3). The use of wearable technology has become an ever-increasing important tool to address both areas. Physiological measurement devices that were once limited in use due to their large size, lack of mobility, or bulk have become much smaller and portable in the last decade. These technological advances have permitted physiological recording instruments to escape the confines of the laboratory or hospital setting for use amongst the general population. There is plentiful interest in both the professional sport and recreational exercise community for accurate measurement devices that are small, unobtrusive, and comfortable to wear. Apparatuses that can accurately and consistently measure physiological functions during exercise, training, or actual competition but minimally influence the wearers movement mechanics when performing an activity can be of value. These devices can potentially provide instantaneous results under real training conditions. This data could then be applied immediately, allowing for coaches, athletes or every-day persons to establish optimal up-to-the-moment training intensities and precisely keep track of bodily measurements.

Devices that are small, accurate and reliable are not only beneficial to those who exercise but can be valuable to the medical community as well. Wearable technology along with wireless advances in information transference have the potential to expand both the concept of remote based medical observation and long distance medical evaluations of patients by physicians (4). It was observed that medical related financial costs can be lowered by up to 20% through self-initiated or home-based remote monitoring of less serious medical conditions by use of an appropriate recording device and a means to transfer data to a medical facility (13). Thus, patients could be monitored remotely, freeing up valuable hospital space.

Lastly, wearable technology can be applied to persons while working. This can be especially helpful for those that work in high stress or physically demanding fields. The ability to monitor vital signs for persons performing dangerous, high-risk jobs can help keep them safe from physical exhaustion or over exertion. These situations can lead to numerous maladies that can be prevented by having knowledge of how that activity is affecting the participant (11). Biofeedback shirts have been used to monitor fire-fighter vital signs while performing duties such as climbing flights of stairs or searching on hands-and-knees for rescue victims both in and out of their full turn-out gear (21). Construction workers have been evaluated with similar bio-feedback shirts to measure the physiological stresses they endure while working for long periods in extreme environments (5). Lastly, occupations that require a seated, stationary position for long periods such as intercontinental truckers and airline pilots could benefit from wearable technology to assist in monitoring their physiological status during the long sedentary periods they experience.

The Hexoskin shirt (HxS) (Carré Technologies Inc. San Francisco, CA, USA) was one of the first wearable technology devices to be released to the public that measured multiple physiological functions simultaneously. The shirt comes in various sizes and has a version for both genders. All versions/sizes have sensors embedded in the fabric and are made of stretchable fabric to be worn tight against the body. Sensor measurements are stored in a recording device (RD) that is connected to the sensors by a plug-type connector. The HxS turns on and off by plugging in and unplugging the RD. The RD is placed into a small, waist level pocket on the right side of the shirt during use. Once connected, the HxS data can be viewed in real time on a smart-phone via an appropriate application or download later to a PC via a USB cable.

Cardiac measurements are made using three cardiac, dry, textile electrodes to produce a one lead electrocardiogram (ECG). Two are embedded in the shirt at sternum level on either side of the pectoris muscles and one is at abdominal level on the wearers right side (6). One elastic, self-hooking strap (ES) is enclosed with the HxS. This ES is wrapped around the user’s body through small fabric loops attached to the HxS and assist the shirt fabric with pressing the sensors tightly against the skin. This helps to ensure HxS sensor connectivity by preventing them from either shifting or loosing direct contact. Male versions of the HxS have loops on both sides of the body in mid-auxiliary positions at both sternum and abdominal level. The female version does not have sternum level loops. It does, however, have a built-in sports bra. Data produced includes heart rate (HR), heart rate variability (HRV), heart rate recovery (HRR), and a one lead electrocardiogram read-out (ECG) (6). Respiratory rate (RR) and minute ventilation (VE) are measured by two magnetic sensors that measure the shape of the body as a person breathes (19). The first is located anteriorly at sternum level in line with the chest ECG electrodes. The second is also located anteriorly along the abdominal area in line with the abdominal ECG electrode. HR, RR, and VE readings along with an internal 3d accelerometer in the RD can estimate values for maximum volume of oxygen (VO2max), estimated energy expenditure (EE), step count (SC), cadence (CA), and 3d acceleration (AC) data (7).

To date, there is little research that indicates whether any data collected by the HxS is reliable or valid. The purpose of this study was to determine reliability and validity of the HxS’s measurement of HR, RR, VE, SC, and EE during a treadmill protocol using three different walking speeds. We hypothesized that the HxS would be both reliable and valid for all five physiological measurements mentioned. Of the research that has been conducted, one study produced reliable and valid data for HR, RR, tidal volume, VE, and hip motion intensity when compared to standard laboratory testing devices. These factors were measured during movements associated with daily living to include lying, sitting, standing, and various walking intensities (24). However, three alternate studies have shown the HxS has both reliability and validity issues during various measurement conditions when HR, RR, VE, SC, and EE are analyzed. The first was a treadmill walking protocol that this article is based on (16). The second was an outdoor hiking data collection session (17). Last, was an outdoor running protocol using the HxS and a COSMED Kb42 (23).

METHODS

Participants

Forty-nine participants (male=26, female=23, age 23.43±6.57 yrs.; height 172.11±11.09 cm; mass 76.15±18.46 kg) were recruited from the University of Nevada, Las Vegas (UNLV) student and faculty populations. This research was approved by the UNLV institutional review board (protocol number 1408-4894) and all participants completed an informed consent and an American College of Sports Medicine (ACSM) health risk questionnaire prior to beginning the first treadmill walk. Body composition was evaluated for all participants with a bio-impedance device (TBF-521 Body Fat Monitor/ Scale, Tanita, Arlington Heights, IL, USA). Body Mass Index (BMI) was determined by the formula [Mass (kg)/Height² (m)). Forty-six of the original participants (male=24, female=22, age 23.39±6.69 yrs.; height 171.39±11.5 cm; mass 76.52±18.73 kg) returned for the second treadmill walk. Thirty-one participants (male=18, female=13, age 24.39±7.59 yrs.; height 173.2±10.45 cm; mass 77.95±21.52 kg) data was used for reliability as they retested 1–2 weeks later at the same time and on the same day of the week as the first walk (Table 1).

Table 1.

Anthropomorphic data.

	Walk #1	Walk #2	Reliability
n	49	46	31
Males	26	24	18
Females	23	22	13
Age (yrs.)	23.43±6.57	23.39±6.69	24.39±7.59
Height (cm)	172.11±11.09	171.39±11.50	173.20±10.45
Mass (kg)	76.15±18.46	76.52±18.73	77.95±21.52
Body fat (%)	27.15±6.92	27.62±6.83	28.24±7.00
Body mass index (BMI)	25.36±3.90	25.46±3.93	25.56±4.53

Values are mean ± standard deviation.

Protocol

All participants were fitted with both a Polar T-31 chest heart rate monitor (PHRM) (Lake Success, NY, USA) and a HxS that best fit their body. The Polar T-31 was used as it is generally accepted as a precision heart rate measurement device. To avoid any interference between the PHRM and the HxS sensor at sternum level, the PHRM strap was lowered by approximately one inch from the HxS sensor. This downward shift did not affect the PHRM’s measuring ability. A visual inspection was conducted on all participants prior to every treadmill walking stage to ensure the separation of the two heart rate sensing devices.

Elastic straps were used for both the sternum and abdominal sensors for both gender versions of the HxS. Though there were no sternum belt loops on the female HxS, a strap was still placed at that location for connectivity assurance. The strap was placed at sternum level, just below the breasts, and directly on chest sensors. When there was difficulty making a connection for either the HxS or PHRM, the sensor and the skin underneath was dampened with water to facilitate and enhance the connection (8).

Participant were instructed to stand on a treadmill (T9.14, Nautilus, Vancouver, WA, USA) while they were connected to a validated respiratory cart (MOXUS) (Applied Electrochemistry Moxus Metabolic System, Bastrop, TX, USA) by use of hoses, head harness, mouthpiece, and nose plugs (1, 19). The MOXUS, PHRM, along with a manual step count provided baseline data. HxS measurements were displayed on an iPad (Apple, mini 2, Cupertino, CA. USA). Prior to walking, readings from the MOXUS, PRHM and the HxS were visually observed to confirm that there was a solid connection from all for at least 15 seconds.

All participants performed a treadmill walking protocol. The protocol consisted of three distinct stages of walking at three different speeds. Each stage was separated by a rest interval that allowed for data collection and preparation for the next stage. Participants began by walking at 1.5mph at 0% grade for three minutes. After the rest interval, the speed was increased by 1.0mph while grade remained the same. This continued until the final stage of 3.5mph was completed. The second day of walking used the same procedure but was performed a minimum of three days later.

We have observed that when the HxS does not detect a user’s HR, it defaults to a value of 70 beats per minute (bpm). If a HR of 70 was viewed during a 1.5mph or 2.5mph stage, the entire preparation process was repeated for the next stage to include visual inspection of the PHRM strap in relation to HxS sensor placement, adjustment of the elastic band, and reconnection confirmation of the HxS and PHRM to ensure they restarted with a solid signal from both devices. If the Hexoskin provided any reading other than 70 bpm or appeared to have fluctuations above and/or below 70, that was considered a binding stage and no adjustments were made.

For HR, RR, and VE, validity was evaluated for each individual speed (1.5mph, 2.5mph and 3.5mph) by using the 1, 2, and 3-minute time point. Both day-1 (n=49) and day-2 (n=46) were used for a cumulative value of participants (n=95). This resulted in 285 data points for each speeds validation analysis. Thirty-one participants were scheduled for the same time/day of the following week and were used to determine reliability for these physiological readings at each speed (n=31). HxS HR values were compared to the PHRM. HxS RR and VE were compared to the MOXUS.

Validity for SC and EE used the value obtained for each speed’s overall 3-minute time interval. Both days walks were combined as above for a total of 95 validation data points. The same thirty-one participants used for HR, RR, and VE were used to determine reliability for SC and EE (n=31). HxS SC was compared to the manual count of steps. EE, or calories estimated by the HxS, were compared to those calculated from the recorded MOXUS data. The HxS shirt calculates EE by the following equation; [(0.6HRmax-HR)*(0.6HRmax-HRrest)*Mifflin equation] + [(HR-HRres)*(0.6HRmax-HRrest)*Keytel equation] (9, 10, 15). Calorie calculations for the MOXUS was determined by multiplying the absolute VO₂ value and the RER caloric equivalent for each 1, 2, and 3-minute point. The three were then added together.

Statistical Analysis

Reliability and validity for HR, RR, VE, SC and EE were analyzed using IBM SPSS Statistics 23.0 software (IBM SPSS Statistics 23, IBM Corporation, Armonk, New York). Because no prior values for validity had been established on the HxS at the time of this study, the authors decided that validity would be acceptable using Pearson’s correlation coefficient where both a p < 0.05 and an (r) of ≥ 0.70 was calculated. A Cronbach’s α ≥ 0.70 was considered reliable. β was set at 0.80. Effect size was calculated using G*Power statistical software (G*Power version 3.1.9.2, Universität Kiel, Kiel, Germany). At the time of this study, there was no previous research data to calculate an “n” size. However, using the indicated α, β, and actual “n”, the calculated effect size for HR, RR, VE was 0.17 and 0.29 for SC and EE.

RESULTS

For HxS HR and RR, all three speeds were reliable and valid. The HxS VE at both 1.5mph and 2.5mph was reliable but not valid. HxS VE at 3.5mph was neither valid nor reliable. The HxS SC at both 1.5mph and 2.5mph was reliable but not valid. HxS SC at 3.5mph was both reliable and valid. The HxS EE at all three speeds was reliable but not valid (Table 2 and 3).

Table 2.

Validity

Variable (MPH)	Pearson’s r	Significance	Observed Measurement	Hexoskin Measurement
HR 1.5	0.86	< 0.01	96.48±13.33	93.00±18.22
HR 2.5	0.81	< 0.01	101.69±12.45	98.18±20.05
HR 3.5	0.85	< 0.01	112.72±14.72	110.98±22.70
RR 1.5	0.87	< 0.01	21.66±5.80	19.74±6.10
RR 2.5	0.86	< 0.01	24.51±6.25	21.86±6.98
RR 3.5	0.71	< 0.01	29.94±7.64	26.25±12.06
VE 1.5	0.11	0.66	18.20±4.07	17.46±4.97
VE 2.5	0.15	0.01	21.10±4.62	21.51±5.76
VE 3.5	0.08	0.31	26.77±6.42	28.94±8.94
SC 1.5	−0.01	0.90	268.95±25.17	90.67±66.31
SC 2.5	0.13	0.22	331.60±21.22	209.93±89.34
SC 3.5	0.74	< 0.01	379.82±21.58	378.46±23.40
EE 1.5	0.56	< 0.01	11.90±3.09	13.07±8.02
EE 2.5	0.50	< 0.01	14.43±3.67	14.47±8.14
EE 3.5	0.51	< 0.01	19.43±4.76	20.05±10.51

Bold Type indicates an (r) of ≥ 0.70 and p < 0.05. Values are mean ±standard deviation.

Table 3.

Reliability

Variable (MPH)	Cronbach’s α	Observed Measurement	Hexoskin Measurement
HR 1.5	0.86	97.16±13.83	95.89±17.52
HR 2.5	0.88	102.54±13.68	102.05±16.64
HR 3.5	0.85	113.47±15.94	114.37±19.46
RR 1.5	0.93	21.01±5.38	19.06±5.53
RR 2.5	0.92	24.13±6.02	21.20±6.73
RR 3.5	0.76	29.06±7.46	25.38±13.07
VE 1.5	0.70	18.60±4.23	17.31±4.81
VE 2.5	0.73	21.47±4.88	21.19±5.61
VE 3.5	0.14	27.37±7.09	28.71±9.42
SC 1.5	0.70	268.45±26.50	83.19±66.01
SC 2.5	0.86	331.32±21.93	212.71±89.28
SC 3.5	0.85	378.48±20.05	375.48 ±23.40
EE 1.5	0.85	12.28±3.46	14.51±7.81
EE 2.5	0.83	14.76±4.17	16.09±7.69
EE 3.5	0.80	19.90±5.45	22.44±10

Bold Type indicates an α ≥ 0.70. Values are mean ±standard deviation.

DISCUSSION

The primary aim of this study was to establish an initial pool of data for the evaluation of the HxS’s ability to accurately and consistently measure select cardiorespiratory variables. The first logical goal was to establish whether the basic concepts of reliability and validity were supported by measurements taken by the HxS. A simple walking protocol utilizing 3-minute stages at three speeds; 1.5mph, 2.5mph, and 3.5mph, all at 0% grade was selected. These speeds would elicit notable differences in the five categories we focused on; HR, RR, VE, EE, and SC. We hypothesized that these variables would be reliable and valid at all speeds.

All HxS HR measurements were reliable and valid. However, during testing, the HxS presented quite a few large, inconsistent measurement intervals that from simple observation did not appear correct. These discrepancies were not anticipated. The results reported here were calculated at each minute point (1, 2, & 3) of the walks. There were, however, noticeable HxS HR detection issues between these points for all speeds. Numerous high and low spikes, intervals of non-connection (default to 70bpm), and periods of steadily high or low readings were more common than was expected. Therefore, it can be argued that our reported results were more favorable than those that could have been produced had a more in-depth comparison using 15 or 30 second data points been performed. Also, while the HxS was reliable and valid for HR, measurements comparisons may have been slightly different if validated measurement equipment such as an ECG was used for recording HR as opposed to the T-31 PHRM.

Our findings for the HR points analyzed were consistent with those found by Villar et al. (24). However, they were not in agreement with those found by Tanner et al. (23). Villar et al. (24) found the HxS to have low variability, good agreement, and consistency while Tanner et al. (23) found it to not be so. Even though our analysis did indicate that the HxS was reliable and valid for the indicated points, the discrepancies mentioned previously may have an impact on whether the HxS can be conclusively determined to be a dependable HRM.

Some of the HxS HR measurement issues may have been due to the fit of the shirt. The HxS is made up of a spandex material designed to form-fit to the wearer’s body. However, it was observed during the actual walks, individuals who appeared to have smaller or larger than average chest diameters had more issues maintaining consistent HR measurements, suggesting that smaller chests, especially flat chests, may allow the HxS ECG sensors to shift on the skin or bunch up even with the ES holding them down. Those with larger chests such as overweight persons or large athletes, seemed to have issues with the ECG sensors staying in proper contact with the skin. The larger chest girth appeared to either gradually shift them to a position on the skin where they had trouble detecting the HR or created a gap between the ECG sensor and the skin, even with the ES pushing them down. In all instances where a connection issue was observed, ECG sensor placement was directly investigated before the next walking stage to ensure that the inconsistent HxS HR readings were not due to interference from the PHRM band. In all cases, there was sufficient clearance between the two to eliminate PHRM interference with the HxS sensors as a cause of the HxS HR detection issues.

One of the actions that assisted with some HxS HR connection issues was to get the HxS ECG sensors and the skin underneath wet. This was done per the Hexoskin online support website (8). Moisture on the sensor and on the skin, helps the ECG sensors detect and record heart activity by providing a better medium for electrical activity of the heart to be detected. While this did solve a few of the HR detection concerns, it does not seem to be a viable, permanent solution for real-time use of the HxS. Under controlled conditions such as laboratory-based research, this may be an option. However, it does not seem to be a realistic resolution for the monitoring of daily activities or sleeping periods for the lay person. Questions arose as to whether the ECG sensors would still work, if after wetting them, they dried out during prolonged periods of wear. Also, will the user be willing, able, or have the resources on hand to remoisten them when needed to re-establish the connection if it was lost in normal daily wear. Using the HxS under work clothes, several clothing layers, or in cold weather environments may not allow for this remedy to be performed. This concern also applies to sleep periods where the user will not be able to dampen the sensors or skin for hours at a time.

Various HxS measurement difficulties were also observed for both RR and VE. While the two-horizontal magnetic respiration loop sensors located along the sternum and abdominal levels continually detected a reading, like the HR, there were fluctuations and intervals of high and low values for both measurements, especially for VE. While a solid reading was consistently measured for both, it is interesting to note that while HxS RR was reliable and valid for all three speeds, none of the HxS VE speeds could be considered the same statistically (no valid VE speeds, reliable only at 1.5mph and 2.5mph). The mix of these statistical results based on the same sensor usage is curious. One of the factors that may have influenced these various results is the use of two respiratory sensors. It was speculated that these sensors may not be reading in suitable unison for certain populations depending on the vital sign being measured. The HxS may be currently calibrated to evaluate sternum and abdominal displacements during breathing for RR and VE within a range of body metrics akin to what a person of average dimensions would produce. Average dimensions being defined as 1) chest circumference of 106.7cm for males and 104.1cm for females (20) and 2) abdominal circumference of 101.6cm for males and 96.8cm for females (2). Because of sensor positioning, a person’s physiological shape may need be considered as not all persons using the HxS will fit this average dimension definition. For example, 1) a person with a larger chest diameter and smaller than standard waist, such as an athlete. 2) a person with a smaller chest diameter but larger than standard waist, such as those who are overweight/obese. These combinations may not be within the physical tolerances and/or the mathematical formula(s) that the HxS is designed to account for. These extremes may be influencing the data in a way that is leading to the general underestimation of RR and mix of VE results.

While all three studies, Montes et al. (16), Villar et al. (24) and Tanner et al. (23) indicate that the RR for the HxS was valid, Tanner et al. (23) and Montes et al. (16) additionally analyzed VE. Both found the HxS was not valid for this measurement. Montes et al. (16) found the HxS VE to be reliable at both 1.5 and 2.5mph but not at 3.5mph.

HxS SC results had no significance and low correlation when compared to the manual step count for both 1.5mph and 2.5mph speeds These results mimic those found in previous studies that show there is a high probability for an accelerometer to significantly underestimate steps taken when walking at speeds of approximately 2.5mph and slower (12, 14). At 3.5mph, the HxS was significant and highly correlated. One reason for the non-significant results in HxS SC at the slower speeds may be in how the RD is worn during movement. The HxS has a small pocket that is positioned along the right mid-axillary line, just above the iliac crest where the data pack is placed during exercise. During walking speeds > 2.5mph, it can be argued that there was enough central body movement for the RD 3D accelerometer to register the appropriate motions as steps. At slower speeds, however, persons may have to modify their normal walking pattern. This can include less swinging of the arms, taking fewer but longer steps, and/or reducing hip motion to accommodate the lower gait required for the slower speed. It is possible that the combination of walking mechanics required to purposefully walk at slower speeds contributes to the SC measurement discrepancy for the HxS.

All HxS EE values compared to the calculated MOXUS EE values produced non-acceptable validity results (all (p) values < 0.01 however all (r) values < 0.7) though all were reliable. Because the HxS EE is heavily reliant on various HR measurements for EE (9, 10, 15), the high validity and reliability values of the HxS HR would logically lead a user to assume the same for HxS EE estimations. Even though HxS EE (r) had close to acceptable ranges of 0.51–0.56 and all (p) values were < 0.01, the values for HxS EE are more likely the result of favorable but coincidentally obtained values and not the result of consistent and steady measurements taken by the HxS for EE. First, there were many HR reading that were high and low for various lengths of time in between the 1, 2, and 3-minute points used in our study. As discussed previously with HR, had more points been used in comparison calculations, the results may not have been as favorable. Also, because HR was not detected or dropped completely for many stages and speeds, the default of 70bpm may have influenced the analyzed values by providing a number that was not realistically registered by the HxS and thus providing inaccurate values for the equation use to determine calories.

Our final conclusions, however, were not conclusive. While some of the measurements were both reliable and valid, others were not. In the case of HR, while it was reliable and valid for the points used in the statistical analysis, it had issues in connectivity and consistency that could easily have rendered it to not be so in both statistical analysis. The HxS’s accuracy and consistency must be further evaluated to determine its ability to provide correct measurements for users in both laboratory and real-time settings. Factors such as Body Mass Index (BMI) and body composition may have had an influence on some measurements and should be investigated further. Also, because there are two versions of the HxS based on gender, it would be interesting to evaluate if one was better at recording measurements.

The Hexoskin is an exciting technological idea that has immense potential for the physiological measurements of athletes, potential medical patients and the workforce. By providing real-time values for vitals such as HR, RR, and VE, accurate, instantaneous training intensities can be set for athletes, potential telehealth/telemedicine duties may be performed, and the gauging of environmental parameters that lead to worker injuries can be identified. The HxS is an impressive instrument that one day may be invaluable for a multitude of purposes. However, the inconsistencies and connection issues are major factors that will need to be thoroughly investigated and corrected before it can be used with confidence.