The Accuracy and Validity of iOS-Based Heart Rate Apps During Moderate to High Intensity Exercise

Abstract

People use their smartphones for everything from web browsing to tracking fitness metrics. However, it is unclear whether smartphone-based apps that use photoplethysmography to measure heart rate are an accurate or valid measure of exercise intensity. Purpose was to determine the accuracy and validity of two iOS-based heart rate monitors, Runtastic Heart Rate Monitor and Pulse Tracker PRO by Runtastic (Runtastic) and Instant Heart Rate+: Heart Rate and Pulse Monitor by Azumio (Instant Heart Rate), when compared to the electrocardiogram (ECG) and Polar® T31 uncoded heart rate monitor from moderate to vigorous intensity exercise. Participants were 15 male and female regularly active college students. Pre-exercise heart rate and blood pressure were recorded and then participants exercised on a stationary bike at a pedal rate of between 50–60 rpms. After completing a warm-up stage at 40% of age estimated maximum heart rate (AEMHR), exercise intensity progressed from 50% of AEMHR through to 85% of AEMHR in eight, 5-minute stages. At the end of each stage, and having achieved steady-state, heart rates were recorded from each apparatus. After completing the final stage, participants completed a cooldown at 40% of their AEMHR. Post-exercise heart rate and blood pressure were also recorded to ensure full recovery to baseline. There was a strong positive correlation between the Polar® monitor and the ECG during all stages. However, there were not strong correlations for either of the smartphone-based apps at any time point. Although there were weak correlations between the smartphone-based apps and ECG and Polar®, further studies need to be conducted to determine if inaccuracy is due to user error (finger placement, finger temperature, etc.) or the technology behind the apps.

INTRODUCTION

Smartphone use is ubiquitous and as the smartphone technology increases, people are beginning to incorporate smartphones into their daily fitness routine. Wearable fitness trackers and smartphone-based applications have become a norm of the fitness world (3, 5, 8, 12, 17, 20). Whether it’s a Fitbit®, Garmin, Apple Watch, or any of the other fitness devices available, people can now choose to wear a portable fitness tracker or use a smartphone app to track and monitor their exercise (3, 5, 6, 10, 14, 21, 22). Many of these smartphones have a number of different sensors (accelerometer, gyroscope, GPS, heart rate monitor) that allow for both daily and activity specific measurements. Accelerometery tracks one’s steps and photoplethysmography is used to measure heart rate on these devices. Photoplethysmography, has been found to be an accurate way to assess heart rate and is the mechanism used by many smartphones and wearable heart rate monitors (1, 6, 12, 13, 16, 20, 22). The mechanism used by many smartphone apps and wearable heart rate monitors, detects the pulsation of capillary blood flow underneath the skin. This measurement can then be used to acquire heart rate. (1, 2) Research has shown that some of these apps are accurate at rest and provide support for the notion that smartphone apps can be used to measure heart rate without a standard ECG (3, 5, 12, 17, 19, 20, 21). Accurate and valid heart rate values during exercise indicate the intensity at which a person is working.

Heart rate monitoring apps are appealing to consumers because they are offered at a minimal cost and are easily accessed (21). Additionally, smartphone-based apps are also changing the medical practice with the ability to measure vital signs almost instantly (21). However, as a person works at higher intensities during an exercise session, the use of these apps can become more difficult because of the needed hand placement to obtain a heart rate. Photoplethysmography has been found to be an accurate measure of heart rate compared to a standard ECG and some of the devices and applications that use this method of measurement are reliable at rest (5). There is, however, limited research investigating the validity and reliability of these smartphone-based apps during increased exercise intensities. (6, 22) Therefore, the purpose of this study was to examine the accuracy and validity of two, iOS-based smartphone heart rate apps, Runtastic Heart Rate Monitor and Pulse Tracker PRO by Runtastic (Runtastic) and Instant Heart Rate+: Heart Rate and Pulse Monitor by Azumio (Instant Heart Rate), when compared to the standard ECG and a Polar® T31 uncoded heart rate monitor, at varying exercise intensities.

METHODS

Participants

Participants were recruited from Ohio Northern University’s Department of Human Performance and Sport Science. The participants were a total of 15 students, both male and female ages 18–23. Participants were recruited through an email sent out to all students in the department. Students were offered extra credit points as compensation to participate. Inclusion criteria included: being regularly active (at least three times per week), and being apparently healthy. Students with any health issues preventing them from exercising at an intensity of 85% of their AEMHR were excluded from the study. These issues included, but were not limited to orthopedic, cardiac, or pulmonary complications.

Protocol

The protocol used was comprised of two separate visits to the Ohio Northern University Exercise Physiology Lab. During the first visit, participants were asked to review and sign a consent form in accordance with the policies of the Ohio Northern University Institutional Review Board, who approved this study prior to data collection. The second visit consisted of a graded exercise test. Ambient environmental conditions in the laboratory were 21.1 °C and 47% relative humidity. These values were determined using an Omega OM-EL-USB-2- LCD-Plus temperature, humidity, and dew point data logger (Omega Engineering, Stamford, CT). The logger was centrally positioned in the room and left to record environmental conditions for one week prior to data collection. Each exercise session began with the participant sitting quietly for ten minutes while personal information, including name, age, height, and weight, was collected. At the end of the ten-minute baseline period, pre-exercise blood pressure and heart rate (via palpation) were recorded. Following the completion of a five-minute warm-up stage, each participant began his or her 40-minute exercise session, during which he/she pedaled at a rate of 50–60 rpms on a standard Monark cycle ergometer for the duration of the session. The 40-minute exercise session was divided into eight (8) individual five-minute stages. Five-minute stages were chosen because in the course of five minutes, a participant can adapt to each new workload and achieve a physiological plateau and steady state within a reasonable amount of time. The first three minutes of each stage (0:00 – 3:00) were used to allow participants to reach a plateau in his/her heart rate. The fourth minute (3:00 – 4:00) allowed participants to reach steady state, and the fifth minute (4:00 – 5:00) was used for data collection. This process was repeated for each stage. Additionally, five-minute stages allow the investigators to collect accurate data while not adding significant stress to the participant.

During the last minute of each stage, heart rates from the ECG and Polar® T31 uncoded heart rate monitor were recorded by one investigator while heart rates from the two iOS apps were collected by a second investigator. Heart rates from the apps were measured using photoplethysmography. To measure the heart rate, the investigator opened the app and participants placed their index finger over the camera and the flash. Measurements were recorded and resistance was increased to the next stage. The process was repeated for each of the eight, five-minute stages, increasing in intensity of 5% each stage. After the eighth stage (85% of AEMHR), resistance was decreased and participants began a five-minute cool down stage, where they exercised at a heart rate equal to 40% of the AEMHR. Heart rates were again recorded from all the devices at the end of the stage. After completing the cool down, participants then sat quietly while their post-exercise heart rate and blood pressures were measured to ensure that they had properly recovered to baseline levels. Once participants had returned to baseline levels and demonstrated no signs of exercise intolerance, they were released from the lab.

During each of the individual stages, the participant exercised at an intensity sufficient enough to elicit a range of percentages of his/her AEMHR. The AEMHR was chosen because it represented the most common way that an average exerciser would measure his/her exercise heart rate. The percentages chosen represent moderate to vigorous exercise intensities (i.e. 50%, 55%, 60%, 65%, 70%, 75%, 80%, and 85% of AEMHR), as offered by the American College of Sports Medicine (ACSM) and the Centers for Disease Control and Prevention (CDC). This range of exercise intensities is a combination of the individual recommendations offered by each of the aforementioned organizations. This range of exercise intensities was chosen because it best represents the exercise intensities that the majority of the general public would exercise at and where/when the iOS-based apps would be used. During the course of the 40-minute exercise session, participants were allowed fluids ad libitum.

Statistical Analysis

A Pearson’s Product Moment Correlation was used to compare each of the predictors, Polar®, Runtastic, and Instant Heart Rate, to the criterion ECG. Alpha (α) was set at .05 a priori.

RESULTS

As expected, results support the accuracy of the Polar® heart rate monitor compared to the standard ECG with correlations above .95 at each time point (See Figure 1.) However, there was no clear relationship between the two iOS-based apps and the ECG (See Figures 2 and 3). It should be noted, however, that there were stronger correlations at the later stages of each trial than at the beginning with Runtastic having two correlations above .9 in the final two stages and Instant Heart Rate with a .810 in stage 6, .676 in stage 7, and .762 in stage 9 (See Table 1). Although the smartphone apps had greater correlations at certain time points, these results were not consistent or significant enough to suggest that the apps are accurate or valid. Table 1 shows a summary of the correlations from the data collected.

Figure 1.

Polar® T31 uncoded heart rate monitor versus the standard ECG. All raw heart rate measurements are listed.

Figure 2.

Runtastic app versus the standard ECG. All raw heart rate measurements are listed.

Figure 3.

Instant Heart Rate app versus the standard ECG. All raw heart rate measurements are listed.

Table 1.

Correlations of Polar, Runtastic, and Instant Heart Rate compared to the criterion ECG.

Criterion/Predictor	Polar	Runtastic	Instant Heart Rate
ECG 1 (50%)	.976	.490	−.213
ECG 2 (55%)	.996	.034	−.074
ECG 3 (60%)	.995	.239	.401
ECG 4 (65%)	1.00	−.690	.582
ECG 5 (70%)	1.00	.580	.028
ECG 6 (75%)	.992	.608	.810
ECG 7 (80%)	.994	−.097	.676
ECG 8 (85%)	.996	.906	−.095
ECG 9 (40%)	1.00	.915	.762

Note: Correlations of each predictor compared to the ECG. Each row indicates a stage of the graded exercise test, starting with ECG 1 (50% AEMHR) and ending with ECG 9 (cool down stage). As expected, the Polar heart rate monitor showed high correlations. The Runtastic and Instant Heart Rate smartphone apps had inconsistent correlations.

DISCUSSION

We found that the two smartphone-based apps, Runtastic and Instant Heart Rate, had relatively weak correlations to the standard ECG and Polar® T31 uncoded heart rate monitor. However, several limitations may account for the poor correlations. Finger placement could potentially have an effect on the apps’ ability to obtain an accurate reading. Photoplethysmography works best when there is direct and consistent contact with the camera and the flash on the phone. If the finger is not fully covering the camera or the flash, it is harder to obtain a heart rate. For our trials, we used different phones to gather the heart rate measurements, therefore there were different cases on each phone. There were phones that had either no case, a thin case, or a thick case that may have affected measurements. This could have led to some of the inconsistency seen in the data. There were times when the smartphones had to be repositioned a number of times to obtain a measurement. The researchers assisted participants with their finger placement to ensure more accurate readings. This makes finger placement an important factor when using these apps outside of a lab setting. Another factor influencing both finger placement and the accuracy of the measurements as the movement of the participants. Participants were stationary during this study, allowing for more consistent finger placement, but exercise that requires an individual to be constantly moving, such as running, could affect the accuracy of the measurement. This means that these apps may only be reliable during specific, more stationary activities or at rest (5). Additionally, having a bulky phone case on the phone can add to the difficulty of achieving proper finger placement on the flash. Another observation that seemed to affect measurement was cold fingers, as the apps often did not provide readings in these situations. The use of these apps may not be accurate during outdoor activities, specifically in colder weather. Our trials took place in the winter months in Ohio. Outside temperatures were around freezing. When a participant came into the lab to complete their trial they may have had cold hands and fingers, even though we were inside the temperature controlled lab. Furthermore, if a participant had an excessively sweaty hand, a heart rate measurement was harder to obtain. In these cases our researchers attempted to dry the participants’ fingers to obtain more accurate heart rates from the apps. Given all of these factors that may influence the accuracy of measurement, the apps may not be the best for heart rate assessment during any form of exercise.

While our study showed the apps had a poor correlation to the criterion ECG, further studies should be conducted to determine the effects of extra movement, incorrect finger positioning, phone case size, finger temperature, and wet or dry conditions on the apps’ ability to measure heart rate. This would determine if these cofactors account for the discrepancy, or if the technology behind these smartphone-based apps needs to be improved to have more external validity. Whether it is an athlete, diabetic, or an at home rehabilitation patient, tracking heart rate during exercise is important to identify the target exercise intensity. As such, having instant access to health apps on phones could contribute to the improved management of chronic diseases such as diabetes, hypertension, or heart failure (3, 21).