Accuracy and usability of single-lead ECG from smartphones - A clinical study

Abstract

Background

Several wireless ECG devices are commercially available for possible screening, monitoring and diagnosis of rhythms. The field is rapidly expanding, and some devices have demonstrated acceptable qualities. The objective was to evaluate the accuracy, usability and diagnostic capabilities of smartphone ECG in both patients and healthy controls.

Methods

We used a commercially available smartphone ECG device, connected wirelessly to a tablet, to record a 30-s lead I ECG in 144 subjects—20 of whom repeated the test after vigorous exercise. The subjects included 94 patients under standard calculated 12-lead ECG surveillance; transcripts were obtained shortly after the smartphone ECG was acquired.

Results

No significant differences were found in the QRS, frequency and QT intervals between the two modalities. Smartphone ECG recordings separated pathologic rhythms (atrial fibrillation (AF)/flutter, atrioventricular block, regular supraventricular rhythm, and pacing) from sinus rhythms with a sensitivity of 0.75 and a specificity of 0.97. The specific diagnosis of AF appeared in 11 patients and was detected with a sensitivity of 1 and a specificity of 0.94. There was a marginal decrease in the interpretability of the smartphone ECG after exercise. Inter- and intraobserver variability was low.

Conclusions

Smartphone ECG accurately measures most baseline intervals and has acceptable sensitivity and specificity for pathological rhythms, especially for AF. Vigorous activity has a minor influence on the readability of the PR interval. Elderly patients may face challenges in recording a smartphone ECG correctly without assistance. According to our findings, the smartphone ECG would be applicable as a screening device for pathological rhythms.

Highlights

• •ECG Check presented a sensitivity of 0.75 and a specificity of 0.97.
• •The sensitivity for atrial fibrillation alone was 1, the specificity 0.94.
• •The smartphone ECG is applicable as a screening device for pathological rhythms.
• •Elderly subjects experience difficulties handling the device.
• •ECG acquisition directly after activity did not significantly affect readability.

1. Introduction

Different wireless, single-lead real-time electrocardiogram (ECG) devices are commercially available for efficient screening, monitoring and on-demand diagnosis. Some devices have demonstrated acceptable detection efficiencies for intervals and rhythms [[1], [2], [3]]. ECG Check is an FDA-approved [4] mobile heart monitor manufactured by Cardiac Designs. The method allows easy acquisition, and the recording can be forwarded to a professional interpreter of choice. Many of these do-it-yourself monitors are available, but the question stands as to whether they truly work or merely provide a false feeling of security. In this article we hope to shed some light on this matter. Wanting to evaluate the accuracy and usability of smartphone ECG (spECG)1, we studied both healthy controls and patients. In the comparison of the spECG to a standard calculated 12-lead ECG (scECG)2, our hypothesis was that the device would prove to be of poor clinical value. In addition, by using a questionnaire, we wanted to quantify the user-friendliness, accessibility, technological confidence, and need for assistance. The study was approved by the Regional Committee for Medical and Health Research Ethics (REK Vest 2016/729).

2. Materials and methods

This study was a prospective, nonrandomized, and adjudicator-blinded study. After obtaining informed consent, we collected a final number of 164 spECGs from a total of 144 persons. The study population consisted of 37% females, and the mean age was 58 years. The participants were divided into subgroups based on age (<25 years, 25–50 years, 50–75 years, or >75 years), each including at least 20 subjects.

2.1. Acquisition

The subjects included 94 patients with ongoing scECG cardiac surveillance admitted to the Cardiac Ward at Haukeland University Hospital, Bergen. The subjects were given basic instructions on how to use the spECG device and send the result to an email address created for the purpose. By putting two fingers on the ECG Check, it registers a 30-s, one-lead ECG and stores it on a device (smartphone, tablet) via Bluetooth. The application's algorithm then classifies the spECGs as either “Normal” or “Abnormal”, and it also estimates the frequency using the RR interval. Neither the frequency estimation nor the application's classification of spECG was regarded in the interpretation. The participants performed the recording as independently as possible, supervised by study investigators and with assistance if needed. Shortly after acquiring the spECG, scECG reports were extracted for comparison. Following the ECG-acquisition process, all of the subjects completed a questionnaire subjectively assessing the performance and usability of the device on a scale from 1 to 4. The observers rated any need for assistance as “Yes” or “No”.

In total 50 asymptomatic controls were included on the criterion of “no previously known heart disease”. In the group of healthy controls <25 years old, 20 subjects were asked to do two spECG acquisitions-one before and one after vigorous exercise. This approach was used to ascertain whether physical activity influenced the readability of the spECG or not. The exercise was high-intensity but nonstandardized in form, and the subjects were at rest during the acquisition process.

2.2. Data

The collected data were anonymized and were analyzed manually and independently by two observers to avoid bias. The agreement between the two observers was tested by the intraclass correlation coefficient (ICC). Four standard ECG intervals were measured (frequency, PR, QRS, and QT). The QTc interval was calculated using a digital calculator based on Bazett's formula [5]. The frequency was determined by measurement of 10 beats and calculation of mean RR time. For scECG, Lead 1 was used to measure all intervals. Lead 2 was used for frequency only, due to longer duration on the transcript. The waveforms were not amplified.

2.3. Process

The subjects were diagnosed by the observers for the purpose of the study, and five pathological diagnoses were employed in addition to sinus rhythm (atrial fibrillation (AF), atrial flutter, atrioventricular block, regular supraventricular rhythm and cardiac pacing). Some of the ECGs were uninterpretable to the reader; these were categorized as such but were included in the analysis in order to quantify readability.

Using a 2 × 2 contingency table, the sensitivity and specificity for two independent situations were estimated. We defined an abnormality seen in both modalities as a true positive, a true negative was no abnormality detected in either modality, a false positive was an abnormality only seen in the spECG and a false negative was an abnormality only found in the scECG. Supraventricular extrasystoles and ventricular extrasystoles were disregarded due to their intermittent nature and the fact that spECG and scECG did not correspond perfectly with each other in the temporal aspect under the current study design.

The measured intervals from the two modalities were compared in Stata/IC [6]. A box plot containing a 95% confidence interval demonstrated the variance. To determine if there was a statistically significant difference between the two, a paired T-test was applied.

3. Results

As demonstrated in Fig. 1, the percentage of smartphone owners decreases with increasing age. The need for assistance in handling the technology, as rated by the observer, increases correspondingly. The same pattern applies to how easy the participants perceived the recording process to be, but a large number in the older groups reported being comfortable using this device independently when given proper training and instruction.

Fig. 1.

Questionnaire. Bar graph displaying the self-reported results from the questionnaire in the different age groups.

3.1. Statistics

A total of 820 spECG-intervals were measured. Of these, 10% were uninterpretable to the reader due to poor quality and noise, inconsistent findings or low wave amplitude on the spECG. These data mean that one or more uninterpretable intervals were found in 27% of the spECGs. The cause of this phenomenon was primarily the PR intervals, of which 22% could not be interpreted (Fig. 3, example 2).

Fig. 3.

Comparison of modalities. Examples from the data collected at Haukeland University Hospital.

The spECGs recorded during exercise in the control group demonstrated a decrease in interpretability of the PR interval: from 95% pre-exercise to 87.5% postexercise. All other intervals remained fully interpretable to the reader. In total, 80% of all spECGs showed either coarse variations or finer higher frequency disturbances. These variations and disturbances often occurred due to hand movements at the beginning of the acquisition, but most were still readable.

Observer agreement was tested by conducting an ICC, and the intraobserver value was estimated to be > 0.8, while the interobserver value was >0.6. These values were interpreted as excellent and good, respectively [7].

By comparing the data from the spECG and scECG in the patient group (n = 94) and conducting a set of paired T-tests with a significance level of 0.05, 2 of the 5 intervals (PR and QTc) demonstrated statistically significant differences. These results are summarized in Table 1. To visualize the variation, the data were presented as box plots, as shown in Fig. 2a and b.

Table 1.

Accuracy of intervals (n = 94).

Significant discrepancies		Insignificant discrepancies
PR	(p = 0,0005)	Frequency	(p = 0,342)
QTc	(p = 0,0089)	QRS	(p = 0,153)
		QT	(p = 0,156)

Fig. 2.

a: spECG vs scECG. Box plot displaying the four intervals (PR, QRS, QT, QTc) as measured by scECG and spECG (n = 94). ECG = Electrocardiogram, Sc=Standard Calculated, Sp = Smartphone. b: spECG vs scECG. Box plot displaying the frequencies as measured by scECG and spECG (n = 94). BPM = Beats per minute, ECG = Electrocardiogram, Sc=Standard Calculated, Sp = Smartphone.

3.2. Sensitivities and specificities

When all of the aforementioned pathological diagnoses (AF (n = 11), atrial flutter (n = 2), atrioventricular block (n = 4), regular supraventricular rhythm (n = 2) or cardiac pacing (n = 9)) were defined as abnormalities, the device demonstrated a sensitivity of 75% and a specificity of 97%. Of the pathological diagnoses, 70% were identified identically in both modalities. When only the presence of AF was defined as an abnormality, the sensitivity was 100%, while the specificity was 94%. However, this study included only 11 individuals with AF. The data are displayed in Table 2.

Table 2.

Contingency table.

Abnormality	Total	True pos.	False neg.	False pos.	True neg.	Sens (%)	Spec (%)	NPV	PPV
Pathology	94	21	7	2	64	75	97	0,9	0,91
AF	94	11	0	5	78	100	94	1	0,69
Pacemaker	94	3	6	0	85	33	1

Neg. = negative, NPV = Neg.predictive value, Pos. = positive, PPV = Pos. predictive value, Sens. = sensitivity, Spec = specificity.

4. Discussion

Many cardiac arrhythmias may be hard to identify, partly due to their intermittent nature—only 47% of symptomatic arrhythmias are discovered using conventional 24-h Holter monitoring [8]. Some of these arrhythmias become manifest only during or after strenuous physical activity and are occasionally symptoms of potentially life-threatening conditions [9].

4.1. Arrhythmias

Exercise-induced arrhythmias would be easier to diagnose in cases when the patient is symptomatic. This diagnosis requires a device that can be used right after intense activity. Analysis of healthy controls before and directly after vigorous exercise showed only a slight decrease in interpretability for the PR intervals (95–87.5%). This result is consistent with our finding that the PR interval is the most frequent among the 10% of noninterpretable intervals. The subjects were able to handle the device with an elevated heart rate and a high level of activation. Disturbances were present in all of the pre-exercise spECGs and they persisted in the postexercise spECGs. In general, a high degree of artifacts and disturbances was found, but the results were interpretable. This result indicates that activity does not dramatically affect the interpretability of the spECG. We believe the results to be applicable to the more physically active part of the population.

Approximately 2–3% [10,11] of the general population has undiagnosed cardiac arrhythmias, and the consequences can be dire [12,13]. It is estimated that 14,500 Norwegians suffer from a stroke each year [14] and that up to 25% of these strokes are caused by undetected and untreated AF [15]. One of the main risk factors for AF is age [16], and the elderly population could potentially benefit greatly from ECG Check. This benefit is especially important in new cases of AF or in patients cardioverted by DC shock or medication. At 6 months, 57% of patients with only AF experienced at least one relapse [17], a condition which may go untreated. AF is easier to treat and results in fewer complications with early detection [18]. However, the data collected in this study show that the need for help in performing the spECG increases with age. These subjects also experienced a significantly lower degree of confidence towards and often a lack of access to the required technology. As this group is probably the largest, one must take into account that they may face problems using the device and will be in need of more extensive education and follow up. Even though many of these subjects reported to be confident using the technology independently, this limitation might pose the biggest disadvantage of the ECG Check.

4.2. Suitability

To properly address these groups, the modern-day doctor may need a supplement to conventional diagnostic methods, namely, a suitable screening tool that fulfills both clinical and functional demands in regard to usability, accuracy and availability. Earlier studies of similar devices have shown reasonable cost-effectiveness in certain populations [19]. Several studies have found that a one lead-device may be useful for measuring different intervals, and some have even stated that it is capable of identifying specific abnormalities in the PR interval [1]. The prerequisite for this is that no significant difference can be detected between a one lead-device and the gold standard. Comparing the spECG with the scECG for accuracy, the paired T-test showed that this criterion is only partly fulfilled with the examined device. This test is robust and suitable, and displayed significant differences in 2 of the 5 intervals. A possible explanation for this may be the PR-intervals low amplitude which often is obscured by disturbances, and therefore unreadable. As for the QTc interval the significant discrepancy may be the effect of two slightly inaccurate measurements combined (Frequency and QT).

Fig. 3 displays some of these challenges by demonstrating both differences and similarities between the two modalities for AF and sinus rhythm.

One of the main questions concerning spECG is whether the sensitivity and specificity are sufficient for practical use or not. The sensitivity and specificity for the five specified cardiac rhythms were evaluated together in order to identify the validity of ECG Check as a screening tool. The presence of a pacing device and/or atrioventricular block was also included and classified as pathology. The reason for not evaluating potentially pathological QRS intervals but instead the PR interval was due to the latter's significant discrepancy between the two modalities.

When we estimated the occurrence of pathological rhythms, we found a sensitivity of 75% to be acceptable, while a specificity of 97% was deemed as very good. The high value given to the specificity is partly due to the large number of true negatives present in the study and the fact that all diagnoses were included. However, a paired comparison of the specific patient diagnoses showed that the two modalities agreed completely in 70% of test subjects. In summary, it would seem that even though spECG has a moderate sensitivity, most abnormal findings will presumably be pathological (positive predictive value 0.91).

To explore the diagnostic capacity of ECG Check further, we chose to focus on AF. In this regard, the device demonstrated both a very high sensitivity and specificity, which is consistent with the results of recent studies [[1], [2], [3]]. The greatest obstacle was the artifacts affecting the interpretability of the PR interval, causing some atrial flutters to be categorized as AF. It should be clarified that these findings are not representative of the general population, as only 11 test subjects presented AF. The data indicate a quality that should be further assessed under more specific study designs.

4.3. Limitations

Our study has certain limitations. First, the included participants were not randomized but were selected by convenience. This selection method resulted in quite a heterogeneous age distribution and a difference in size between the age groups. Second, the sample size was relatively small and might therefore not be applicable to the general population. Furthermore, the spECG presented a certain amount of disturbances and uninterpretable waveforms, which might have affected the interpretation of the results. The temporal aspect of the spECG and standard calculated 12 lead ECG (scECG) may cause errors assessing intermittent phenomena.

5. Conclusion

In conclusion, the ECG Check from Cardiac Designs demonstrated acceptable sensitivities and specificities in detecting certain pathological rhythms. The high detection rate of AF indicates that this device is useful in this capacity, but further studies are needed to confirm this finding. ECG Check is an easily accessible tool that varies non-significantly from scECG when measuring the frequency, QRS interval and QT interval. However, the PR and QTc intervals showed significant variations. Vigorous exercise influenced these intervals minimally; therefore, the device may be used for activity-related symptoms. These qualities, combined with a noncostly profile, make the spECG a valid screening device for the general population. Elderly subjects do, to a larger extent, experience difficulties handling the device, and the need for help in performing the spECG increases with age. Even though most of the subjects belonging this group reported being confident using the technology independently, this limitation might pose the biggest disadvantage for ECG Check.

Funding

This research received no grant from any funding agency in the public, commercial or not-for-profit sectors.

Disclosures

The authors declare that there is no conflict of interest.