Novel Tool for Complete Digitization of Paper Electrocardiography Data

Abstract

Objective: We present a Matlab-based tool to convert electrocardiography (ECG) information from paper charts into digital ECG signals. The tool can be used for long-term retrospective studies of cardiac patients to study the evolving features with prognostic value. Methods and procedures: To perform the conversion, we: 1) detect the graphical grid on ECG charts using grayscale thresholding; 2) digitize the ECG signal based on its contour using a column-wise pixel scan; and 3) use template-based optical character recognition to extract patient demographic information from the paper ECG in order to interface the data with the patients' medical record. To validate the digitization technique: 1) correlation between the digital signals and signals digitized from paper ECG are performed and 2) clinically significant ECG parameters are measured and compared from both the paper-based ECG signals and the digitized ECG. Results: The validation demonstrates a correlation value of 0.85–0.9 between the digital ECG signal and the signal digitized from the paper ECG. There is a high correlation in the clinical parameters between the ECG information from the paper charts and digitized signal, with intra-observer and inter-observer correlations of 0.8–0.9 , and kappa statistics ranging from 0.85 (inter-observer) to 1.00 (intra-observer). Conclusion: The important features of the ECG signal, especially the QRST complex and the associated intervals, are preserved by obtaining the contour from the paper ECG. The differences between the measures of clinically important features extracted from the original signal and the reconstructed signal are insignificant, thus highlighting the accuracy of this technique. Clinical impact: Using this type of ECG digitization tool to carry out retrospective studies on large databases, which rely on paper ECG records, studies of emerging ECG features can be performed. In addition, this tool can be used to potentially integrate digitized ECG information with digital ECG analysis programs and with the patient's electronic medical record.

I. Introduction

Electrocardiography (ECG) has existed for more than a century [1], and digital ECG has existed for more than two decades [2]. Digital ECG offers the opportunity to rapidly calculate ECG attributes such as vector angles and waveform intervals previously laborious and imprecise with paper tracings. Its utility is actively expanding as new ECG features with prognostic value are discovered [3]. Unfortunately, as new ECG features are discovered, determining if these features have prognostic significance can involve many years if a prospective evaluation is pursued. On the other hand, with retrospective evaluation, one can quickly correlate ECG features with patient outcomes. Features such as QRS/T angle are promising prognostic tools for predicting coronary heart disease mortality, but are currently clinically underutilized [3]. However, most long-term retrospective cardiac patient databases, which span over a duration of two decades or more have paper ECG data. Additionally, not all digital ECG systems offer software to generate advanced measures. Therefore, a system that can re-digitize to a universal format has added utility. Accordingly, tools for digitizing paper ECG are being developed [4]–[7] to permit such retrospective studies as well as prospective studies that utilize measures not commonly available on standard commercial systems [8].

A second motivation for this work is that in addition to permitting rapid calculation of ECG features, digitizing paper ECG permits the integration of ECG data with the patient's Electronic Medical Record (EMR). with the government-mandated push for Electronic Health Records to be implemented nationally by 2014, such tools will be needed for routine clinical use. Furthermore, such tools would allow for comparison of a patient's current ECG with a baseline ECG, enhancing confidence in the diagnosis of acute changes [9].

In this paper, we present a Matlab (MathWorks Inc., Natick, MA)-based tool for digitization of paper-ECG data. This involves scanning of a paper ECG to an image, thresholding for graphic grid removal, and detecting the ECG signal contour followed by digitization. We validate our technique by demonstrating that there is no significant difference between the digitized signal and the paper ECG. A major, distinguishing feature of our method is that we have used optical character recognition (OCR) to automate the process of reading patient demographic information present on the ECG paper. Thus, with very minimal human intervention, our tool can rapidly digitize a large number of paper ECGs and interface this data with the corresponding patients' EMR.

The rest of the paper is organized as follows. Section II describes the digitization process, the OCR technique and the two methods that are employed to validate the digitization process. In Section III, results from the two different validation steps are presented, followed by a discussion in Section IV and conclusion in Section V.

II. Methods

Standard ECG paper has a graphical grid upon which the voltage information from the 12 ECG leads is printed as a function of time (Fig. 1). The extraction of ECG information from this two-dimensional (2-D) image and conversion to a one-dimensional (1-D) signal is referred to as digitization of ECG. To identify the ECG signal from the standard ECG paper, image processing techniques involving thresholding [10], pixel-to-vector conversion [11]–[15] and transform techniques [16], [17] have been used. We describe below the process of converting ECG paper data to obtain 1-D digital ECG signals. This process can either be carried out on data from a single lead or from all 12 leads.

Fig. 1.

Standard ECG paper with the information from 12 leads printed as a function of time. The regions with the test pulse for one of the leads and the patient demographic information are shown in boxes. The patient demographic information itself is redacted to protect patient privacy.

In the following subsections, the digitization and optical character recognition processes are first described (Fig. 2) followed by the validation procedure for the digitization.

Fig. 2.

Outline of the ECG Digitization Process. The optional steps of the process are represented by dashed boxes.

A. Digitization Process

Fig. 2 outlines the ECG digitization process. First, the ECG paper is scanned as a digital image and is converted to 8-bit grayscale. This is followed by an optional step of scaling down the image resolution to 300 dpi. This scaling down serves to reduce computational costs without compromising reconstructed signal quality. In addition, a process of skew correction, i.e., correcting the angular orientation of the ECG paper, may need to be performed. This is performed by calculating the slope of the gridlines by picking two points on the same gridline and rotating the scanned image using the angle obtained. The tool has a graphical user interface that requests a user to mark the two points.

The next step is a binary thresholding operation to remove the graphical grid and convert the grayscale image to a binary image. Since the pixel intensity value of the graphical grid is usually greater than that of the printed ECG signal, the threshold is determined using the histogram of the image (Fig. 3). By considering the pixel intensity of the graphical grid region alone ( and ), the threshold for the grid removal process is determined automatically. In this example, the range 130–160 was contributed by the grid lines alone).

Fig. 3.

A histogram of the grayscale ECG image where the separation between desired signal component (lower pixel intensity) and the graphical grid (higher pixel intensity) is observed in the range 130–160. Empirically, we determined that a threshold in this region will ensure efficient removal of the graphical grid from the ECG image.

However, binary thresholding may introduce “salt and pepper” noise in the background, which often occurs as a result of the scanning process. In addition, there may be discontinuities or gaps in the ECG signal from removal of major grid lines due to reduced printer ink retention on the major grid lines. Performing median filtering and interpolation after the thresholding process eliminates these artifacts.

On this binary image, a column-wise pixel scan is performed to find the contour of the ECG waveform based on the zero locations (black). Since the printed ECG signal has some thickness associated with it, a pair of row indices corresponding to the upper and the lower contour is obtained for each column. Based on the x- and the y-axis scales, the sampling rate is derived from the resolution of the image, and we obtain time-voltage pairs for the mean of the contour of the ECG waveform. In the case of low-resolution images, significant quantization errors may be introduced in the signal at this stage.

An optional step of smoothing the reconstructed signal to remove the high-frequency noise introduced due to pixeling is carried out next. This pixeling occurs when there is non-uniform rendering of the ink along the gridlines on the ECG paper. Following this, the DC voltage level of the digitized signal is adjusted based on the DC level of the test square pulse, which is an integral part of the paper ECG signal (Fig. 1). The time scale of each ECG signal is also adjusted based on the time duration of the test square pulse.

For purposes of evaluation, we consider ECG paper scanned with a resolution of 300 dpi. Following the process of grayscaling and thresholding to remove the gridlines on the ECG paper, we consider the image segments corresponding to the 12 leads individually. In order to remove salt and pepper noise, we use a 3×3 median filter on these image segments. with high-resolution scans (e.g., 600 dpi), an additional step of linear interpolation between the image pixels on the gridlines may need to be performed to remove discontinuities, if any, in the signal. Following this, the digitized signal is obtained from the outer contour of the waveform. In order to remove the high frequency noise content in this signal, which results from pixeling, a length-5 moving average filter is used, since using a longer filter might smooth out the peaks and troughs in the QRST complex of the waveform. It was observed that reconstructed signals from both 300 dpi and 600 dpi scans were identical. Therefore, to reduce computational costs we reconstruct the ECG signal from the 300 dpi scan.

B. Optical Character Recognition (OCR)

OCR is performed on the patient demographic information printed on the ECG paper. The OCR process involves correlation of every character to be recognized with a pre-defined template of characters to identify the best correlated pair. This is performed after segmenting each character in the image, based on the spacing between them, and scaling it to the size of the images in the template. The patient demographic information extracted by the OCR process is then introduced into the header of the digital record.

The OCR stage has a basic template library with sixty-two images, twenty-six each for the lowercase and uppercase letters and ten for the digits. The segment of the image with the patient information on which OCR is to be performed is localized manually. Our tool has a graphical user interface that requests a user to mark where the patient information occurs for a given format of ECG paper and this space is used for all subsequent ECG evaluations until changed by the user. This is done to reduce computational costs that would be incurred by performing OCR on the entire image. This is a reasonable compromise, since all the ECG paper data produced by a single machine will have demographic information in the same location on the paper.

C. Validation Procedure

The ECG digitization and OCR techniques were implemented on Matlab version 7.10 on scanned images with resolution of 300 dpi. We performed two types of validation to evaluate if the ECG signal digitization process generated digitized waveforms that were comparable to the waveforms printed on ECG paper.

1. Validation Test 1A

The first form of validation involved obtaining the objective fit of the paper ECG image with the digitized ECG plotted over the paper ECG image with the same scale (Fig. 5). The objective fit is determined by observing how well the digitized signal fits within the contour of the paper ECG signal. This can also be viewed as a signal correlation match. for this test, we randomly selected 10 ECGs from a set of 53 ECG samples from the Emory Vietnam Era Twins (VET) Study [18].

Fig. 5.

Plot showing paper ECG signal (black) and digitized ECG signal (blue) for one lead. Note the similarity in the two signals, and the graininess in the paper ECG signal.

2. Validation Test 1B

In order to further validate the digitized signal with the original ECG signal, we obtained the raw digital ECG data (1-D signal) for each of the leads for four test cases. for each of these cases, we printed out a paper ECG and then digitized it. Then the correlation between the raw ECG signal and the digitized ECG signal was calculated after resampling the two signals to the same rate. This resampling process is necessary because the sampling rate for the raw signal is greater than the sampling rate of the digitized ECG signal obtained from the paper ECG.

The sampling rate for the raw ECG signal is 500 Hz and the sampling rate for signal printed on the paper chart is 150 Hz. Following the digitization process, the raw digital ECG signal is downsampled to the sampling rate of the paper ECG signal obtained after digitization. Note that this sampling rate depends on the resolution of the scanned ECG image. This can be attributed to the fact that the signal on the paper ECG chart is a downsampled version of the raw ECG signal and the sampling rate further changes during the digitization process based on the resolution of the scanned image, and this is expected to reduce the correlation value.

3. Validation Test 2

As a second form of validation, we determined if five clinically important features of the ECG signal are preserved through the digitization process [19]. The five features that we considered are: PR interval (lead II), QRS interval (lead V1), QT interval (lead V3), RR interval (lead V6) and the interval. In Fig. 6, three of the measurements are shown. Note that the RR interval is obtained by measuring the interval between two consecutive R peaks. The interval is obtained using . Clinical diagnoses were also calculated, including first-degree atrioventricular block , intra-ventricular conduction delay , QTc prolongation , and bradycardia . Based on these values, the kappa statistic was calculated to measure agreement above and beyond that which would occur due to chance, calculated by . Kappa values above 0.8 are generally considered to represent excellent agreement.

Fig. 6.

Three features of the ECG waveform used in the clinical validation: PR interval, QRS interval, QT interval.

Two cardiologists performed measurements on the 10 ECG samples (labeled) that were used for Validation Test 1A. To calculate intra-observer correlation, reader A measured the five parameters on all 10 ECGs in both digitized form and then in the scanned form after a 24-hour washout period. for inter-observer statistics, reader B then measured the first 5 ECGs , , in scanned form, and the last 5 ECGs, , in digitized form. These 10 measures were then compared to those from the opposite format measured by reader A. Although the intra-observer correlation may over-estimate correlation because of bias, the inter-observer correlation may be under-estimated due to variation in technique. Therefore, both were performed with the concept that the true clinical correlation is in between the two estimates.

III. Results

The results of intermediate steps in the process for a representative ECG signal are shown in Fig. 4.

Fig. 4.

(a) Sample ECG paper image. (b) Result of the thresholding for grid removal on the image shown in (a). (c) Digitized ECG signal obtained from the columnwise pixel scan on the image shown in (b).

1. Validation Test 1A

For the 10 cases the signal fit between paper and digitized ECG images ranged from 0.75–0.8, with the match for an ideal case being 1. Fig. 5 shows the digitized ECG signal and the paper ECG signal on the same scale.

2. Validation Test 1B

For the four cases that we examined, the correlation between the raw ECG signal and the digitized ECG signal ranged from 0.85–0.9. We still see a slightly reduced correlation between the two signals (ideal correlation ).

No changes in the critical clinically important features of the ECG signal due to this digitization were observed, and this is documented in the results of the Validation Test 2 that are presented next.

3. Validation Test 2

Intra-observer and inter-observer correlations of the 5 clinical parameters by the two cardiologists are presented in Table I. The correlations are generally in the range of 0.8–1 (except for QRS of the inter-observer variability), which again emphasizes that the digitized signal preserves critical clinical features from the paper ECG data. Kappa statistic calculated to be 0.85 for inter-observer agreement and 1.00 for inter-observer agreement.

Table I.

Intra-Observer and Inter-Observer Correlations for 5 ECG Parameters

Parameter	Correlation Coefficient, intra-observer	p-value	Correlation Coefficient, inter-observer	p-value
PR	0.985	<0.001	0.805	0.005
QRS	0.803	0.005	0.636	0.048
QT	0.902	<0.001	0.801	0.006
RR	0.993	<0.001	0.992	<0.001
QTc	0.901	<0.001	0.895	<0.001

The OCR process was successful in obtaining the patient demographic information, such as name and date of birth. To further improve the OCR process, the misidentification of alphabets bearing resemblance to numerals (or vice versa) can be reduced by using a template with only alphabetical (or only numeric) characters.

IV. Discussion

The signal obtained from the digitization process, as observed in our validation tests, retains the information from the original ECG signal.

In Validation Test 1, we used raw correlation between images of the original and digitized ECG, which gives a lower correlation measure. This imperfect correlation is due to the graininess in the paper ECG signal in comparison with the smoother digitized ECG signal (Fig. 5). This graininess in the source signal is likely due to decreased printer ink retention on the grid lines compared to the rest of the ECG paper. Nevertheless, the results show that digitization does not compromise the integrity of the ECG signal and the reconstructed signal is close to the signal on the paper ECG. When we correlated the raw digital ECG signals with the signal digitized from the paper ECG, the correlation was higher 0.85–0.90. However, it was still not perfect because of differences in the sampling rates of the two signals.

We could consider a modified correlation measure, which performs a column-wise fit of the digitized signal within the ECG signal, on the paper image with the same scale. This may provide a better image similarity measure. However, this correlation measure will be biased, since the digitized signal is obtained as an average of the column-wise contour of the original ECG signal.

The key feature of our digitization technique relative to known techniques is that the important features of the ECG signal, especially the QRST complex and the associated intervals, are preserved by obtaining the contour from the paper ECG. This is achieved by performing linear interpolation on the signal to be digitized ensuring that accurate reconstruction of the signal takes place, particularly along regions with sharp transitions. Of note, grid removal by thresholding does not remove any significant features from the ECG signal, in contradistinction to other techniques that use signal transforms, which may alter the signal to be digitized [16].

Another new feature of our system is the integration of the OCR block. We are currently augmenting this template library with duplicates of letters and common letter combinations that tend to bleed together when printed.

One limitation of this study is regarding the inter-observer and intra-observer correlation calculations. While intra-observer correlation is prone to bias, the 24-hour washout period should help minimize this. The inter-observer correlation is prone to variation in judgement regarding the exact start and end-point of the waveform, especially in cases of baseline artifact, as well as cases in which the transition from one interval to another is subtle. Therefore, the inter-observer variability is more likely to underestimate the true correlation significantly. Regardless, the kappa statistic was acceptable even in this condition, which further validates the transformation from scanned to digitized format.

Another limitation of this study is that we did not directly compare our digitization method with existing methods, since there are no publicly-available non-commercial tools to carry out these comparisons. The ECGScan tool [13] has an online demonstration; however, this demonstration only operates on preselected ECGs, and does not allow evaluation on external ECGs. To the best of our knowledge none of the existing tools considered the clinical ECG features, which we extracted from both paper ECGs and digitized ECGs, to evaluate the digitization quality.

V. Conclusion

We have developed a Matlab-based tool for completely digitizing paper ECG records and simultaneously extracting patient demographic information from them. We showed that the differences between the measures of clinically important features extracted from the original signal and the reconstructed signal is insignificant, thus highlighting the accuracy of this technique.

Using such an ECG digitization tool with large retrospective patient studies, which rely upon paper ECG records, will allow us to study emerging ECG features. As new clinically meaningful ECG features are discovered, digital translation of various known ECG systems into non-proprietary formats may become increasingly useful. This may then allow analysis of any ECG in custom signal analysis programs as well as commercially available digital ECG analysis programs, such as MUSE Cardiology Information System (GE Healthcare, Finland).

Biographies

Lakshminarayan Ravichandran (S'08–M'10–S'11–M'11) received the B.E. degree in electronics and communication engineering from Visvesvaraya Technological University, Karnataka, India, in 2005, and the M.S. and Ph.D. degrees in electrical engineering from Arizona State University, Tempe, AZ, USA, in 2008 and 2011, respectively. He was a Post-Doctoral Fellow with the Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA, from 2011 to 2012.

His current research interests include the areas of development of signal and image processing algorithms, analysis of biomedical data, and architectural implementation of signal processing algorithms. He received the Teaching Excellence Award from the Graduate and Professional Student Association at Arizona State University in 2011.

Chris Harless (S'11) was born in Fairfax, VA, USA, in 1990. He is currently pursuing the B.S. degree in electrical engineering from the Georgia Institute of Technology, Atlanta, GA, USA. He has plans to begin graduate education in 2014.

His current research interests include signal processing, embedded systems, and biotechnology. He is an Active Member of the Beta Mu chapter of Eta Kappa Nu with the Georgia Institute of Technology.

Amit J. Shah was born in Lexington, KY, USA, in 1979. He received the A.B. degree in physics with certificate in biophysics from Princeton University, Princeton, NJ, USA, in 2002, and the M.D. degree from the University of Pennsylvania, Phildelphia, PA, USA, in 2006. He then completed internal medicine training in 2009 with Albert Einstein Montefiore, and completed the clinical cardiology training in 2013 with Emory University. He completed the Masters of Science in Clinical Research in 2011 at Emory University. He is the recipient of many institutional and international research awards, including the Curtis Carl Johnson Award for the Best Student Poster Presentation at the18th Annual International Bioelectromagnetics Society Conference.

His current research interests include electrocardiographic signal processing, cardiac risk prediction, autonomic function, cardiac electrophysiology, and mind-body interactions.

Carson A. Wick (S'10) received the B.S. and M.S. degrees in electrical engineering in from the Georgia Institute of Technology, Atlanta, GA, USA, in 2006 and 2007, respectively, where he is currently pursuing the Ph.D. degree. In 2008 and 2009, he received the Ph.D. Student Internships with the Digitally Enhanced Analog Systems Group, Texas Instruments DSP Research and Development Center, Dallas, TX, USA.

His current research interests include digital signal processing of cardiac signals, with applications for motion analysis and tracking. He conducts his research at the Center for Signal and Image Processing at Georgia Tech and the Department of Radiology and Imaging Sciences, Emory University School of Medicine. He is a member of Eta Kappa Nu.

James H. Mcclellan (S'69–M'74–SM'79–F'85) received the B.S. degree in electrical engineering from L.S.U. in 1969, and the M.S. and Ph.D. degrees from Rice University, Houston, TX, USA, in 1972 and 1973, respectively. from 1973 to 1982, he was a Research Staff Member with the Lincoln Laboratory and a Professor at MIT. from 1982 to 1987, He was with Schlumberger Well Services. Since 1987, he has been a Professor with the School of Electrical and Computer Engineering, Georgia Tech, Atlanta, GA, USA, where he is currently holds the John and Marilu McCarty Chair.

He is the co-author of the book entitled Number Theory in Digital Signal Processing, Computer Exercises for Signal Processing, DSP First: A Multimedia Approach, and Signal Processing First, which received the McGraw-Hill Jacob Millman Award for an Outstanding Innovative Textbook in 2003. In 1998, he received the W. Howard Ector Outstanding Teacher Award at Georgia Tech, and in 2001, the Education Award from the IEEE Signal Processing Society. In 1987, he received the Technical Achievement Award for work on FIR filter design, and in 1996, the Society Award, both from the IEEE Signal Processing Society. In 2004, he was a co-recipient of the IEEE Jack S. Kilby Signal Processing Medal. He is a member of Tau Beta Pi and Eta Kappa Nu.

Srini Tridandapani (S'86–M'95–SM'12) received the B.E. degree from Anna University, Madras, India, and the M.S.E.E. and Ph.D. degrees from the University of Washington, Seattle, WA, USA, all in electrical engineering. After postdoctoral training of computer science with the University of California, Davis, CA, USA, he was an Assistant Professor of electrical and computer engineering with Iowa State University, Ames, IO, USA. He then took the bold plunge into medical school and received the M.D. degree from the University of Michigan, Ann Arbor, MI, USA, followed by residency training in Radiology at Michigan. He then obtained clinical fellowships in Cardiothoracic Imaging and Abdominal Imaging at Emory University. He recently completed the Masters degree in clinical and translational research at Emory University.

He is currently a Faculty Member with the Department of Radiology and Imaging Sciences at Emory University and an Adjunct Professor with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA. His current research interests include the development of novel gating strategies for optimizing cardiac computed tomography and innovative tools to increase patient safety in medical imaging.