The Consortium for Electrocardiographic Imaging

Abstract

Electrocardiographic imaging (ECGI) has recently gained attention as a viable diagnostic tool for reconstructing cardiac electrical activity in normal hearts as well as in cardiac arrhythmias. However, progress has been limited by the lack of both standards and unbiased comparisons of approaches and techniques across the community, as well as the consequent difficulty of effective collaboration across research groups.. To address these limitations, we created the Consortium for Electrocardiographic Imaging (CEI), with the objective of facilitating collaboration across the research community in ECGI and creating standards for comparisons and reproducibility.

Here we introduce CEI and describe its two main efforts, the creation of EDGAR, a public data repository, and the organization of three collaborative workgroups that address key components and applications in ECGI. Both EDGAR and the workgroups will facilitate the sharing of ideas, data and methods across the ECGI community and thus address the current lack of reproducibility, broad collaboration, and unbiased comparisons.

1. Introduction

Electrocardiographic imaging (ECGI) is an idea with a history of over forty years [1–4] that has recently gained attention as a viable diagnostic tool for reconstructing cardiac electrical activity in normal hearts [5] as well as in cardiac arrhythmias[6–8], especially as an aide to treating atrial fibrillation (AF) [9,10] and premature ventricular contractions (PVC)[11]. ECGI requires body-surface electrocardiogram recordings, usually with high spatial coverage and density, referred to as body surface potential mapping (BSPM), together with a geometric model, typically derived from medical images [12], and a computational implementation of a numerical solution to an ill-posed inverse problem [3]. Perhaps because of the diverse and specialized knowledge and experience required to pursue research in ECGI, progress has typically been tied to collaborative research among mathematicians, physicists, biomedical engineers, computer scientists, and cardiologists. Industry is now also playing a major role in the advancement of ECGI technology and translation discovery to clinical practice.

A major obstacle to ECGI research has always been the lack of both standards and unbiased comparisons of approaches and techniques across the community. Collaborations have typically involved two or three partners and the details of the associated measurements, numerical methods, computational implementations, and validation data have remained either proprietary or very difficult to replicate, even across academic centers. Publications have documented progress but cannot fully describe essential solution components in sufficient detail to allow meaningful comparison or confirmation of the results. With many agencies, including the US National Institutes of Health, expressing justified concern at the lack of reproducibility of biomedical science[13], it is essential to pursue meaningful sharing of all elements of ECGI. Only through such broadly scaled collaboration and sharing will it be possible to provide reasonable estimates of accuracy and the bounds of uncertainty of ECGI. Even with the best of intentions, however, sharing of data and techniques is a challenging and resource consuming task and the ECGI field has lacked a consolidated plan for such collaborative infrastructure. To overcome this limitation, we created the Consortium for Electrocardiographic Imaging (CEI).

The first formal discussions for the CEI began at the Computing in Cardiology conference (CinC) 2014 in Boston, USA. During those discussions, we identified the interest within the community in such a structure and started the organization of a series of events that would be the foundations of CEI. The first event, organized by investigators at the Karlsruhe Institute of Technology (KIT), was held in March 2015 in Bad Herrenalb, Germany, and had broad international participation from most of the research groups working on ECGI. That meeting was organized as an ECGI conference where groups presented their newest developments and discussed their perspectives about ECGI and CEI. During this meeting we also created a mailing list (with over 70 members) and introduced the CEI data repository: the “Experimental Data and Geometric Analysis Repository: (EDGAR)” [14]. At that time it consisted of 3 datasets, but it already established the data sharing standard that we currently use.

The next CEI event took place during CinC 2015 in Nice, France. There, we held meetings before and during the conference in dedicated rooms provided by the CinC organizers, introduced the new CEI website (www.ecg-imaging.org), reported on the progress of the EDGAR repository (now available online via the CEI web-site) and presented the idea of organizing collaborative workgroups to tackle specific challenges in ECGI research. We initiated the Forward Modeling Workgroup and the Premature Ventricular Contraction Workgroup and, following the open access philosophy of CEI, we discussed about the objectives, datasets and validation metrics to be used. The initial guidelines defined in those meetings, were refined in follow-up online meetings and resulted in the frameworks that we describe in Section 3.

Since then, CEI has continued increasing visibility of ECGI with presence in other conferences such as the International Symposium of Biomedical Imaging (ISBI) and the International Society for Computerized Electrocardiology (ISCE). An executive committee has been meeting regularly to advance the organization of EDGAR and the workgroups.

At CinC 2016 we expanded the consortium with the addition of new members and presented the latest updates on EDGAR and the workgroups. These include the creation of a web-based interface to participate in the work-groups, based on the “Covalic” system, and the addition of an atrial arrhythmias dataset and the creation of a work-group to address this application of ECGI. In addition, CEI impacted the CinC meeting itself, including a special session on ECGI for atrial arrythmias and several talks featuring collaborative use of data from EDGAR.

2. EDGAR Data Repository

We created the data repository EDGAR to allow researchers to test their new algorithms and methods such that their findings can be reproduced by other groups. The working philosophy of EDGAR is to make available a wide range of data examples, all de-identified by the donators and hence freely available to others. The online repository currently contains data from 6 different institutions around the world. It includes data from animal experiments, simulations, and human studies and from various clinical applications such as premature ventricular contractions, ventricular tachycardia, location of scar tissue and ishemia detection (see Section 4 for upcoming additions). Each study may contain medical images, segmentations of organs, and surface and volume polygonal meshes, but must always include time signals linked to locations in a geometric model that represent the cardiac bioelectric sources and/or the body surface potentials, i.e. the minimal information needed to perform validation studies. In many cases, data in EDGAR are just exemplars from a larger dataset acquired by the ”data donor”. Researchers interested in access to a larger dataset are expected to contact the donator of the data to arrange a suitable collaboration.

To facilitate the exchange of data, all datasets in EDGAR follow a flexible standard that allows researchers in various groups to easily load the data into their existing pipelines. This standard is described by the met-dadata model that specifies the five modules that form a dataset: Time Signals, Geometric Models, Forward & Inverse Transforms, Registration Information, and Medical Images [14].

3. Workgroups

The workgroups are an initiative designed to initiate and facilitate collaboration among groups and to address key current components and applications of ECGI. They establish common frameworks in which researchers can test their algorithms and methods with standarized datasets and validation metrics. There are three ongoing workgroups, the forward modeling workgroup, which seeks to identify the role of geometric model construction in error and variability of inverse solutions, and two workgroups focusing on clinical applications: localization of the originating sites of premature ventricular contractions and the reconstruction of atrial arrhythmias.

To support the quantitative comparisons of results, we have employed the Covalic Challenge system (https://challenge.kitware.com), which is a web based, open access platform for providing training and test data sets along with automated support for metrics to compare results across submissions. We sought not to create formal challenges (the original context of Covalic) but to adapt Covalic as a collaborative research tool. Two of the work-groups described in the following can already be found as challenges on the Covalic website as “Dalhousie Segmentation” and “CEI Pacing Site Localization”.

3.1. Forward Modeling Workgroup

Solving what is known as the “forward problem”, that is the generation of a forward model that relates the electrical activity on the heart to body surface measurements, is a necessary step to carry out ECGI. Although this problem is typically assumed to be solved during the design of new ECGI methods, each group typically has their own practices, methods and software solutions to resolve this problem. Models differ not only in terms of source description (and thus form and boundary conditions of the partial differential equation to be solved) but also in geometric model parameters such as which organs to include or what conductivities to specify. Moreover, there is variability in segmentation of the structural images obtained from MRI or CT scans and any subsequent mesh generation. All these variations across groups unavoidably introduce uncertainty into the ECGI results. The objective of the forward modeling workgroup is to resolve part of this uncertainty by quantifying the variability in the pipelines used to generate forward models, from the segmentation step to the computation of the forward model. The first step that this workgroup will address is quantification of variability in segmentation of the organs from structural images.

To contribute, participants can access the image data from Covalic and then upload their resulting segmentations, which Covalic will automatically compare to a common reference and from which we will generate a community average segmentation. This segmentation repository will not only allow each group to determine how their segmentations differ from the group average but also may enable the creation of realistic statistical models that characterize the variability across segmentation pipelines. Once we reach a reasonable level of participation in this step, we will proceed with the following phases in the forward model pipeline.

3.2. Premature Ventricular Contraction Reconstruction Workgroup

Non-invasive imaging of PVCs is of interest since it can help pre-procedure planning of the ablation interventions and reduce the associated duration, risks and costs. The main objective of this imaging problem is to localize the site of premature excitation on the ventricle during a PVC.

Many groups have reported on solutions to this problem and most have reported the localization error of the site of premature excitation. Others also include results on reconstructed heart potentials, activation times and sometimes on clinical outcome (ablation success, number of ablation sites or duration of the intervention). Although there are many such publications, it is hard to track the progress of the community since each group reports on different subjects, uses different pipelines and has varying quality and completeness of the ground truth. Thus, the objective of the PVC workgroup is to create benchmarks that allow for fair and repeatable testing of ECGI algorithms.

The first benchmark is based on a dataset contributed to EDGAR by the KIT group. This dataset consists of both synthetic and real electrical measurements from PVCs of a human subject, combined with multiple mesh geometries, for various source models and numerical solutions. In this benchmark, there is a different “phase” (the nomenclature used by Covalic) per geometry type, where each user can download the BSPM, the torso and heart geometries, and a forward matrix. The separation of geometry types into different “phases” allows for fair comparison between methods using different source models and levels of complexity of the geometry (epicardial surface, epicardial + endocardial surface or full volumetric representation). After computing the inverse solutions with the method of choice, each group can then upload the solutions to the Covalic system in the form of heart potentials, activation times and/or 3D location of the site of premature excitation and it will automatically compute a suite error metrics and publicly report the results. Specifically, Covalic has been configured to evaluate uploaded solutions with the most widely used metrics: relative error of the inverse reconstruction of the heart potentials, correlation between the estimated and true activation times and localization error of the site of premature excitation measured in Euclidean distance (in millimeters).

With this structure, we hope to set a validation standard that is shared among the community and with which we can make ongoing evaluations of the state-of-the-art in non-invasive PVC localization. After the structure is established, we aim to expand this validation framework to new datasets and metrics that will allow a more complete comparison.

3.3. Atrial Arrhythmias Workgroup

The latest addition to the workgroups is the atrial arrhythmias workgroup. Atrial arrhythmias, and particularly atrial fibrillation (AF), are a topic of great clinical interest due to their high prevalence and the lack of other imaging techniques that characterize them with the promise of improved ablation protocols. ECGI of atrial arrhythmias is an open and challenging problem in the research community due to the low signal-to-noise ratio and complex behavior of the signals involved. Many researchers have proposed different imaging techniques to characterize the arrhythmias. However, to determine the clinical value of these methods it is first necessary to validate the quality of their results.

The objective of the atrial arrhythmias workgroup is to create a standarized testing environment that allows for fair and repeatable testing of ECGI algorithms applied to the imaging of the atria. Due to the distinct behaviour of different arrhythmias, this workgroup will focuss on three: tachycardia, flutter and fibrillation.

We are still forming the specific datasets and metrics for this workgroup. We are currently preparing a dataset containing examples of these types of arrhythmias that will be uploaded to EDGAR. However, as we did with the other workgroups, we want to involve as many groups as possible in the design of this workgroup. For this reason, during the meetings around CinC 2016 we identified research groups that will contribute to the discussion about datasets, measures of interest and the corresponding validation metrics. Tentative plans for metrics include are the estimation of activation patterns, location of premature excitation site (for atrial tachycardia or atrial flutter), estimation of dominant frequencies, and localization of rotors and sites with phase singularities or areas of low voltage.

Due to the high level of interest clinically, comercially, and within the ECGI community on this topic, we expect this workgroup to grow quickly and become a reference benchmark for validation of imaging techniques of arrhythmogenic behavior in the atria.

4. Future Work

In its two years of existence, CEI has made much progress in bringing researchers together as well as facilitating the validation and reproducibility of ECGI methods. However, there is still much to be done. The EDGAR repository continues to grow, with recent inclusion of a dataset composed of atrial signals from the University of Valencia and an example case provided by a group at Radboud University.

We will continue to expand the workgroups while identifying and correcting their limitations. These initiatives will form a continuous evaluation of the field, which is expected to yield collaborative and consensus publications reporting on the state-of-the-art of ECGI.

We welcome any group or individual interested in joining our efforts, contributing new datasets or just suggesting new ideas to contact us at cei-info@ecg-imaging.org.