Abstract
Background:
Timely and accurate discrimination of wide complex tachycardias (WCTs) into ventricular tachycardia (VT) or supraventricular WCT (SWCT) is critically important. Previously we developed and validated an automated VT Prediction Model that provides a VT probability estimate using the paired WCT and baseline 12-lead ECGs. Whether this model improves physicians’ diagnostic accuracy has not been evaluated.
Objective:
We sought to determine whether the VT Prediction Model improves physicians’ WCT differentiation accuracy.
Methods:
Over four consecutive days, nine physicians independently interpreted fifty WCT ECGs (25 VTs and 25 SWCTs confirmed by electrophysiological study) as either VT or SWCT. Day 1 used the WCT ECG only, Day 2 used the WCT and baseline ECG, Day 3 used the WCT ECG and the VT Prediction Model’s estimation of VT probability, and Day 4 used the WCT ECG, baseline ECG, and the VT Prediction Model’s estimation of VT probability.
Results:
Inclusion of the VT Prediction Model data increased diagnostic accuracy versus the WCT ECG alone (Day 3: 84.2% vs. Day 1: 68.7%, p 0.009) and WCT and baseline ECGs together (Day 3: 84.2% vs. Day 2: 76.4%, p 0.003). There was no further improvement of accuracy with addition of the baseline ECG comparison to the VT Prediction Model (Day 3: 84.2% vs. Day 4: 84.0%, p 0.928). Overall sensitivity (Day 3: 78.2% vs. Day 1: 67.6%, p 0.005) and specificity (Day 3: 90.2% vs. Day 1: 69.8%, p 0.016) for VT were superior after the addition of the VT Prediction Model.
Conclusion:
The VT Prediction Model improves physician ECG diagnostic accuracy for discriminating WCTs.
Keywords: wide complex tachycardias, ventricular tachycardia, supraventricular wide complex tachycardia, electrocardiogram, ECG interpretation, ECG competency
INTRODUCTION
Critical patient management decisions rely on timely and accurate wide complex tachycardia (WCT) discrimination into ventricular tachycardia (VT) or supraventricular WCT (SWCT). Erroneous VT or SWCT diagnoses may lead to inappropriate medical decisions and ultimately result in unfavorable patient outcomes, including use of potentially harmful medications (1,2). Furthermore, a missed diagnosis of VT may result in delays in diagnostic workup and suboptimal treatment strategies. Conversely, an erroneous VT diagnosis, instead of SWCT, can lead to improper therapies including external defibrillation and implantable cardioverter defibrillator (ICD) placement.
Given the high stakes, the problem of discriminating VT from SWCT has been long examined for an improved ECG based solution. Over the last 50–60 years, a plethora of manually applied ECG criteria and algorithms have been proposed (3–13). While these algorithms can potentially improve discrimination of VT from SWCT, each has relevant diagnostic limitations that pose a challenge for clinical implementation (14–20).
The crux of the problem has been the manual application of ECG algorithms by providers with varied expertise levels. Our group has developed novel diagnostic solutions that enable accurate and automated WCT discrimination by CEI software (21–24). One such approach is the VT Prediction Model, which is a novel algorithm capable of automatically delivering VT probability estimates using readily accessible ECG data routinely displayed on 12-lead ECGs (22). This algorithm has demonstrated highly effective VT and SWCT discrimination for WCT ECGs expected to be encountered in clinical practice. However, whether automated diagnostic tools like the VT Prediction Model help physicians differentiate VT and SWCT remains unclear. Therefore, we sought to determine the impact of VT Prediction Model use on physicians’ WCT discrimination accuracy and ECG interpretation confidence.
METHODS
i. ECG selection
Standard 12-lead ECG recordings (paper speed: 25 mm/s, voltage: 10 mm/mV) demonstrating WCT were selected from patients who presented to the Mayo Clinic Health System of Southeastern Minnesota between September 2011 and November 2016, with a corroborating electrophysiological (EP) study or procedure confirming the VT or SWCT diagnosis. This represented a total of 213 patients with an EP study who had paired WCT and baseline ECGs available. A total of fifty non-consecutive WCTs (i.e., 25 VTs and 25 SWCTs) from 50 patients were chosen for this analysis. All ECGs were recorded using the GE-Marquette 12SL ECG analysis program (GE Healthcare, Milwaukee, WI, USA). WCT ECGs were required to demonstrate a wide QRS complex (≥ 120 ms) and rapid ventricular rate (≥ 100 beats/minute). WCT ECGs were also required to possess an official ECG laboratory interpretation of “ventricular tachycardia,” “supraventricular tachycardia”, or “wide complex tachycardia.” All selected WCT ECGs were required to have a paired baseline ECG, which was defined as the most proximate ECG that no longer fulfilled WCT criteria.
ii. VT Prediction Model
The VT Prediction Model was designed as an automated tool to facilitate successful WCT discrimination independent of clinicians’ manual application of traditional ECG criteria or algorithms. The VT Prediction Model reports an estimation of VT probability based on a logistic regression model composed of computerized ECG measurements and calculations derived from paired WCT and baseline ECGs (Supplemental Figure S1): WCT QRS duration, QRS duration change, QRS axis change, and T wave axis change.
The VT Prediction Model requires the input of 3 computerized ECG measurements routinely displayed on 12-lead ECG paper recordings from both the WCT ECG and baseline ECG: QRS duration, QRS (R wave) axis, and T wave axis. Using these 6 variables, the VT Prediction Model reports an estimation of VT probability (i.e., likelihood of VT). Thus, successful VT Prediction Model application requires computerized data from paired WCT and baseline ECGs, which is an important limitation. Thus, unless computerized data from a baseline ECG is available the VT Prediction Model cannot be applied to a given WCT.
Details pertaining to the VT Prediction Model logistic regression structure, derivation, and validation are described in prior works (22,25). In our original report (22), VT Prediction Model application achieved effective WCT discrimination (AUC 0.924). Upon using a 50% VT probability cut-off to adjudicate VT or SWCT diagnoses (i.e., VT for ≥ 50% VT probability; SWCT for < 50% VT probability), the VT Prediction Model yielded an overall diagnostic accuracy, sensitivity, and specificity for VT of 84.9%, 80.6%, and 88.4%, respectively. The VT Prediction Model’s diagnostic performance did not differ whether it was applied to patients with or without an accompanying EP procedure.
iii. VT Prediction Model application on selected ECGs
The VT Prediction Model was applied to the 50 selected WCT ECGs (i.e., 25 VTs and 25 SWCTs) to generate its estimation of VT probability (0.0% – 100.0%). Using a 50% VT probability cut-off to establish VT or SWCT diagnoses (i.e., VT for ≥ 50% VT probability; SWCT for < 50% VT probability) in this sample, the VT Prediction Model’s overall diagnostic accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for VT were 86.0%, 84.0%, 88.0%, 87.5%, and 84.6%, respectively.
iv. Participating physician interpreters
Nine physicians belonging to a Mayo Clinic (Rochester, MN) training program (6 internal medicine residents and 3 cardiology fellows) volunteered to participate in this study, and completed all 4 days of the study protocol.
v. ECG interpretation session protocol
Physician interpreters were asked to independently discriminate WCT ECGs into VT or SWCT over the course of four separate ECG interpretations sessions, which were conducted over consecutive days (Days 1 through 4). Participants were asked to report their level of confidence (i.e., ‘confident’, ‘somewhat confident’, or ‘not confident’) for each ECG they interpreted.
Figure 1 outlines the differences in ECG interpretation protocol according to ECG interpretation day (i.e., Days 1 through 4). On Day 1, physicians were provided only the WCT ECG; on Day 2, they were provided the WCT and baseline ECGs; on Day 3, they were provided the WCT ECG and VT Prediction Model’s VT probability estimate; and, on Day 4, they were provided the WCT ECG, baseline ECG, and VT Prediction Model’s estimate of VT probability.
Figure 1: Overview of four-day virtual study protocol.
Notice the variations of information provided to interpreters on each study day. The VT Prediction Model’s probability estimate was pre-calculated for interpreters and only included on Days 3 and 4.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
Figure 2 describes the organizational framework applied to each ECG interpretation session. For each ECG interpretation day, a total of 70 WCT ECGs were interpreted. ECG interpreters were first asked to interpret 50 unique WCT ECGs (i.e., 25 VTs and 25 SWCTs) presented in sequential order (i.e., ECG interpretations 1 through 50). After interpreting the first 50 ECGs (i.e., ECG interpretations 1 through 50), interpreters were asked to interpret an additional 20 WCT ECGs that were randomly selected out of the previously interpreted 50 WCT ECGs (i.e., ECG interpretations 51 through 70). The sequential order the first 50 WCT ECGs were presented was randomized each interpretation day. Participants were not made aware that a random selection (i.e., 10 VTs and 10 SWCTs), derived from the first 50 WCT ECGs, would be interpreted more than once, and they were kept unapprised of the fact that the last 20 WCT ECGs (i.e., ECG interpretations 51 through 70) were composed of previously interpreted WCT ECGs.
Figure 2: WCT ECG distribution for daily diagnostic and rater agreement assessment.
All WCT ECGs were randomized each day. WCT ECGs for rater agreement assessment represented an equal split amongst diagnoses (i.e., 10 VTs and 10 SWCTs) and were randomly selected each day.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia; SWCT, supraventricular wide complex tachycardia.
ECG interpretation sessions were organized remotely via a video conference platform (Zoom by Zoom Video Communications, Inc.), with each session proctored by the first author (A.H.K.), who explained each session’s organizational framework and ECG interpretation instructions to the participants. ECGs were presented individually, in sequential order (1 through 70), allowing 60 seconds for ECG interpretation. Interpreters were not allotted additional time to view ECGs after 60 seconds elapsed, and they were not allowed to revisit previously interpreted ECGs. Participants were not permitted to confer with each other during the interpretation sessions. This study was open label by nature of its protocol; however, the overall intent and design of the study was withheld from the participants as the study protocol unfolded (e.g., participants did not know what was to occur before each day of the study). Participants were not provided with answers or performance feedback until study protocol completion (i.e., after Day 4).
The VT Prediction Model’s VT probability estimates were calculated prior to study protocol initiation. For ECG interpretation Days 3 and 4, VT probability estimates were presented to the participating interpreters as a percentage of VT probability conspicuously displayed on the 12-lead ECG recording of the WCT. Details regarding the VT Prediction Model’s diagnostic performance for WCT ECGs included in this study was not provided to the interpreters.
vi. Assessment of physician ECG interpretation accuracy and confidence
Physician ECG interpretation accuracy was assessed according to their agreement with the actual underlying diagnosis (i.e., VT or SWCT). Furthermore, ECG interpreter accuracy (i.e., correct or incorrect) was linked to the level of ECG interpretation confidence (i.e., ‘confident’, ‘somewhat confident’, or ‘not confident’), as defined by the ECG interpreter. There were six possible outcomes for each interpreted ECG: (i) correct and confident (C|C), (ii) correct and somewhat confident (C|S), (iii) correct and not confident (C|N), (iv) incorrect and confident (I|C), (v) incorrect and somewhat confident (I|S), or (vi) incorrect and not confident [I|N]).
vii. Surveys
Physicians were asked to complete a brief survey before study protocol initiation (i.e., baseline survey) and after each ECG interpretation session (i.e., post-session surveys) (Supplemental Table S1).
viii. Outcomes of interest and statistical analysis
The primary outcomes of interest were overall diagnostic performance and the level of ECG interpretation confidence. Diagnostic performance was compared to the gold standard diagnosis (i.e., VT or SWCT) established by an EP study or procedure. ECG interpreter diagnostic performance was compared using unadjusted and Bonferroni-adjusted p-values. Secondary outcomes of interest included (i) inter-rater agreement based off the first 50 WCT ECGs interpreted (i.e., ECG interpretations 1 through 50) for each interpretation day and (ii) intra-rater agreement based off the last 20 WCT ECGs that were repeated (i.e., ECG interpretations 51 through 70) for each interpretation day. Intra-rater agreement was assessed for each interpreter using Cohen’s kappa coefficient. Inter-rater agreement was also evaluated between individual interpreters using Cohen’s kappa coefficient and collectively using Krippendorff’s alpha coefficient (26–28).
ix. Disclosures
Anthony Kashou, Peter Noseworthy, Christopher DeSimone, Abhishek Deshmukh, and Adam May are considered potential beneficiaries of intellectual property discussed in this work. The remaining authors have no disclosures to report.
x. Funding
This work was conceived, executed, and supported by Department of Cardiovascular Diseases Mayo Clinic in Rochester, MN. The authors also acknowledge support by NIH T32 HL007111. Patient data was solely used from individuals who provided consent to use of their anonymized records for research. The Mayo Clinic Institutional Review Board approved this work.
RESULTS
i. Baseline survey
Six internal medicine residents and three cardiology fellows participated in the study. Of the six internal medicine residents, four were in their first post-graduate year (PGY) and two were in their second PGY. Two cardiology fellows were in their first clinical year of fellowship (PGY-4) and one cardiology fellow was in the second clinical year (PGY-6). All residents were interested in cardiology as a career and all cardiology fellows had an interest in ECG interpretation. With regard to time dedicated to learning WCT differentiation prior to study participation: one resident had never dedicated time, two residents and one fellow dedicated 0–2 hours, two residents and one fellow dedicated 2–5 hours, one resident dedicated 5–10 hours, and one fellow dedicated more than 10 hours.
ii. Diagnostic performance
Table 1 summarizes the physicians’ diagnostic performance for each ECG interpretation day (i.e., Days 1 through 4). Individual and composite accuracies for each interpretation day are presented in Supplemental Table S2. Overall accuracy when the model prediction data was first introduced (Day 3, 84.2%) was superior to Day 1 (68.7%, p < 0.0001) and Day 2 (76.4%, p 0.0033), with no further improvement on Day 4 (84.0%, p 0.9281). Overall sensitivity for VT on Day 3 (78.2%) was superior to Day 1 (67.6%, p 0.0003), similar to Day 2 (76.0%, p 0.4282), and inferior to Day 4 (84.4%, p 0.0167). Overall specificity for VT on Day 3 (90.2%) was superior to Day 1 (69.8%, p < 0.0001), Day 2 (76.9%, p < 0.0001), and Day 4 (83.6%, p 0.0031). PPV for VT on Day 3 (88.9%) was superior to Day 1 (69.1%, p < 0.0001), Day 2 (76.7%, p < 0.0001), and Day 4 (83.7%, p 0.0237). NPV for VT on Day 3 (80.6%) was superior to Day 1 (68.3%, p < 0.0001) and similar to Day 2 (76.2%, p 0.1131) and Day 4 (84.3%, p 0.1407).
Table 1:
Physicians’ diagnostic performance in differentiating WCTs for each day.
| Study Day | Interpreter (# ECGs assessed) | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) |
|---|---|---|---|---|---|---|
|
Day 1: WCT ECG only |
All physicians (450) | 68.67 (64.38–72.95)b,c,d | 67.56 (63.23–71.88)b,c,d | 69.78 (65.53–74.02)b,c,d | 69.09 (64.82–73.36)b,c,d | 68.26 (63.96–72.56)b,c,d |
| Residents (300) | 64.67 (59.26–70.08)b,c,d | 64.00 (58.57–69.43)b,c,d | 65.33 (59.95–70.72)b,c,d | 64.86 (59.46–70.27)b,c,d | 64.47 (59.06–69.89)b,c,d | |
| Fellows (150) | 76.67 (69.90–83.44)c | 74.67 (67.71–81.63)d | 78.67 (72.11–85.22)c | 77.78 (71.12–84.43)c | 75.64 (68.77–82.51) | |
|
Day 2: WCT ECG + Baseline ECG |
All physicians (450) | 76.44 (72.52–80.37)a,c,d | 76.00 (72.05–79.95)a,d | 76.89 (72.99–80.78)a,c,d | 76.68 (72.77–80.59)a,c,d | 76.21 (72.52–80.37)a,d |
| Residents (300) | 73.67 (68.68–78.65)a,c,d | 74.00 (69.04–78.96)a,d | 73.33 (68.33–78.34)a,c,d | 73.51 (68.52–78.50)a,d | 73.83 (68.85–78.80)a,d | |
| Fellows (150) | 82.00 (75.85–88.15) | 80.00 (73.60–86.40) | 84.00 (78.13–89.87) | 83.33 (77.37–89.30) | 80.77 (74.46–87.08) | |
|
Day 3: WCT ECG + VT Prediction Model |
All physicians (450) | 84.22 (80.85–87.59)a,b | 78.22 (74.41–82.04)a,d | 90.22 (87.48–92.97)a,b,d | 88.89 (85.99–91.79)a,b,d | 80.56 (76.90–84.21)a |
| Residents (300) | 83.00 (78.75–87.25)a,b | 76.00 (71.17–80.83)a,d | 90.00 (86.61–93.39)a,b,d | 88.37 (84.74–92.00)a,b,d | 78.95 (74.33–83.56)a | |
| Fellows (150) | 86.67 (81.23–92.11)a | 82.67 (76.61–88.72) | 90.67 (86.01–95.32)a | 89.86 (85.02–94.69)a | 83.95 (78.08–89.82)a | |
|
Day 4: WCT ECG + Prediction Model |
All physicians (450) | 84.00 (80.61–87.39)a,b | 84.44 (81.10–87.79)a,b,c | 83.56 (80.13–86.98)a,b,c | 83.70 (80.29–87.11)a,b,c | 84.30 (80.94–87.67)a,b |
| Residents (300) | 83.33 (79.12–87.55)a,b | 84.67 (80.59–88.74)a,b,c | 82.00 (77.65–86.35)a,b,c | 82.47 (78.16–86.77)a,b,c | 84.25 (80.12–88.37)a,b | |
| Fellows (150) | 85.33 (79.67–90.99) | 84.00 (78.13–89.87)a | 86.67 (81.23–92.11) | 86.30 (80.80–91.80) | 84.42 (78.61–90.22) |
Abbreviations WCT = wide complex tachycardia, VT = ventricular tachycardia, PPV = positive predictive value, NPV = negative predictive value
Numbers in parentheses are the 95% confidence intervals.
p-value <0.05 for Day 2, 3, or 4 when compared to Day 1
p-value <0.05 for Day 1, 3, or 4 when compared to Day 2
p-value <0.05 for Day 1, 2, or 4 when compared to Day 3
p-value <0.05 for Day 1,2, or 3 when compared to Day 4
Figures 3 and 4 illustrate the differences in overall ECG interpretation accuracy between interpretation days (i.e., Day 1 vs. Day 3 and Day 2 vs. Day 4) according to VT probability estimate subgroups (i.e., VT probability < 30%, 30–70%, and >70%), as assigned by the VT Prediction Model. Compared to Day 1, Day 3 demonstrated in increase in overall accuracy across all VT probability subgroups. Compared to Day 2, Day 3 demonstrated in increase in overall accuracy across all VT probability subgroups.
Figure 3: Overall Accuracy according VT Prediction Model Estimates: Day 1 vs. Day 3.
Overall accuracy increased between Day 1 and Day 3 irrespective of VT Probability range (i.e., <30%, 30–70%, and >70%).
Day 1: WCT ECG only
Day 3: WCT ECG + VT Prediction Model
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
Figure 4: Overall Accuracy according VT Prediction Model Estimates: Day 2 vs. Day 4.
Overall accuracy increased between Day 2 and Day 4 irrespective of VT Probability range (i.e., <30%, 30–70%, and >70%).
Day 2: WCT ECG + baseline ECG
Day 4: WCT ECG + baseline ECG + VT Prediction Model
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
On Day 1, mean interpretation accuracy for residents was inferior to cardiology fellows (residents 65.6% vs. fellows 77.3%, p 0.0111). On interpretation Days 3 and 4, the mean interpretation accuracy for residents was not significantly different from that of the cardiology fellows (Day 3: residents 83.0% vs. fellows 86.7%, p 0.3146; Day 4: residents 83.3% vs. fellows 85.3%, p 0.5858).
iii. ECG interpretation accuracy and confidence
Figure 5 illustrates the distribution of diagnostic accuracy and confidence categories for each of the four study days (i.e., Days 1 through 4). Compared to Day 1, Day 3 demonstrated a greater incidence of C|C diagnoses (absolute increase 17.1%, relative increase 74.7%, p < 0.001), decreased incidence of I|C diagnoses (absolute decrease 4.0%, relative decrease 64.3%, p 0.003), and decreased incidence of any non-confident diagnosis (i.e., C|N plus I|N diagnoses) (absolute decrease 14.2%, relative decrease 39.5%, p < 0.001). Compared to Day 2, Day 4 demonstrated greater incidence of C|C diagnoses (absolute increase 22.2%, relative increase 78.7%, p < 0.001), comparable incidence of I|C diagnoses (absolute decrease 0.4%, relative decrease 10.4%, p 0.736), and decreased incidence of any non-confident diagnosis (absolute decrease 17.1%, relative decrease 53.0%, p < 0.001).
Figure 5: Physicians’ diagnostic accuracy and confidence in differentiating WCTs.
Day 1: WCT ECG only
Day 2: WCT ECG + baseline ECG
Day 3: WCT ECG + VT Prediction Model
Day 4: WCT ECG + baseline ECG + VT Prediction Model
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
iv. Rater-interpretation agreement
Assessment of intra-rater interpretation agreement with 20 duplicated WCT ECGs (i.e., ECG interpretations 51 through 70) on each interpretation day is presented in Supplemental Table S3. Intra-rater agreement was highest on Days 3 and 4 compared to Days 1 and 2.
Assessment of inter-rater interpretation agreement for the first 50 WCT ECGs interpreted on each interpretation day (i.e., ECG interpretations 1 through 50) is presented in Supplemental Table S4. Among residents, inter-rater agreement was 0.266 for Day 1, 0.423 for Day 2, 0.705 for Day 3, and 0.644 for Day 4. Among cardiology fellows, inter-rater agreement was 0.389 for Day 1, 0.522 for Day 2, 0.722 for Day 3, and 0.682 for Day 4. Overall composite inter-rater agreement among all interpreters was 0.282 for Day 1, 0.457 for Day 2, 0.728 for Day 3, and 0.667 for Day 4.
v. Post interpretation day surveys
Responses to survey questions are presented in Supplemental Table S5. Day 1 was regarded as the most challenging interpretation day. Interpreters rated interpretation Day 4 to be comparatively easier than Days 1, 2, and 3. Interpreters also felt that 60 seconds was sufficient time for ECG interpretation.
DISCUSSION
To our knowledge, this was the first study to prospectively evaluate the diagnostic impact of an automated WCT discrimination model on ECG interpretation performance. Furthermore, this was the first study to elucidate how an automated model’s practical application can improve user’s diagnostic performance in discriminating VT and SWCT. The major findings from this study are that use of the VT Prediction Model, when compared to use of the WCT ECG alone, improved (i) physician overall diagnostic accuracy, (ii) sensitivity, specificity, PPV, and NPV for VT diagnoses, and (iii) interpreter confidence in diagnosis. Additionally, we observed that VT Prediction Model use was a more useful than the addition of the baseline ECG for improving WCT differentiation accuracy.
If our study results were reproduced in clinical practice, the VT Prediction Model use would decrease the risk of clinicians inappropriately diagnosing SWCT as VT (i.e., over-diagnosis of VT) and VT as SWCT (i.e., missed VT diagnosis). As such, physicians would more accurately differentiate VT and SWCT, thereby enabling the most appropriate clinical management decisions for patients presenting with WCT. Thus, it is conceivable that VT Prediction Model use can reduce the likelihood of exposing patients with VT to harmful medical treatments (e.g., calcium-channel blockers) and other unsafe clinical decisions (e.g., dismissal to home) due to an erroneous SWCT diagnosis. Additionally, VT Prediction Model use could help avoid problematic downstream consequences after wrongly diagnosing VT for an actual SWCT, including use of medications with long-term toxicity (e.g., amiodarone), unnecessary utilization of healthcare resources (e.g., intensive care unit admission), or inappropriate long-term therapeutic strategies (e.g., ICD implantation).
Improved ECG interpretation confidence
In addition to improved ECG interpretation accuracy, VT Prediction Model use favorably impacted ECG interpretation confidence in several important ways. Our results show that Days 3 and 4, which incorporated the VT probability estimates derived from the VT Prediction Model, demonstrated favorable changes in physician interpretation confidence compared to Days 1 and 2, respectively. First, VT Prediction Model use led to an increased proportion of WCTs that were accurately and confidently differentiated into VT or SWCT (i.e., greater C|C diagnoses), which has the presumed advantage of promoting timely and accurate clinical decision-making. Second, VT Prediction Model use led to a reduction in WCTs that were incorrectly but confidently diagnosed (i.e., less I|C diagnoses), which would potentially help reduce serious clinical errors (e.g., administration of calcium-channel blockers for patients with VT). Lastly, VT Prediction Model use was found to reduce the proportion of WCTs to which a confident diagnosis was not able to be established (i.e., C|N and I|N diagnoses), which could translate into more timely therapeutic decisions and potentially less costs from healthcare resource utilization.
i. Prospective clinical applications
At present the VT Prediction Model is not available for general clinical use. However, VT Prediction Model applications could be operationalized using various embodiments well-suited to provide diagnostic assistance to clinicians, especially for providers lacking expertise in using manual WCT differentiation approaches. For example, online calculators and mobile device applications could be used to implement the VT Prediction Model. The VT prediction Model can generate an estimation of VT probability following the input of QRS duration, QRS axis, and T wave axis from paired WCT and baseline ECGs. Another, and perhaps more exciting, prospective application would be the integration of the VT Prediction Model, or other comparable automated methods (e.g., WCT Formula (21) or WCT Formula II (23)), into existing CEI software. In this case, automated models like the VT Prediction Model could provide clinicians with (i) an adjudicated VT or SWCT diagnosis and/or (ii) an impartial estimate of VT probability. If the latter tactic is elected, the VT Prediction Model would be delivering cognitively meaningful information that will help guide clinicians towards the correct VT or SWCT diagnosis, while also allowing the opportunity of other diagnostic approaches (e.g., manual ECG algorithms and clinical determinates) to influence clinicians’ final clinical diagnosis.
Based off the results of this study, we anticipate that the automatic provision of VT probability estimates by automated approaches like the VT Prediction Model, when used in conjunction with patient-specific clinical factors and other traditional ECG interpretation approaches, will greatly improve physicians ECG interpretation accuracy and confidence in real-life clinical practice.
ii. Study strengths and limitations
Strengths
We believe our study design will serve as a valuable reference for those who wish to evaluate the practical value of other automated WCT discrimination models. This prospective analysis was conducted in such a manner to enable a realistic assessment of the overall diagnostic influence of the VT Prediction Model on both ECG interpretation accuracy and confidence for non-heart rhythm experts. This was accomplished through the careful design and execution of an ECG interpretation protocol that (i) isolated the impact of VT Prediction Model application on physician ECG interpretation and (ii) removed potential confounders (e.g., participants were blinded to auxiliary clinical information) or bias (e.g., participants were unaware of the purpose or design of the study before it unfolded) that may influence ECG interpretation.
Limitations
This analysis is best considered in the context of its limitations. First, our study was performed within highly regulated experimental conditions quite different from expected in actual clinical practice. Thus, our study design does not permit a comprehensive understanding of the VT Prediction Model’s diagnostic value within genuine clinical circumstances. Second, we only evaluated the interpretation performance of internal medicine residents and cardiology fellows. Consequently, we cannot infer whether the VT Prediction Model would provide similar benefit to more advanced ECG interpreters such as board-certified cardiologists and electrophysiologists. Third, we could not determine what heuristics were used by the participants to identify VT versus SWCT, and we did not directly compare the VT Prediction Model to manual algorithms based on the WCT ECG. Fourth, the short time allowed for ECG interpretation (i.e., 60 seconds) may have created a pressure situation that caused participants to rely more on the VT Prediction Model than they would have otherwise. Lastly, it is conceivable that our results were partially confounded by the fact that the participants interpreted the same WCT ECG multiple times over the course of the study (i.e., at least four times over the 4 days). It is possible that participants consciously or unconsciously refined their ECG interpretation due their prior exposure to the same WCT ECG. However, we believe that such a risk was adequately mitigated by ensuring (i) daily randomization of WCT ECG presentation order, (ii) time separation between interpretation days (i.e., one full day between interpretations sessions), (iii) concealment of the actual underlying heart rhythm diagnoses until study protocol completion, (iv) concealment of study protocol and design pre-study (e.g., participants did not know if the same ECGs were used for each study day), and (v) tightly regulating the amount of time to which the participants were allowed to view ECGs during interpretation sessions (i.e., 60 seconds without an opportunity to review the ECG again).
iii. Future directions
The next steps are to conduct a similar study that evaluates the diagnostic value of the VT Prediction Model when used by physicians considered to have superior heart rhythm expertise (i.e., general cardiologists and cardiac electrophysiologists). Beyond determining whether the VT Prediction Model can improve the diagnostic performance of more advanced ECG interpreters, it would be interesting to evaluate whether the ‘accuracy gap’ between non-expert and expert physicians can be closed by using the VT Prediction Model. Additionally, another area of great interest will be to explore whether other novel automated WCT discrimination models (e.g., WCT Formula and WCT Formula II), which are known to have stronger diagnostic performance than the VT Prediction Model, can achieve even better diagnostic results for its users. The results presented herein buoy the belief that other highly accurate automated WCT discrimination tools will significantly improve physician ECG interpretation accuracy and confidence.
CONCLUSION
VT Prediction Model use improved physicians’ ECG interpretation accuracy and confidence for differentiating WCTs.
Supplementary Material
Logistic regression structure of the VT Prediction Model. VT predictors (Xx) are assigned a beta coefficient (βx) according to their effect on the binary outcome (i.e., VT or SWCT). The “constant” term (β0) represents the y-intercept for the least-squares regression line. The weighted sum predictor (Xβ) or VT probability (P) is calculated after integrating VT predictor (Xx) values derived from paired WCT and baseline ECG data.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia; SWCT, supraventricular wide complex tachycardia.
Individual and composite overall accuracy for each day.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
Survey questions participants answered over the four days.
Intra-rater agreement of interpretations (i.e., VT or not VT) on individual interpreter level on each study day. Individual interpreter agreement (κ, kappa coefficient) for each day was assessed using the randomly selected duplicate 20 WCT ECGs (10 VTs, 10 SWCTs) from the total 50 WCT ECG set (25 VTs, 25 SWCTs).
Survey questions and participant responses.
Abbreviations: PGY, post-graduate year; WCT, wide complex tachycardia.
Inter-rater agreement of interpretations (i.e., VT or not VT) between internal medicine residents, cardiology fellows, and as a group on each study day. Interpreter agreement for each day was assessed individually (κ, kappa coefficient) and collectively (α, Krippendorff’s alpha coefficient).
Abbreviations: R, internal medicine resident; F, cardiology fellow.
Funding:
This work was supported by the Department of Cardiovascular Medicine at Mayo Clinic in Rochester, MN. The authors also acknowledge support by NIH T32 HL007111.
Footnotes
Disclosures: Anthony Kashou, Peter Noseworthy, Christopher DeSimone, Abhishek Deshmukh, and Adam May are potential beneficiaries of intellectual property discussed in the article. The remaining authors have no disclosures to report.
REFERENCES
- 1.Buxton AE, Marchlinski FE, Doherty JU, Flores B, Josephson ME. Hazards of intravenous verapamil for sustained ventricular tachycardia. Am J Cardiol 1987;59:1107–10. [DOI] [PubMed] [Google Scholar]
- 2.Stewart RB, Bardy GH, Greene HL. Wide complex tachycardia: misdiagnosis and outcome after emergent therapy. Ann Intern Med 1986;104:766–71. [DOI] [PubMed] [Google Scholar]
- 3.Sandler IA, Marriott HJ. The Differential Morphology of Anomalous Ventricular Complexes of Rbbb-Type in Lead V; Ventricular Ectopy Versus Aberration. Circulation 1965;31:551–6. [DOI] [PubMed] [Google Scholar]
- 4.Marriott HJ. Differential diagnosis of supraventricular and ventricular tachycardia. Geriatrics 1970;25:91–101. [PubMed] [Google Scholar]
- 5.Swanick EJ, LaCamera F Jr, Marriott HJ Morphologic features of right ventricular ectopic beats. Am J Cardiol 1972;30:888–91. [DOI] [PubMed] [Google Scholar]
- 6.Akhtar M, Shenasa M, Jazayeri M, Caceres J, Tchou PJ. Wide QRS complex tachycardia. Reappraisal of a common clinical problem. Ann Intern Med 1988;109:905–12. [DOI] [PubMed] [Google Scholar]
- 7.Kindwall KE, Brown J, Josephson ME. Electrocardiographic criteria for ventricular tachycardia in wide complex left bundle branch block morphology tachycardias. Am J Cardiol 1988;61:1279–83. [DOI] [PubMed] [Google Scholar]
- 8.Brugada P, Brugada J, Mont L, Smeets J, Andries EW. A new approach to the differential diagnosis of a regular tachycardia with a wide QRS complex. Circulation 1991;83:1649–59. [DOI] [PubMed] [Google Scholar]
- 9.Vereckei A, Duray G, Szenasi G, Altemose GT, Miller JM. Application of a new algorithm in the differential diagnosis of wide QRS complex tachycardia. Eur Heart J 2007;28:589–600. [DOI] [PubMed] [Google Scholar]
- 10.Vereckei A, Duray G, Szenasi G, Altemose GT, Miller JM. New algorithm using only lead aVR for differential diagnosis of wide QRS complex tachycardia. Heart Rhythm 2008;5:89–98. [DOI] [PubMed] [Google Scholar]
- 11.Pava LF, Perafan P, Badiel M et al. R-wave peak time at DII: a new criterion for differentiating between wide complex QRS tachycardias. Heart Rhythm 2010;7:922–6. [DOI] [PubMed] [Google Scholar]
- 12.Jastrzebski M, Sasaki K, Kukla P, Fijorek K, Stec S, Czarnecka D. The ventricular tachycardia score: a novel approach to electrocardiographic diagnosis of ventricular tachycardia. Europace 2016;18:578–84. [DOI] [PubMed] [Google Scholar]
- 13.Chen Q, Xu J, Gianni C et al. Simple electrocardiographic criteria for the rapid identification of wide QRS complex tachycardia: The new limb lead algorithm. Heart Rhythm 2019. [DOI] [PubMed] [Google Scholar]
- 14.Isenhour JL, Craig S, Gibbs M, Littmann L, Rose G, Risch R. Wide-complex tachycardia: continued evaluation of diagnostic criteria. Acad Emerg Med 2000;7:769–73. [DOI] [PubMed] [Google Scholar]
- 15.Baxi RP, Hart KW, Vereckei A et al. Vereckei criteria used as a diagnostic tool by emergency medicine residents to distinguish between ventricular tachycardia and supraventricular tachycardia with aberrancy. J Cardiol 2012;59:307–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kaiser E, Darrieux FC, Barbosa SA et al. Differential diagnosis of wide QRS tachycardias: comparison of two electrocardiographic algorithms. Europace 2015;17:1422–7. [DOI] [PubMed] [Google Scholar]
- 17.Szelenyi Z, Duray G, Katona G et al. Comparison of the “real-life” diagnostic value of two recently published electrocardiogram methods for the differential diagnosis of wide QRS complex tachycardias. Acad Emerg Med 2013;20:1121–30. [DOI] [PubMed] [Google Scholar]
- 18.May AM, Brenes-Salazar JA, DeSimone CV et al. Electrocardiogram algorithms used to differentiate wide complex tachycardias demonstrate diagnostic limitations when applied by non-cardiologists. J Electrocardiol 2018;51:1103–1109. [DOI] [PubMed] [Google Scholar]
- 19.McGill TD, Kashou AH, Deshmukh AJ, LoCoco S, May AM, DeSimone CV. Wide complex tachycardia differentiation: An examination of traditional and contemporary approaches. J Electrocardiol 2020;60:203–208. [DOI] [PubMed] [Google Scholar]
- 20.Kashou AH, Noseworthy PA, DeSimone CV, Deshmukh AJ, Asirvatham SJ, May AM. Wide Complex Tachycardia Differentiation: A Reappraisal of the State-of-the-Art. J Am Heart Assoc 2020;9:e016598. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.May AM, DeSimone CV, Kashou AH et al. The WCT Formula: A novel algorithm designed to automatically differentiate wide-complex tachycardias. J Electrocardiol 2019;54:61–68. [DOI] [PubMed] [Google Scholar]
- 22.May AM, DeSimone CV, Kashou AH et al. The VT Prediction Model: A simplified means to differentiate wide complex tachycardias. J Cardiovasc Electrophysiol 2020;31:185–195. [DOI] [PubMed] [Google Scholar]
- 23.Kashou AH, DeSimone CV, Deshmukh AJ et al. The WCT Formula II: An effective means to automatically differentiate wide complex tachycardias. Journal of Electrocardiology 2020. [DOI] [PubMed] [Google Scholar]
- 24.Kashou AH, LoCoco S, McGill TD et al. Automatic wide complex tachycardia differentiation using mathematically synthesized vectorcardiogram signals. Ann Noninvasive Electrocardiol 2021:e12890. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kashou AH, DeSimone CV, Hodge DO et al. The ventricular tachycardia prediction model: Derivation and validation data. Data Brief 2020;30:105515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cohen J A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas 1960;20:37–46. [Google Scholar]
- 27.R Core Team. R: A language and environment for statistical computing.: R Foundation for Statistical Computing, Vienna, Austria., 2019. [Google Scholar]
- 28.Gamer M, Lenon J, Fellows I, Singh P. Various Coefficients of Interrater Reliability and Agreement. 2019. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Logistic regression structure of the VT Prediction Model. VT predictors (Xx) are assigned a beta coefficient (βx) according to their effect on the binary outcome (i.e., VT or SWCT). The “constant” term (β0) represents the y-intercept for the least-squares regression line. The weighted sum predictor (Xβ) or VT probability (P) is calculated after integrating VT predictor (Xx) values derived from paired WCT and baseline ECG data.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia; SWCT, supraventricular wide complex tachycardia.
Individual and composite overall accuracy for each day.
Abbreviations: WCT, wide complex tachycardia; VT, ventricular tachycardia.
Survey questions participants answered over the four days.
Intra-rater agreement of interpretations (i.e., VT or not VT) on individual interpreter level on each study day. Individual interpreter agreement (κ, kappa coefficient) for each day was assessed using the randomly selected duplicate 20 WCT ECGs (10 VTs, 10 SWCTs) from the total 50 WCT ECG set (25 VTs, 25 SWCTs).
Survey questions and participant responses.
Abbreviations: PGY, post-graduate year; WCT, wide complex tachycardia.
Inter-rater agreement of interpretations (i.e., VT or not VT) between internal medicine residents, cardiology fellows, and as a group on each study day. Interpreter agreement for each day was assessed individually (κ, kappa coefficient) and collectively (α, Krippendorff’s alpha coefficient).
Abbreviations: R, internal medicine resident; F, cardiology fellow.