Inconsistent methodology as a barrier to meaningful research outputs from studies of Atrial Fibrillation after Cardiac Surgery

Abstract

Atrial fibrillation after cardiac surgery (AFACS) is an important postoperative complication. There is significant research interest in this field but also relevant heterogeneity in reported AFACS definitions and approaches used for its identification. Few data exist on the extent of this variation in clinical studies.

We reviewed the literature since 2001 and included manuscripts reporting outcomes of AFACS in adults. We excluded smaller studies and studies where patient did not undergo a sternotomy. The documented protocol in each manuscript was analyzed according to six different categories to determine how AFACS was defined, which techniques were used to identify it and inclusion / exclusion criteria. We also noted when a category was not described in the documented protocol.

We identified 302 studies, of which 92 were included. 62% of studies were randomised controlled trials. There was significant heterogeneity in the manuscripts, including the exclusion of patients with AF pre-surgery, the definition and duration of AF needed to meet the primary endpoint, the type of screening approach ( continuous, episodic or opportunistic), the duration of monitoring during the study period in days, the diagnosis with pre-defined ECG criteria, and the requirement for independent confirmation by study investigators. Furthermore, the definitions of these criteria were also frequently not described.

Consistent reporting standards for AFACS research are needed to advance scientific progress in the field. We propose pragmatic standards for trial design and reporting standards. These include adequate sample size estimation, clear definition of the AFACS endpoints and a protocol for AFACS detection.

Introduction:

Atrial Fibrillation (AF) is the most frequent complication after cardiac surgery^1,2 and is associated with morbidity, mortality and increased resource utilization^3–7. Extensive efforts are therefore made to prevent AFACS before it occurs and manage it proactively when it does.

There remain sizeable gaps in knowledge regarding the best way to predict higher-risk patients, which interventions should be used to prevent and treat AFACS, and which patient groups should receive these interventions^{8 9,10} Most interventions that are currently used to prevent AFACS carry potential unwanted adverse effects and have not been tested robustly in properly powered randomised controlled studies¹¹. There is no consensus between the professional societies on the definition of AFACS. The Heart Rhythm Society defines an AF/flutter/tachycardia episode as present if it is documented by ECG and lasts at least 30 seconds¹². However, the Society of Thoracic Surgeons (STS) describes it as “needing treatment”¹³. The American Association for Thoracic Surgery (AATS) defines clinically significant post-operative AF as requiring treatment with rate or rhythm control agents, or requiring anticoagulation, and/or extends the duration of hospitalization¹⁴. The Heart Rhythm Society and European Heart Rhythm Association have published a consensus statement regarding ablation of AF that describes standards required for reporting outcomes in clinical trials, which includes the standardised definitions of AF recurrence that should be used for research studies¹². No such consensus statement exists for reporting standards of AFACS. Without a documented definition of AF, the reader is forced to guess whether the authors’ understanding of AF is the same as theirs.

Whilst seemingly obvious, it warrants explicitly stating that the extent to which AFACS is identified is dependent on the type and duration of surveillance undertaken. AF is frequently paroxysmal so the continuity of monitoring and the methods for event capture will determine detection rates. AF may be asymptomatic or occur when an individual is asleep and will therefore only be diagnosed when continuous ECG monitoring is performed and episodes of AF are either flagged automatically or proactively sought and identified. The prognostic significance of very short episodes of AF identified on continuous monitoring remains uncertain¹⁵. Following cardiac surgery, patients are intensively monitored in a critical care unit and are then frequently ‘stepped down’ to the ward, where monitoring is less intensive, prior to discharge. AFACS most frequently first occurs 2–3 days post-operatively in patients who are continuously monitored until discharge¹⁶. However, AF is not readily identified in the later stages of a patient’s admission or after discharge when a patient is unmonitored, unless symptoms or haemodynamic compromise also occur.

The effect of this is that if two reported clinical studies on AFACS in the medical literature use a different way of screening for AF, the incidence rates will be different. Furthermore, including shorter asymptomatic episodes of AF may not represent the same pathology or have the same prognostic significance as episodes of symptomatic persistent AF. Indeed, not all AF is the same and there is clinical significance to not view AFACS as a binary event (yes or no), but to also consider the concept of ‘AF Burden’. It is critically important that these distinctions are clearly defined to enable a degree of head-to-head comparisons between clinical studies and in meta-analyses.

Trialists recently standardised clinical relevant cardiovascular outcomes and defined AF as “new onset of irregularly irregular heart rate in the absence of P waves lasting at least 30 seconds or for the duration of the ECG recording (if <30 seconds)”¹⁷. As AF can be asymptomatic, the lack of continuous ECG monitoring for AF surveillance should be stated as a major limitation. Conversely, although such statements are warranted to improve clinical trial reporting, the implication from the second aspect of this consensus statement is that the clinical studies meeting this standard may have limited ‘real world’ impact where continuous monitoring is not always applied throughout the entire post-operative period until discharge. However, we expect monitoring technology to further evolve rapidly and before too long, continuous rhythm monitoring of some kind might well become a standard, possibly even beyond the hospital.

A minimum standard and / or a gold standard for AFACS definitions along with study design, statistical powering and analyses has the potential to improve future AFACS research. We hypothesized that there is heterogeneity of how clinical studies currently capture and report AFACS. Therefore, prior to defining these standards for AFACS clinical studies, we reviewed the literature to examine this variation, with a view to informing a discussion on which criteria should be included in future reporting standards.

Methods:

In order to study the consistency of AF definition in clinical studies, we reviewed the literature for manuscripts on AFACS. We queried the MEDLINE database using PubMed to identify pertinent literature using the following search terms: (“Cardiac Surgical Procedures”[MeSH] OR Cardiac surger*[tiab] OR Heart surger*[tiab]) AND (Predict*[tiab] OR risk[tiab]) AND (“Atrial Fibrillation”[MeSH] OR Atrial Fibrillation[tiab]). We arbitrarily included all clinical studies from the year 2001 onwards in the context of adult cardiac surgery reporting outcomes of incidence / prevalence of AFACS as a primary outcome measure or implied central focus of the study. We excluded small studies and case reports with fewer than 60 patients. We subsequently excluded studies involving interventions without sternotomy (i.e. cardiac catheterisation laboratory or structural heart interventions), studies that assessed an intervention to treat AF, protocol papers and studies that reported subgroup analyses from previously published datasets.

Two independent researchers analysed an initial 20 manuscripts meeting these search criteria to examine patterns between the approaches used to both diagnose and describe the AF outcome measure and also report gaps. After doing so, the authors reached consensus on six criteria, each one of which could allow manuscripts with similar study protocols to be grouped together and therefore compared.

These six criteria were: a) minimum length of AF duration, b) criteria by which AF was diagnosed (multiple choices possible), c) need for validation of ECG or Holter, d) inclusion of non-AF arrhythmias, e) opportunistic clinical diagnosis vs. episodic monitoring vs. continuous monitoring, and f) patients with known AF pre-operatively excluded, or not. The distribution of categories for the whole cohort of manuscripts was then analyzed by the two researchers, while inter-observer agreement was assessed. For all criteria, each clinical study was only categorised once and was not counted in multiple categories. When data were not reported, this was noted. For monitoring period definitions, “Episodic” was defined as an evaluation at certain defined time points only and “Opportunistic” was defined as an evaluation when clinical suspicion was raised. When assessing whether patients with known AF pre-operatively were excluded, we defined “partially excluded” when patients only needed to be in sinus rhythm at the time of recruitment and/or for a defined time-period pre-operatively, or when a history of persistent AF was excluded, but paroxysmal AF was not. When reporting incidence of AF, if a range of AFACS occurrence was reported (e.g. comparing two groups), the higher border percentage was used. The authors did not attempt to make any inferences about study methodology or reporting when this was not explicitly stated.

We subsequently interpreted the results in the context of current guidelines for AF. The reporting of our manuscript followed the Scale for the Assessment of Narrative Review Articles (SANRA) recommendations¹⁸.

Results:

Our search of the MEDLINE database identified 302 studies. We had to exclude 210 studies which led to 92 studies that were analyzed (see Fig. 1).

Fig. 1:

Study Flow Chart

The studies included utilised different cohorts, including CABG patients only or mixed surgical cohorts. The studies analysed a median of 150 patients (range 60 – 506,110). 62 reported studies (67%) were randomised controlled trials, 17 studies (18%) had a retrospective observational design and 13 studies (14%) were prospective observational or non-randomised controlled trials.

Inclusion and exclusion criteria for AF studies

In the analyzed studies, patients with a history of paroxysmal or persistent AF prior to surgery were not excluded from 32% of the studies or the authors did not specify. Regarding endpoints, 93% of the manuscripts did not report if they accepted other non-AF atrial tachyarrhythmias as diagnostic endpoints (see Fig. 2).

Fig. 2:

Common inclusion and exclusion criteria of analysed studies: (A) Reporting if patients with prior history of AF were excluded from the study. The definition of “partially” excluded is when patients only needed to be in sinus rhythm at the time of recruitment and/or for a defined time-period pre-operatively, or when a history of persistent AF was excluded, but paroxysmal AF was not. (B) Reporting of non-AF atrial tachyarrhythmias being included in the study population.

Monitoring for AF

By monitoring for AF, 33% reported to use a telemetry system until the end of a defined monitoring period (Fig. 3A). 4% of studies used intermittent ECGs to diagnose AF and 18% required telemetry with subsequent confirmation by ECG. Overall, 25% of studies included a period of continuous telemetry monitoring followed by a period of intermittent ECGs, while 16% did not report the modality of monitoring. Monitoring was performed continuously in 47% (Fig. 3B). Episodic surveillance (meaning evaluation at certain time points only) was utilised in 1% of the studies, while opportunistic monitoring only (i.e. raised clinical suspicion) was not performed as a single modality in the analyzed manuscripts. However, a combination of continuous and/or episodic and/or opportunistic monitoring was used in the rest of the studies.

Fig. 3:

Monitoring for Atrial Fibrillation: (A) Reported modalities in AF monitoring, and (B) continuity in monitoring to detect AF.

Duration of monitoring

The monitoring period lasted ‘until ICU discharge’ in 1% of studies. In 4% the monitoring ended on POD1 or POD2, 7% of studies reported a monitoring period until POD3, 4% until POD4, 13% until POD5, 2% until POD6 and 14% until POD7. Monitoring until hospital discharge was reported in 34% of studies, while 14% did not clearly state the length of the surveillance period.

Pre-defined criteria to diagnose AF

The duration of pathological ECG rhythm to formally diagnose AF varied greatly among the studies (see Table 1). In the studies where a duration of AF was defined, the mean AFACS incidence reported was 35%, compared to a mean incidence of 29% when it was not defined. To formally diagnose AF, 87% of the manuscripts reported using typical ECG changes. The rest required a combination of ECG changes plus symptoms, interventions, etc. Formal definitions of ECG criteria (e.g. absence of P waves and irregularly irregular QRS complexes) were not reported in 72% of the studies. Validation of a telemetric or ECG-based suspicion for AF was established in a blinded manner in 25% and in a non-blinded manner in 14%. 61% of studies did not report validation.

Table 1:

Reported criteria to fulfill diagnosis of AFACS. For ECG criteria, “partially defined” when definition documented for Holter analysis but not ECG

Requirements for diagnosis:
	ECG changes	87%
	ECG changes and need for intervention	2%
	ECG changes and reported symptoms	0%
	ECG changes and caretaker diagnosis	2%
	ECG changes and notes in the chart	1%
	Not reported	8%
Duration of AF needed to meet diagnostic criteria:
	30 seconds	17%
	5 minutes	28%
	10 minutes	9%
	15 minutes	2%
	20 minutes	1%
	30 minutes	2%
	>60 minutes in 24 hours	0%
	Continuous AF on ECG rhythm strip	0%
	Need for urgent treatment or intervention	2%
	Not defined	33%
	Other	6%
ECG criteria defined:
	Yes	0%
	No	27%
	Partially	1%
	Not reported	72%
Validation of ECG or telemetry:
	Yes, data blinded	14%
	Yes, blinding not reported	25%
	No	0%
	Not reported	61%

If a study used opportunistic criteria along with other criteria to diagnose AF, 72% did not clearly report these criteria. 13% reported “general symptoms” and each 1% of studies either accepted either a hemodynamic compromise or an irregular pulse assessment as an opportunistic criterion for AF.

Incidence of AF in the analysed studies

The reported incidence for AFACS varied substantially in the studies: 29% stated an incidence in the range of 20–29%, and 29% reported a range of 30–39%. In 16% of the studies, an incidence between 0–19% was reported. The incidence in the remaining 26% of studies varied between 40–89%. If a range of AFACS occurrence was reported (e.g. comparing two groups), the upper border percentage was used for our analysis. The studies were performed in a variety of different surgical cohorts, including CABG only cohorts and mixed surgical cohorts. There was no change in the incidence of AF detected over the 20 year study period examined: studies from the years 2001–2010 reported a mean incidence of AF of 35±18% compared to studies published in the years 2011–2021 with a mean incidence of 32±14% (Student’s t-test, P=0.431).

Discussion:

To our knowledge, we are the first to present quantifiable data on the heterogeneity of reported criteria in studies assessing AFACS. This is relevant given that it is fundamentally important for clinical studies to use unequivocal and clear definitions that avoid the need for assumptions from their readers. However, in the many studies focusing on AFACS it becomes clear that AF is often poorly defined and methods to detect it vary widely. There are no current standards for specifically reporting AFACS studies.

Our results show that there is a large variability between studies in how AFACS is examined and reported. Some of the common shortcomings include poor consideration and planning, retrospective study design and methodology, and subsequent limited data availability. There is no consistency among the studies for the minimum duration of AF required to meet the endpoint. A duration of either ≥ 30 seconds or ≥ 5 minutes accounted for about half of all studies where a duration was defined, but there was otherwise no agreement with durations ranging from 10 seconds to ‘sustained’. No duration of AFACS was defined in almost one third of cases. Studies in which a duration of AF was not defined reported a lower incidence of AF. This suggests that studies where a duration was not reported may have not included (or indeed detected) all true AFACS episodes.

There was also a wide variation regarding the duration of monitoring. Approximately one third of studies used a duration until post-operative day 5 to 7 and another third of studies defined AFACS as occurring during the hospital stay. There was otherwise little agreement between the duration used and 14% of studies did not report the duration of monitoring that the endpoint covered. About half of all studies reported did not monitor patients continuously throughout the investigation period and depended on episodic or opportunistic diagnosis; this means that the true rate of AF may have been under-reported.

AF may be incorrectly diagnosed on an ECG or continuous monitoring as it can resemble premature atrial complexes or an artifact. Some clinicians rely on an automated computer algorithm to diagnose AF which may result in under- or overdiagnosis with adverse clinical consequences¹⁹. Atrial flutter and atrial tachyarrhythmias may be challenging to distinguish from AF. It has been reported that up to one third of ECG interpretation has some error compared to expert reference²⁰. Even experienced electrophysiologists do not concur in their ECG reporting in 1 in 20 cases²¹. It is therefore critical that standardised ECG definitions are used. However, in 72% of the studies we reviewed, an ECG definition of AF was not provided, while 61% of studies did not describe whether the investigators validated the AF diagnosis.

AF is much more likely to occur again in a patient who has already experienced AF. Therefore, in a study assessing risk factors for AFACS prior AF needs to be identified as possibly one of the strongest predictors of AFACS. In studies examining the effects of an intervention to prevent new-onset AFACS, it is counterintuitive to include patients who have a history of AF. It is noteworthy that one third of reported studies did not exclude patients with a history of AF. Some studies included patients if they had been in sinus rhythm for a significant period (for example 6 months) prior to surgery or if they were in sinus rhythm at the time of recruitment, but including such patients will dilute the effect of the intervention being assessed.

These limitations in study methodology, manuscript reporting, or both hinder the field of AFACS research. Heterogeneity in study design is an inevitable part of clinical research, but it is incumbent on clinical researchers and journal editors to accurately describe the steps taken in trial design.

High quality research in the field will be facilitated by highlighting current inconsistent practices and shortcomings and ultimately providing a supportive framework for best practice. As part of a quality improvement exercise for AFACS studies, we propose pragmatic minimum standards and gold standards for trial design and trial reporting standards for studies where incidence of AFACS is a primary or secondary outcome measure or a measure of central concern to the research project. We want to propose a framework available to the scientific community to advance best practice in trial design, planning, execution and interpretation. This framework may also help inform (and standardise) clinical trial adjudication and endpoint evaluation. We suggest future clinical studies of AFACS should consider the suggestions in Table 2, along with the Methodological Considerations in the Appendix.

Table 2:

Proposed trial characteristics for consistent methodology in AFACS research

Trial design	Manuscript reporting standard
MINIMUM
All patients with pre-operative AF should be excluded from the analysis or clearly highlighted and treated as a separate powerful risk factor.	Described in methods.
Prospectively define duration of AF as an endpoint in advance of the analysis	Described in methods
Define duration of monitoring	Described in methods
Type of monitoring should be described (continuous / episodic / opportunistic)	Described in methods
Criteria needed for AF diagnosis should be defined (e.g. ECG alone / ECG + symptoms / ECG + need for treatment / ECG + haemodynamic compromise)	Described in methods
Are atrial flutter and atrial tachyarrhythmias included in the definition of AF? Yes / No	Described in methods
GOLD STANDARD: (as above, and include)
Where ECG criteria are used, definition of AF should be defined	Described in methods. If the diagnosis is not validated, this should be described in the limitations.
AF diagnosis should be validated by the research team when it is firstly made by someone outside of the research team	Described in methods. If the diagnosis is not validated, this should be described in the limitations.
Continuous ECG monitoring with a separate ambulatory device, independently analysed (possibly by a blinded core lab) for corroboration and quality control	Described in methods
Duration of observation of at least 120 hours or until discharge (whichever is sooner) from the end of surgery	Described in methods

Reviewing the studies in our cohort, 38 of 92 studies met the first 5 minimum criteria in Table 2. None of these 38 studies described whether atrial flutter and atrial tachyarrhythmias were included in the definition of AF so none of the studies we reviewed met all six minimum criteria. The reported incidence of AF was not statistically different between studies with a higher number versus a lower number of minimum reporting criteria (see Supplemental Figure 1).

We plan to arrange an expert consensus meeting with cardiac intensivists, cardiologists, cardiac surgeons, cardiac anesthesiologists and clinical trialists, involving the specialist national bodies in both Europe and North America. To that end, we also propose the following four recommendations for investigators investigating AFACS, which should also be explicitly described in the methods of any AFACS clinical research study:

1. It is essential to adequately power the study to prevent further anecdotal or hypothesis-generating research.

2. Define the AF endpoint consistent with peer-reviewed guidelines and recommendations and consider ways to quality-assure the assessment

3. Design your study so that AFACS gets reliably detected. Ideally apply continuous monitoring throughout the observation period

4. Ensure that the follow up period is sufficiently long to capture the majority of events. For in-hospital studies, consider the entire hospitalization period.

Closing remarks

There is significant heterogeneity in trial design and trial reporting in the field of AFACS research. Lack of consistency and variation in reporting standards risks hindering the scientific field. Minimum reporting standards for AFACS research are urgently needed and have the potential to improve the research quality in this space and allow for meaningful meta-analyses.

Supplementary Material

Supplementary Figure 1

Supplemental Fig. 1: Incidence of AF across categories of minimum reporting criteria

Abbreviations:

AF

Atrial fibrillation

AFACS

Atrial fibrillation after cardiac surgery

SD

Standard Deviation