Comparative performance of stroke risk scores in patients with atrial fibrillation with low stroke risk

Abstract

Background

Australian guidelines recommend the CHA₂DS₂-VA (congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years) score to identify patients with atrial fibrillation (AF) at low stroke risk who should avoid oral anticoagulation. Treatment thresholds were derived from heterogeneous international data, and alternate scores require validation in an Australian population. We evaluated whether existing stroke scores (CHADS₂ (congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke (double weight)), CHA₂DS₂-VA, ATRIA (anticoagulation and risk factors in atrial fibrillation) and Modified-CHADS₂) accurately identify truly low-risk patients in Australia.

Methods

We conducted a population-based cohort study using linked data from the New South Wales Admitted Patient Data Collection, the National Death Index and the Pharmaceutical Benefits Scheme databases. Oral anticoagulant-naïve patients with a first hospital admission for AF between July 2003 and January 2021 were included. Patients were followed for ≥12 months and classified into low-risk, intermediate-risk or high-risk categories for each score. ‘Truly low-risk’ was defined as annual ischaemic stroke incidence <0.9%. Predictive accuracy was assessed using the concordance-statistic.

Results

Among 224 451 eligible patients, 3552 (1.6%) were hospitalised for ischaemic stroke within 12 months. The proportion labelled low-risk ranged from 2.0% (Modified-CHADS₂) to 50.9% (ATRIA). Patients assigned low-risk using each score, and intermediate-risk with the CHA₂DS₂-VA score, met the definition of truly low-risk. Concordance-statistics were modest, with the weakest performing score being the CHA₂DS₂-VA score (0.61, 95% CI 0.60 to 0.61) and the best performing model being the ATRIA score (0.66, 95% CI 0.66 to 0.67).

Conclusions

The guideline-endorsed CHA₂DS₂-VA score showed the lowest discrimination, and patients classified as intermediate-risk using this score had stroke incidence below the treatment threshold. Strong consideration should be given to the development and validation of locally applicable risk tools.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Clinical stroke risk scores are used to guide anticoagulation in atrial fibrillation, but their discriminative performance is modest, and the guideline-endorsed CHA₂DS₂-VA (congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years) score may classify only a small proportion of patients as truly low risk.

WHAT THIS STUDY ADDS

• In this large Australian cohort of 224 451 anticoagulant-naïve patients with atrial fibrillation, all four tested scores showed only modest discrimination for ischaemic stroke. ATRIA (anticoagulation and risk factors in atrial fibrillation) performed best, whereas CHA₂DS₂-VA performed worst and patients classified as intermediate risk by CHA₂DS₂-VA had stroke rates below the 0.9% per year treatment threshold.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• These findings support reappraisal of how intermediate CHA₂DS₂-VA risk is interpreted and reinforce the need for external validation of stroke prediction tools across contemporary cohorts.

Introduction

Atrial fibrillation (AF) is associated with a fivefold increased risk of ischaemic stroke, which is modified by the presence or absence of several risk factors.¹ Oral anticoagulation reduces the risk of stroke by two-thirds.² However, anticoagulant therapy increases the risk of intracranial haemorrhage (0.23%–0.50% per year)^{3 4} which has a higher 1 month case fatality rate than ischaemic stroke. With the advent of safer, more effective thromboprophylaxis (ie, direct oral anticoagulants (DOACs)), a greater proportion of patients are candidates for oral anticoagulation. It has been estimated that the threshold to obtain a net clinical benefit from oral anticoagulation is a stroke rate of 0.9%.⁵ Hence, risk assessment is focused on identifying ‘truly low-risk’ patients, for whom the risk of anticoagulant-associated bleeding outweighs the benefit in stroke risk reduction.

The Australian clinical practice guidelines recommend the sexless CHA₂DS₂-VA (congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years) score for identifying low-risk patients who should avoid oral anticoagulation.⁶ While this did occur prior to external validation of this score, removal of sex category from the parent CHA₂DS₂-VASc (congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years, female sex category) score simplified application. In addition, it avoided the practical limitations of applying a binary sex variable to patients who identify as non-binary, transgender or who are receiving gender-affirming hormone therapy.⁷

Subsequent Finnish registry data demonstrated marginal improvement in stroke prediction using the CHA₂DS₂-VA score compared with the CHA₂DS₂-VASc score.⁸ A Taiwanese cohort of 495 569 anticoagulant-naïve patients found no significant difference in discriminative performance between either score. However, in this study, female sex only independently increased stroke incidence among high-risk patients, supporting exclusion of sex for the specific purpose of identifying genuinely low-risk patients.⁹ Consequently, the more recent European guideline also recommends use of the CHA₂DS₂-VA score (adjust refs).

The CHA₂DS₂-VA score only assigns a very small proportion (~7%) of the AF cohort as low-risk.¹⁰ Hence, there may be intermediate-risk patients (5%–19% of AF cohort) for whom the risk of stroke does not justify oral anticoagulation. Reported stroke rates vary significantly for CHA₂DS₂-VA assigned intermediate-risk patients among international studies (0.2%–6.6%), with 10 of the 17 cohorts reporting a stroke rate <0.9%, strongly suggesting there are a large number of intermediate-risk patients for whom the risk of stroke does not justify oral anticoagulation.¹¹ True differences between populations, disparities in study design may account for this variation.

The CHA₂DS₂-VA(Sc) score is noteworthy for its simplicity. Yet, neither this score nor its predecessor—the CHADS₂ (congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke (double weight)) score—reliably capture stroke burden as each displays only modest predictive value.¹⁰ To improve predictive accuracy, other more complex scores have been proposed, including the ATRIA (anticoagulation and risk factors in atrial fibrillation) and Modified-CHADS₂ scores. Both scores demonstrated favourable discriminative performance when assessed in large Swedish datasets.^{12 13} Neither score has been externally validated using Australian data. This study sought to comparatively evaluate four stroke risk scores (CHADS₂, CHA₂DS₂-VA, ATRIA and Modified-CHADS₂) with respect to both discriminative performances and the relative proportions of patients with AF identified as having a truly low-risk of stroke in an Australian cohort.

Methods

Study data and population

We conducted a retrospective cohort study using linked administrative health databases from New South Wales (NSW), Australia. Data were obtained from: the Admitted Patient Data Collection (NSW APDC), which captures ≥97% of admission records from healthcare facilities in NSW¹⁴; the National Death Index for mortality records and the Pharmaceutical Benefits Scheme (PBS) for subsidised medicines dispensed in the community. Record linkage was performed using a project-specific personal identification number assigned by the Australian Institute of Health and Welfare. Only individuals linked across all datasets were eligible for inclusion.

The study population comprised patients aged 18 years or older discharged from hospital with a primary or secondary diagnosis of new-onset non-valvular AF or atrial flutter who were oral anticoagulant (OAC) naïve at baseline. A 2-year retrospective look-back period was used to ascertain comorbidities and to prevent misclassification of recurrent AF as an index diagnosis. Patients were followed for a minimum period of 12 months after the index AF admission. The cohort included hospital admissions from 1 July 2003 to 1 January 2021. PBS records were available for the full study period, enabling censoring of OAC initiation during follow-up. The following exclusion criteria were applied to identify anticoagulant-naïve patients with non-valvular AF: moderate-to-severe mitral stenosis and/or prosthetic heart valves; OAC use prior to index admission (defined as PBS dispensing in the 90 days before the index admission); OAC initiation within 14 days post-discharge and death within 14 days of discharge. Medicines classified as OACs included vitamin K antagonists and DOACs dabigatran, rivaroxaban, apixaban, edoxaban and betrixaban. The International Statistical Classification of Diseases and Related Health Problems, Tenth Revision, Australian Modification (ICD-10-AM) and Anatomical Therapeutic Chemical codes used to define study variables are listed in online supplemental eTables 1–7.

Patient and public involvement

No patients or members of the public were involved in the design, conduct, reporting or dissemination of this study. The study used retrospectively linked administrative health data and did not involve direct patient recruitment.

Outcomes

The primary outcome was ischaemic stroke within 12 months of the index AF admission. This time interval was selected as patients may accrue additional risk factors over time and increasing age is a strong independent determinant of AF-related stroke. Patients were observed until OAC initiation, death or at the end of follow-up. Events occurring within 14 days of the index admission were excluded. This blanking period was applied to account for the elevated risk of stroke in the first weeks after the onset of AF and to minimise duplicate event capture related to interhospital transfers.

Sensitivity analyses were conducted to assess the impact of methodological variance on outcomes. We examined a composite endpoint of ischaemic stroke, transient ischaemic attack (TIA) and/or systemic arterial thromboembolism, and repeated analyses for both endpoints with follow-up extended to 24 months and to the end of the study period.

Statistical analysis

The four risk scores defined in table 1 (CHADS₂, CHA₂DS₂-VA, Modified-CHADS₂ and ATRIA) were calculated at baseline and patients were categorised as low-risk, intermediate-risk or high-risk. The CHA₂DS₂-VASc score was not analysed separately because, when sex-specific guideline thresholds are applied, its low-risk, intermediate-risk and high-risk categories are equivalent to those generated by CHA₂DS₂-VA. For each score, we fitted a logistic regression model with risk category as the predictor. Discrimination (ability of a model to distinguish patients who develop stroke from those who do not) was quantified using the c-statistic with corresponding 95% CIs. Model discrimination was defined as either poor (c-statistic <0.6), modest (c-statistic 0.6–0.69), moderate (c-statistic 0.70–0.79) or excellent (c-statistic ≥0.8). Calibration (agreement between predicted and observed risk) was measured using the Hosmer-Lemeshow goodness-of-fit test.

Table 1. Risk stratification models for evaluating stroke risk in patients with atrial fibrillation.

Risk factors	CHADS2*	CHA2DS2-VA	CHA2DS2-VASc	Modified-CHADS2†	ATRIA
					Without prior stroke	With prior stroke
Congestive heart failure	(1)	(1)	(1)		(1)	(1)
Hypertension	(1)	(1)	(1)		(1)	(1)
Age	≥75 years (1)	≥75 years (2)	≥75 years (2)	85–115 years (6)	≥85 years (6)	≥85 years (9)
		65–74 years (1)	65–74 years (1)	80–84 years (5)	75–84 years (5)	75–84 years (7)
				75–79 years (4)	65–74 years (3)	65–74 years (7)
				70–74 (3)	<65 years (0)	<65 years (8)
				65–69 (2)
				40–64 (1)
Diabetes mellitus	(1)	(1)	(1)	(1)	(1)	(1)
Previous stroke/TIA	(2)	(2)	(2)	(6)
Vascular disease		(1)	(1)
Female sex			(1)	(1)	(1)	(1)
Proteinuria					(1)	(1)
Chronic kidney disease (eGFR <45 mL/min or ESRD)					(1)	(1)
Total score sum	6	8	9	14	12	15
Low-risk	0	0	0 (m), 1 (f)	0	0-5
Intermediate-risk	1	1	1 (m), 2 (f)	1–5	6
High-risk	≥2	≥2	≥2 (m), ≥3 (f)	6–14	7–15

The (number) indicates the value awarded for a particular risk factor. These values are added together to produce a final score which is used to risk-stratify patients.

^*Revised CHADS₂ categories used for analysis.³⁸

^†Categories proposed by Lip et al used for analysis.³⁴

ATRIA, anticoagulation and risk factors in atrial fibrillation; CHADS₂, congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous Stroke; CHA₂DS₂-VA, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years; CHA₂DS₂-VASc, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years, female sex category; eGFR, estimated glomerular filtration rate; ESRD, end-stage renal disease; TIA, transient ischaemic attack.

Incidence rates were calculated overall and by baseline risk-score category. Approximate 95% CIs were derived on the log scale assuming a Poisson distribution for event counts. Patients with a predicted annual stroke rate <0.9% (equivalent to 0.9 events per 100-person years) were considered truly low-risk, following established convention in the literature.⁵

For all analyses, a two-sided p<0.05 was considered statistically significant. Analyses were performed using SAS V.9.4 and R V.4.5.1. This manuscript is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies; the completed STROBE checklist is provided as the online supplemental file.

Results

We identified 2 24 451 patients with an index diagnosis of non-valvular AF who were OAC-naïve at cohort entry (online supplemental eFigure 1). Baseline characteristics are provided in table 2. The median age was 75.5 years (IQR 64.9–83.9) and 101 745 patients (45%) were women. Among risk score components, hypertension was most prevalent (37%), followed by vascular disease (29%), diabetes mellitus (19%) and heart failure (16%). Patients who developed stroke during follow-up were more likely to be women, older at AF diagnosis and have a history of hypertension and/or previous stroke.

Table 2. Characteristics of patients with atrial fibrillation, stratified by the occurrence of ischaemic stroke.

	Total, n (%)	No IS, n (%)	IS, n (%)	P value
Total patients	224 451	220 899 (98.4)	3552 (1.6)	–
Age at diagnosis, years, median (IQR)	75.5 (64.9–83.9)	75.4 (64.8–83.8)	82.6 (75.0–88.1)	<0.001
Sex
Male	122 715 (54.7)	121 112 (54.8)	1603 (45.1)	<0.001
Female	101 735 (45.3)	99 786 (45.2)	1949 (54.9)
Median risk score (IQR)
CHA2DS2-VA	2 (1–3)	2 (1–3)	3 (2–4)	<0.001
CHA2DS2-VASc	3 (2–4)	3 (2–4)	4 (3–5)	<0.001
Modified-CHADS2	4 (2–6)	4 (2–6)	6 (5–7)	<0.001
CHADS2	1 (0–2)	1 (0–2)	2 (1–3)	<0.001
ATRIA	5 (3–7)	5 (3–7)	7 (6–8)	<0.001
Medical history
Chronic kidney disease	23 291 (10.4)	22 804 (10.3)	487 (13.7)	<0.001
Diabetes mellitus	42 079 (18.7)	41 286 (18.7)	793 (22.3)	<0.001
Heart failure	36 773 (16.4)	35 975 (16.3)	798 (22.5)	<0.001
Hypertension	83 050 (37.0)	81 333 (36.8)	1717 (48.3)	<0.001
Stroke	8491 (3.8)	7923 (3.6)	568 (16.0)	<0.001
Transient ischaemic attack	4158 (1.9)	4007 (1.8)	151 (4.3)	<0.001
Vascular disease	65 605 (29.2)	64 567 (29.2)	1038 (29.2)	0.9935
Acute coronary syndrome	62 532 (27.9)	61 563 (27.9)	969 (27.3)	0.437
Smoking	71 199 (31.7)	70 195 (31.8)	1004 (28.3)	<0.001
Bleeding	25 137 (11.2)	24 657 (11.2)	480 (13.5)	<0.001
Cancer	29 312 (13.1)	28 873 (13.1)	439 (12.4)	0.212
Cardiomyopathy	4431 (2.0)	4368 (2.0)	63 (1.8)	0.387
Sleep apnoea	4455 (2.0)	4408 (2.0)	47 (1.3)	0.004

Comparisons between ischaemic stroke and no-ischaemic stroke groups were made using Wilcoxon rank-sum test for non-parametric continuous variables and Pearson’s χ2 tests for categorical variables.

ATRIA, anticoagulation and risk factors in atrial fibrillation; CHADS₂, congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke; CHA₂DS₂-VA, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years; CHA₂DS₂-VASc, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years, female sex category; IS, ischaemic stroke.

The proportion of patients categorised as low-risk ranged from 2.0% with the Modified-CHADS₂ score to 50.9% with the ATRIA score (figure 1). The CHA₂DS₂-VA score assigned the fewest patients as intermediate-risk (14.4%) while the Modified-CHADS₂ score allocated the largest proportion to this category (62.6%).

Figure 1. Proportion of patients with atrial fibrillation assigned as low-risk, intermediate-risk or high-risk for stroke at diagnosis according to each stroke risk score. ATRIA, anticoagulation and risk factors in atrial fibrillation; CHADS₂, congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke (double weight); CHA₂DS₂-VA, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years.

In the first year of follow-up, 3552 patients (1.6%) were hospitalised for ischaemic stroke (table 2). During follow-up, 29 193 patients (13.0%) were censored due to death and 38 436 patients (17.1%) were censored for initiating oral anticoagulation more than 14 days after the index presentation (online supplemental eFigure 1). The incidence of ischaemic stroke increased stepwise across low-risk, intermediate-risk and high-risk categories (figure 2). Patients classified as low-risk using all scores, and intermediate-risk with the CHA₂DS₂-VA score, recorded stroke incidence rates below the 0.9% per year treatment threshold. If a composite endpoint of ischaemic stroke, TIA and/or systemic thromboembolism was used, the event rate increased to 2.2% (a 42% relative increase), and patients with a low-risk ATRIA score (1.33 events per 100 person years, 95% CI 1.26 to 1.41) or an intermediate-risk CHA₂DS₂-VA score (1.24 events per 100 person years, 95% CI 1.11 to 1.38) were no longer below the treatment threshold (online supplemental eTable 9).

Figure 2. Forest plot of incidence rates (events per 100 person-years and 95% CIs). ATRIA, anticoagulation and risk factors in atrial fibrillation; CHADS₂, congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke (double weight); CHA₂DS₂-VA, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years.

The ATRIA score recorded the highest c-statistic for predicting ischaemic stroke (0.66, 95% CI 0.66 to 0.67) while the CHA₂DS₂-VA score demonstrated the lowest discrimination (0.61, 95% CI 0.60 to 0.61) (figure 3). The Hosmer-Lemeshow goodness-of-fit test indicated good calibration between observed and predicted risk for each score at all lengths of follow-up. The ATRIA score recorded the lowest negative predictive value for ischaemic stroke and among patients classified as low-risk, incidence varied by point score. Annual stroke incidence exceeded the 0.9% threshold for those with scores ≥2, who comprised 60.1% of the low-risk ATRIA cohort (online supplemental eTable 10).

Figure 3. Performance map showing, for each risk score, the c–statistic (x-axis), the negative predictive value for the low-risk category (y-axis), and the proportion classified as low-risk (point size). ATRIA, anticoagulation and risk factors in atrial fibrillation; CHADS₂, congestive heart failure, hypertension, age ≥75 years, diabetes mellitus, previous stroke (double weight); CHA₂DS₂-VA, congestive heart failure, hypertension, age ≥75 years (double weight), diabetes mellitus, previous stroke (double weight), vascular disease, age 65–74 years; NPV, negative predictive value.

Discussion

Patient-specific risk assessment provides the rationale for selectively using therapies that confer both benefit and harm. In the management of AF, stroke risk scores estimate thromboembolic risk and inform decisions regarding systemic anticoagulation. In this large Australian cohort of anticoagulant-naïve patients with non-valvular AF, the ATRIA score predicted ischaemic stroke better than the Modified-CHADS₂, CHADS₂ and, importantly, the current guideline recommended CHA₂DS₂-VA score. However, despite widespread use of such risk models in clinical practice, discriminatory power for each score was modest and lower than reported in their original validation cohorts.¹⁵,¹⁸ Of notable clinical importance, we demonstrated that the stroke incidence for patients with an intermediate-risk CHA₂DS₂-VA score was below the 0.9% per year treatment threshold, despite current guideline recommendations to consider anticoagulation in this risk group.

The proportion of patients assigned low-risk varied considerably between risk scores, as noted in previous cohort studies.¹² In clinical practice, this would facilitate significant differences in treatment, with 2% of patients advised to avoid oral anticoagulation with the Modified-CHADS₂ score compared with 51% with the ATRIA score. The ideal risk score should aim to identify the largest proportion of patients who are genuinely at low-risk of stroke and can safely avoid anticoagulation. An annual stroke risk of 0.9% is a commonly cited threshold, above which the benefits of anticoagulation outweigh the risk of bleeding.⁵ In the original derivation studies for each risk score, the low-risk groups were consistently defined by annual stroke rates below 1%, aligning closely with this threshold.¹⁶,¹⁸

Identifying low-risk patients is important as a growing proportion of patients with AF will have a truly low-risk of stroke at diagnosis. The incidence of stroke in this cohort is lower than previously reported, continuing a well-established trend in stroke risk reductions.¹⁵,²⁰ This could reflect the earlier identification of patients with lower disease burden and the improved treatment of comorbidities.^{21 22} These considerations urge caution in setting very low point-score treatment thresholds.

Stroke risk in AF is dynamic and patients classified low-risk should be regularly evaluated for incident comorbidities. Chao et al demonstrated this in a cohort of 31 039 anticoagulant-naïve patients with no baseline CHA₂DS₂-VA comorbidities besides age.²³ Among patients who experienced ischaemic stroke during follow-up, approximately 90% had acquired at least one new risk factor preceding the event.²³ Stroke risk was highest within 3 months of a change in score, possibly reflecting uncontrolled comorbid disease or the increase in stroke susceptibility during acute illness.

The European Society of Cardiology recommends re-evaluation at 6 months and then at least annually thereafter, while the Asia Pacific Heart Rhythm Society advises reassessment at 4 months.^{7 24} The shorter interval accurately reflects the faster progression of cardiovascular risk factors in some Asian populations compared with European.^{25 26} In support of this approach, a Taiwanese cohort of low-risk patients found that the interval between acquiring incident comorbidities and ischaemic stroke exceeded 4.4 months in 90% of cases.²⁶

The CHA₂DS₂-VAsc score was developed in response to concerns that its predecessor, the CHADS₂ score, was overly inclusive in its low and intermediate-risk classification. Upward reclassification of patients was achieved through the inclusion of vascular disease and by dividing age into three categories (<65 years, 65–74 years and ≥75 years). However, in our cohort, the ischaemic stroke rate in patients with both a low-risk CHA₂DS₂-VA score (0.32% per year, 95% CI 0.27 to 0.40) and a low-risk CHADS₂ score (0.57% per year, 95% CI 0.51 to 0.64) is sufficiently low to forego thromboprophylaxis. Moreover, stroke incidence in patients with an intermediate-risk CHA₂DS₂-VA score of 1 (0.80% per year, 95% CI 0.70 to 0.92) is below the treatment threshold. Hence, contrary to guideline advice, anticoagulation should be withheld from intermediate-risk patients.⁶ Collectively, these findings raise concern that efforts to reduce stroke risk may have expanded anticoagulant exposure beyond groups likely to derive net clinical benefit. Our study cannot be discounted as a low-risk outlier, with a systematic review identifying ischaemic stroke rates <1% per year in intermediate-risk patients (as defined by the CHA₂DS₂-VA(Sc) score) in 26 of 34 cohorts.¹¹ A Western Australian cohort of 49 114 patients with AF, although with unknown anticoagulation status, documented an annual stroke rate of 0.42% in low risk non-Aboriginal patients and 0.92% in Aboriginal patients within the same risk category.²⁷

In addition, clinicians should consider population-specific factors which may modify stroke risk when applying existing scores. Regional differences in stroke incidence are well documented. In a comparison of Korean nationwide and UK Biobank cohorts, East Asian patients had a substantially higher 5-year incidence of ischaemic stroke than predominantly Caucasian participants (incidence rate ratio 3.7, 95% CI 3.3 to 4.2).²⁸ Consistent with these findings, Chao et al demonstrated that lowering the age threshold of the CHA₂DS₂-VA(Sc) score from 65 years to 50 years improved discrimination and reclassification of genuinely low risk patients in a Taiwanese AF cohort.²⁹ These data highlight that stroke-risk thresholds may not be directly transferable across populations and support the need for region-specific validation of risk scores.

The superiority of the ATRIA score in predicting AF-related ischaemic stroke has been demonstrated previously in several cohort studies and in a systematic review of 6 267 728 patients.^{12 18 30} The ATRIA score uses a complex, although more accurate weighting system derived from formal statistical modelling.¹⁸ In addition, incorporation of renal biomarkers linked to elevated stroke risk is likely to partially account for its improved C-statistic. However, this gain in predictive performance should be interpreted in the context of reduced clinical simplicity, as the ATRIA score is too cumbersome to calculate without online aids. In contrast, the CHADS₂ and CHA₂DS₂-VA(Sc) scores both favour simplicity over statistical accuracy.¹² Assigning a single point to each risk factor is problematic, because the magnitude of risk is variable. This is particularly concerning for intermediate-risk patients, where a score of 1 attributed to age 65–74 may reflect a substantially higher stroke risk than an equivalent score attributed to hypertension. The Modified-CHADS₂ score sought to address this issue by excluding hypertension as a variable and by assigning greater weighting to prior stroke and age, thereby better reflecting the differential impact of individual risk factors.¹⁷

The major risk factors in the ATRIA score are prior stroke and patient age, subdivided by decade. In this study, the ATRIA score identified the largest number of low-risk patients. This is consistent with evaluation of the ATRIA score in two Californian cohorts, which reported annual stroke rates of 0.63% and 0.68% in low-risk patients respectively.¹⁸ However, the annual stroke risk varied among patients with a low-risk ATRIA score, from 0.31% (95% CI 0.25 to 0.40) (score=0) to 1.61% (95% CI 1.43 to 1.80) (score=5). Using existing cut-points, an 84-year-old male patient with no additional stroke risk factors would be assigned low-risk. This inclusive age criterion may be especially problematic for East-Asian, Aboriginal Australian and African American patients, where age greater than 65 years already significantly increases stroke risk.^{27 29 31}

The Modified-CHADS₂ score subdivides age into six categories with varying point allocations which is similarly impractical. The score was developed to improve predictive accuracy for stroke and identify high-risk patients.¹⁷ Hence, while the Modified-CHADS₂ score recorded a higher C-statistic than the CHA₂DS₂-VA score, it classified a very small proportion of patients as low-risk (2.0%), clearly limiting its use as a decision-tool for thromboprophylaxis.

This study highlights the limitations of traditional stroke prediction tools, with respect to poor predictive accuracy and misclassification of low-risk patients. Given these shortcomings, attention has increasingly shifted towards novel approaches that incorporate structural and biochemical markers to refine stroke prediction in AF. Echocardiography enables the structural and functional changes of AF on the left atrium to be measured and incorporated into stroke prediction. In a cohort of 932 patients, non-chicken wing left atrial appendage morphology was the strongest independent predictor of stroke in those with a CHADS₂ score of 0–1³². However, classification of atrial pathology by echocardiography requires some expertise and is not universally available.

Biomarkers may improve risk prediction by capturing biological pathways involved in thrombosis, but improved discrimination does not necessarily translate into better clinical outcomes. This was illustrated in a Swedish multi-centre randomised controlled trial, in which biomarker-guided management using the ABC-AF (Age, Biomarkers (NT-proBNP and hs-troponin T) and Clinical history of stroke/TIA) stroke and bleeding scores did not reduce stroke, death or major bleeding compared with standard guideline-based care.³³ Although the trial was terminated early and underpowered, there was a non-significant trend towards higher mortality in the ABC-AF strategy arm, highlighting the need for prospective evaluation before biomarker-based tools are implemented in routine practice.³³ Similarly, despite advances in artificial intelligence, simple risk scores are likely to remain embedded in clinical practice, warranting continued scrutiny given their influence on treatment decisions.

Strengths and weaknesses

The NSW APDC enables large-scale, high-quality epidemiological research with comprehensive follow-up. This is the first Australian registry study to externally validate the CHA₂DS₂-VA score in an anticoagulant-naïve cohort and comparatively evaluate its performance against three alternate scores. The majority of cohort studies in this field, including those from which the risk scores were derived, were conducted in populations with unknown anticoagulation status.^{1618 27 34},³⁶ In excluding anticoagulated patients, our study reduces treatment-related bias on outcome rates and better reflects the real-world application of these scores on anticoagulant-naïve patients.

This study included sub-analyses which demonstrated the clinically significant sensitivity of stroke estimates to methodological variance (follow-up and univariate vs multivariate stroke endpoints). Therefore, it is unsurprising that significant variations in stroke risk have been reported between studies. This was explored in a systematic review by Quinn et al, in which a wide intra-study variation in stroke risk for patients with an intermediate risk CHA₂DS₂-VASc score was reported (0.2%–6.6% per year).¹¹ In addition to varied outcome definitions, the included studies differed substantially in population, ethnicity and handling of anticoagulation during follow-up. Hospital-based cohorts recorded higher event rates than in community settings, while studies excluding patients who subsequently commenced anticoagulation may have underestimated stroke risk by conditioning on future treatment decisions.¹¹ This is particularly relevant to our study, which evaluates risk-score performance in an Australian anticoagulant-naïve cohort with censoring at OAC initiation, addressing the methodological limitations that complicate interpretation of prior observational studies.

The present study has several limitations, inherent to its retrospective, observational design. First, the ‘true’ stroke rate may have been underestimated as events were not registered for fatal undiagnosed stroke or for stroke admissions recorded in another state or territory. Second, including only hospitalised patients with AF may have captured a higher risk profile cohort, overestimating absolute stroke rates relative to patients in general practice. The study also relied on diagnosis according to ICD-10 codes, which could have resulted in either over- or under-reporting of endpoints and patient demographics. The ischaemic stroke outcome used in this study, however, has been validated against a medical records audit conducted in four regional NSW hospitals and ICD-10-AM coding accuracy has been demonstrated in Australia previously.³⁷ Third, we did not capture AF burden or differentiate primary from secondary AF presentations, potentially introducing heterogeneity in underlying risk and anticoagulation decisions. Finally, the c-statistic is a relatively inelastic measure of predictive accuracy and covariates with substantial ORs may only generate small improvements. This is because the c-statistic measures the rank-order of predicted risks for cases and non-cases, rather than the magnitude of these predicted risks. Consequently, the c-statistic does not assess calibration, absolute risk estimation or clarify whether a model influences treatment decisions around clinically relevant thresholds. Further studies should supplement our analyses with reclassification metrics and decision curve analysis to identify whether differences in discrimination translate into clinically meaningful improvements in anticoagulant decision-making. In addition, future research should assess whether serial reassessment of initially low-risk patients improves timely anticoagulation initiation and reduces subsequent stroke risk.

Conclusions

Current guidelines recommend considering oral anticoagulation in patients with a CHA₂DS₂-VA score >0. However, in this large Australian cohort, patients with an intermediate-risk CHA₂DS₂-VA score were still very low-risk. Moreover, the CHA₂DS₂-VA showed poor discriminative ability in stratifying stroke risk. In contrast, the ATRIA score showed superior predictive accuracy and identified a larger subgroup of low-risk patients. Yet, this score is unlikely to be widely used in its current complex form. The simplicity of existing risk stratification scores has resulted in their widespread utilisation; however, this is of little benefit if the tool lacks accuracy. The challenge that remains is to develop locally valid risk stratification scores that retain simplicity and therefore user acceptability but have greater predictive accuracy than those that currently exist.

Data availability statement

Data are available upon reasonable request.