Ischaemic stroke etiological classification system: the agreement analysis of CISS, SPARKLE and TOAST

Abstract

Background and purpose

The ideal stroke classification system needs to have validity, high reliability and applicability among different stroke research settings. The Chinese Ischemic Stroke Subclassification (CISS) and the Subtypes of Ischemic Stroke Classification System (SPARKLE) have emerged recently but have not been tested using agreement analysis. As a result, the objective of this study is to investigate the level of agreement among stroke subtype classifications using CISS, SPARKLE and Trial of Org 10172 in Acute Stroke Treatment (TOAST). We also analyse the inter-rater reliability of CISS.

Methods

The data include 623 inpatients who have had an ischaemic stroke, accrued from Beijing Tiantan Hospital between 1 October 2015 and 19 April 2016. According to the diagnostic standards of the three subtype classification systems, 299 inpatients who satisfied the requirements of our study were independently classified with etiological subtypes, and we compared the three subclassifications.

Results

There was substantial overall agreement among the three classification systems: CISS versus SPARKLE (kappa value=0.684, p<0.001), CISS versus TOAST (kappa value=0.615, p<0.001) and SPARKLE versus TOAST (kappa value=0.675, p<0.001). The inter-rater reliability of CISS was excellent (kappa value=0.857, p<0.001). Furthermore, among the three subtype classification systems, the variance analysis results of the etiological subtypes were not uniform.

Conclusion

There were generally substantial agreements among three ischaemic stroke etiological classification systems. CISS is a valid and reliable classification system, with which different stroke research centres can apply and compare data.

A precisely etiological classification system is highly significant in the treatment and prognostication of ischaemic stroke, and one ideal stroke classification system needs to have high validity and reliability. Trial of Org 10172 in Acute Stroke Treatment (TOAST) is one of the most broadly used ischaemic stroke etiological classification systems and is well known for simplicity, logic and practicability; however, one of its important limitations is that the categorisation of ‘undetermined’ cause of stroke is as high as approximately 40%.¹ The significant character of Subtypes of Ischemic Stroke Classification System (SPARKLE) is to take the measurement of the carotid plaque burden as one criterion of large artery atherosclerosis (LAA), which increases the proportion of cases attributable to LAA and reduces the proportion classified as being of ‘undetermined’ etiology.² The Chinese Ischemic Stroke Subclassification (CISS) is an innovative system that offers much more detailed information on the pathophysiology of a stroke, such as incorporating vulnerable plaques into the classification of LAA, weakening the essentiality of lacunar syndromes and introducing the mechanisms of ischaemic strokes caused by LAA.³

Although TOAST, SPARKLE and CISS are different etiological classification systems, they use broadly similar categories of stroke diagnoses (eg, LAA, cardioembolism (CE), small vessel disease, other etiology (OE) and undetermined), creating a foundation for the three well-established systems to be compared. Because each etiological classification system independently enacts its own classification criteria and diagnostic standard, and each stroke research centre may adopt a different classification system to categorise aetiologies, it is necessary to perform agreement analysis for the different etiological classification systems, to communicate and compare the data of diverse clinical and research settings and to promote the development of ischaemic stroke research.

Accessing the agreement among the three systems is the primary purpose of this report, with the data coming from Beijing Tiantan Hospital Stroke Research Center. Because there is no gold standard for etiological stroke classification, we are not going to discuss which system is ‘better’ or ‘weaker’, but we will analyse the agreement among the three systems and test the inter-rater reliability of CISS.

Methods

Patients

We retrospectively analysed 623 patients with ischaemic stroke who were admitted to the Beijing Tiantan Hospital from 1 October 2015 to 19 April 2016. Eliminating 108 patients who had a medical history of ischaemic stroke, 4 with cerebral haemorrhage, 13 with transient ischaemic attack and 91 with other diagnosis, we finally consecutively extracted 299 cases from 407 first-time ischaemic stroke patients, which is similar to the sample size of other such studies^{2 4 5} (figure 1). Demographic data and vascular risk factors such as age, sex, smoking, diabetes mellitus, hypertension, dyslipidaemia, atrial fibrillation (AF), coronary atherosclerotic heart disease (CAHD) and myocardial infarction (MI) were recorded.

Figure 1.

Technological route. CISS, Chinese Ischemic Stroke Subclassification; SPARKLE, Subtypes of Ischemic Stroke Classification System; TOAST, Trial of Org 10 172 in Acute Stroke Treatment.

To include the diagnosis of hypertension, the patient had to have a documented hospital record of hypertension. Diabetes mellitus was defined as the prescribed treatment of hypoglycaemic agents, or two fasting hyperglycaemia tests 7.0 mmol/L or higher or plasma glucose 11.1 mmol/L or higher at 2 hours into an oral glucose tolerance test. If total cholesterol was greater than 5.17 mmol/L, low-density lipoprotein was greater than 3.1 mmol/L, high-density lipoprotein was less than 1.0 mmol/L or the patient was taking a cholesterol-lowering agent, dyslipidaemia would be recorded. A patient was considered to be a smoker when he (or she) had a past or current history of smoking more than one cigarette per day for more than 6 months. AF, CAHD and MI were recorded from medical history and would be further clarified according to the relevant tests after admission.

Each eligible case was confirmed to be a cerebral infarction according to CT, MRI or both and was a first-time ischaemic stroke. Furthermore, auxiliary tests were accomplished within the first week after being admitted to the hospital, including colour ultrasound of the carotid artery, vertebral artery, subclavian artery and aortic arch; transcranial Doppler sonography including microemboli monitoring and microbubble injections with Valsalva manoeuvre test; ECG; 24 hours Holter monitoring; head imaging (CT, MRI, CT angiography or magnetic resonance angiography); and immunology screening. In addition, some special tests (eg, transoesophageal echocardiography, high-resolution MRI (HR-MRI) and digital subtraction angiography (DSA)) are conditionally fulfilled. Not all patients receive every test; therefore, after the primary examination and evaluation, the more important tests should be considered based on what is necessary for the determination of a clear-cut diagnosis. For instance, transoesophageal echocardiography will be performed if the result of the microbubble injection test is positive; the HR-MRI or the DSA will be performed when the possibility of vertebral artery dissection is ruled out or to clarify the underlying pathology of the affected artery. Determining and assessing the clinical correlation tests require the knowledge of a senior physician.

First, according to the published aetiologically diagnostic criteria of TOAST, SPARKLE and CISS, one experienced neurologist performed a comprehensive analysis of the patient’s clinical medical history and pertinent test data and classified the etiological subtype for the three stroke cause classification systems. Generally, there are five unified causes of ischaemic stroke: LAA, CE, small-vessel disease (SVD, also known as penetrating artery disease), OE and undetermined etiology (UE). Next, a second veteran neurologist respectively categorised the aetiology subtypes of the CISS system to the same 299 cases (figure 1).

For the SPARKLE system, it was total plaque area (TPA) of 1.19 cm² or more that defined the large-artery atherosclerosis aetiology, also included as the diagnostic criterion. The definition of TPA is the sum of the cross-sectional areas of all plaques seen between the bilateral clavicle and the angle of the jaw.⁶ The colour Doppler ultrasound machines used in this research were primarily Philips iu22.

Statistical analysis

SPSS V.23.0 was used for the statistical analysis. Normality test was performed on the measurement data. The data that conform to the normal distribution was expressed as mean±SD; if the normal distribution was not consistent, the median and the IQR (median, P ₂₅−P ₇₅) were used to describe the data. Enumeration data were expressed by the frequency and composition ratio (%).

The common agreement among the three systems and the inter-rater agreement of CISS was determined by Cohen’s kappa, in which the power of the kappa value was interpreted as poor (k=0.00), slight (k=0.00–0.20), fair (k=0.21–0.40), moderate (k=0.41–0.60), substantial (k=0.61–0.80) and excellent (k>0.81) agreement. McNemar’s test was performed to assess the variance analysis of the five etiological subtypes for the three systems. Alpha was set at 0.05.

Results

Patient characteristics

On the whole, the age of the population was normally distributed, with an average age of 58.3±13.5 years, and ranged from 17 years to 95 years. The proportion of male was more than 77.9%. Other risk factors were 71.2% of dyslipidaemia, 52.6% of smoking, 38.8% of hypertension, 17.4% of diabetes and 11.7% of CAHD. Percentage of MI and AF was low, 3.3% and 2.7%, respectively (figure 2).

Figure 2.

Patient characteristics. AF, atrial fibrillation; CAHD, coronary atherosclerotic heart disease; MI, myocardial infarction.

The mean age of patients in the CE subtype was the highest, and the youngest patients in the three classification systems were in the OE phenotype (64.7 vs 40.2 year, 64.0 vs 41.2 year and 64.8 vs 41.2 year, respectively). Diabetes and hyperlipidaemia in the LAA subtype were significantly higher than the other stroke aetiologies for all the three classification systems. CAD, MI and AF were the most common risk factors for the CE causes in each of the stroke systems (see online supplementary tables 1–3).

Frequencies of the aetiological subtypes

For the TOAST and SPARKLE systems, the frequency of the five causal subtypes was consistent, and the order from high to low was LAA (46.5% and 55.2%), UE (26.1% an 18.7%), CE (11.7% and 10.7%), SVD (10.4% and 9.7%) and OE (5.4% and 5.7%).

In line with the judging criteria of the CISS system, the highest proportion of the cause was still LAA (199/299, 66.6%), but the second was CE (36/299, 12.0%), which was more than UE (28/299, 9.4%); the lowest ratio was OE (17/299, 5.7%) (table 1).

Table 1.

The frequency distribution and comparison of the etiological subtypes of the CISS, SPARKLE and TOAST systems

Subtype	TOAST	SPARKLE	CISS	CISS versus CISS versus TOAST versus
				SPARKLE	TOAST	SPARKLE
	n (%)	n (%)	n (%)	P value	P value	P value
LAA	139 (46.5)	165 (55.2)	199 (66.6)	<0.001	<0.001	<0.001
CE	35 (11.7)	32 (10.7)	36 (12.0)	0.289	1	0.581
SVD	31 (10.4)	29 (9.7)	19 (6.4)	0.021	0.02	0.824
OE	16 (5.4)	17 (5.7)	17 (5.7)	1	1	1
UE	78 (26.1)	56 (18.7)	28 (9.4)	<0.001	<0.001	0.007
Total	299	299	299

CE, cardioembolism; CISS, Chinese Ischemic Stroke Subclassification; LAA, large artery atherosclerosis; OE, other etiology; SPARKLE, Subtypes of Ischemic Stroke Classification System; SVD, small-vessel disease; TOAST, Trial of Org 10172 in Acute Stroke Treatment; UE, undetermined etiology.

Agreement and variance of the three systems

Generally, the outcome of the agreement test indicated that agreement among the three systems was substantial: CISS versus SPARKLE (k=0.684, p<0.001), CISS versus TOAST (k=0.615, p<0.001) and SPARKLE versus TOAST (k=0.675, p<0.001). In addition, the inter-rater agreement analysis of the CISS system, in which one neurologist and a second neurologist independently grouped the stroke phenotypes of CISS, showed an excellent agreement relationship (k=0.857, p<0.001).

However, there were inconsistent results in the difference analysis of the five etiological subtypes among the three systems. In the cross-tabulation of the phenotypes of the three etiological classification systems (figure 3), the highest coincidence proportion was the OE subtype, with 16 cases in accordance; an exception was one patient in the TOAST system was placed in the UE subtype, possibly because of combining with other possible stroke causes. Furthermore, the CE category was also less variable, with no significant difference among the three systems (CISS vs SPARKLE, p=0.289; CISS vs TOAST, p=1; and SPARKLE vs TOAST, p=0.581). When the SVD subtype was tested, there was no significant difference between SPARKLE and TOAST (p=0.824), with 20 cases in common. A significant variance was found between LAA and UE: there was only p=0.007 in the UE sort between SPARKLE and TOAST, and all the other groups were p<0.001 (table 1).

Figure 3.

The cross-tabulation of the phenotypes of the three systems. CE, cardioembolism; CISS, Chinese Ischemic Stroke Subclassification; LAA, large artery atherosclerosis; OE, other etiology; SPARKLE, Subtypes of Ischemic Stroke Classification System; SVD, small-vessel disease; TOAST, Trial of Org 10172 in Acute Stroke Treatment; UE, undetermined etiology.

Discussion

To promote the advancement of treatment and the prevention of ischaemic stroke and to discover genetic and other novel vascular risk factors, it is obligatory for us to distinguish ischaemic stroke heterogeneity.⁴ However, it was a complex process to identify new risk factors, therapeutic methods and genetic variations, requiring plenty of patient samples from the research foundation.⁷ When an individual stroke centre could not satisfy the condition, the collaboration of various research settings was a feasible solution. One issue that was exposed during our study is how to ensure the reliability in the combination process of different settings of data; obviously, standardisation and unification of the etiological phenotypes was strikingly essential for the judgements. The aim of this investigation is to discuss the agreement among the diverse ischaemic stroke etiological classification systems.

On the whole, the substantial agreements among three systems gave testament for the integration of data from many stroke research centres. Whereas, when the difference analysis of etiological phenotypes was delineated, there were atypical characters among the five subtypes: LAA and UE subtypes were both significantly different, which may be associated with the difference of their definition and requirements of auxiliary examinations in each stroke classification system. Undoubtedly, re-evaluation should be considered when these causes were combined; yet on the other hand, the other three etiological subtypes were not significantly different.

The LAA subclassification had the highest proportion of patients in the CISS system (199/299, 66.6%), followed by SPARKLE (165/299, 55.2%) and TOAST (139/299, 46.5%), with significant differences among the three systems. This may be relative to the discrepancy of the diagnostic criteria. Because the TOAST system was subject to the limitation of technology at 1990s, the definition of LAA emphasised relevant intracranial/extracranial artery of 50% or greater stenosis or occlusion, cortical or cerebellar lesions and brain stem or subcortical hemispheric infarcts greater than 1.5 cm in diameter, not involving the aspects of plaque vulnerability or the evidence of systemic atherosclerosis. Based on the diagnostic standards of the LAA cause in the TOAST system, the SPARKLE system added TPA 1.19 cm² or higher as a criterion for the LAA aetiology, which previously regarded TPA as a stronger predictor of stroke, MI or death than carotid stenosis.⁸ Moreover, whereas the CISS system did not place much emphasis on the degree of associated artery stenosis, it did highlight some new risk factors, such as vulnerability of plaques and aortic arch plaque thickness greater than 4 mm, and categorise the aortic arch atherosclerosis into LAA. Obviously, more diagnostic requirement points of LAA aetiology were included in the CISS classification, which resulted in a higher detection rate for the LAA cause (199/299, 66.6%). Whereas, this difference requires more tests and an increased the cost of hospitalisation to some extent.

When the analysis of the SVD subtype was performed, there was a non-significant difference between SPARKLE and TOAST (p=0.824), but there were significant differences for both systems with CISS (p=0.021 and p=0.02, respectively). The reason may be that the SPARKLE and TOAST systems use the same definition for the SVD subtype, with both systems stressing lacunar infarction syndrome and the position and diameter of cerebral infarction. In contrast, the CISS system overlooked the concept of lacunar infarction syndrome and did not emphasise the importance of the infarction diameter. However, it highlights no evidence of atherosclerotic plaque and any degree of stenosis in the parent artery. Expectedly, with the more strictly diagnostic criteria of the SVD phenotype, CISS identified a smaller number of cases (19, 6.4%) with SVD than did the other two systems.

There were no significant differences among the three systems within the CE and OE subtypes, for which the criteria of both these etiological phenotypes were relatively objective. The diagnosis of the CE cause primarily relied on merging high-risk and medium-risk sources of CE, and multitemporal lesions in the brain imaging. The OE cause was primarily based on the fact that some other rare causes were specifically relevant to the stroke index and could be verified by auxiliary examinations (eg, arterial dissection, haematological system disorders and vasculitis). Therefore, it can be said that the definition is more objective and distinct, improving the consistency between the subtypes.

Concerning large-artery atherosclerosis, the three classification systems showed significant differences regarding the UE subtype, yet the issue of distribution of UE was justly contrary to that of LAA: highest in the TOAST system (78/299, 26.1%), followed by SPARKLE (56/299, 18.7%) and CISS (28/299, 9.4%). The reason may be that both CISS and SPARKLE identified cases with LAA that would be classified into the UE category in the TOAST system, resulting in decreasing the scale of the UE subtype and in turn increasing the number of definite stroke causes. This finding is similar to the result of previous relevant research² and meant that more appropriate and effective therapy could be applied for more patients to reduce the number of recurrent strokes.

The CISS system had high inter-rater agreement (k=0.857) showing that the uniformity of the classification outcomes was very high between the two neurological physicians, providing an evidence for more stroke research settings to use the CISS system in clinical practice.

This report demonstrated that the overall agreement among TOAST, SPARKLE and CISS was substantial and showed the excellent inter-rater reliability of the CISS system. However, our study found significant difference between some stroke etiological phenotypes. Besides the differences of diagnostic criteria, the reason for the significant difference may also be related to the adjudicators’ personal ability and the number of auxiliary examinations that were performed, which is coincident with the outcomes of some other published analyses of ischaemic stroke aetiology classification agreement.^{4 5 9}

The data of this study were all taken from the inpatients of Beijing Tiantan Hospital. The advantages are the authenticity, reliability and integrity of data, as well as convenience for clinical analysis and follow-up tests. However, we should acknowledge the limitations of single-centre studies and that the hospitalised patients generally had severe neurological deficits and could not fully represent the outpatients and community patients. Consequently, it is critical to study more cases in more research centres to determine the consistency of the CISS system.

Conclusion

This analysis manifested the overall substantial agreement among the CISS, SPARKLE and TOAST etiological classification systems. The proportion of UE subtype was somewhat lower in both the CISS and SPARKLE systems compared with the TOAST system; an excellent inter-rater agreement was exhibited for the CISS system, which supplied proof for diverse stroke research settings to use this system in clinical practice or trials. However, it is necessary to perform more multicentre trials to further identify the validity of the CISS system.