Abstract
Background:
The aim of this study was to validate the 2019 European League Against Rheumatism/American College of Rheumatology (EULAR/ACR) classification criteria for systemic lupus erythematosus (SLE) in antinuclear antibody (ANA)-positive Chinese patients.
Methods:
Medical records of all adult patients who attended the rheumatology out-patient clinics between May and September 2019 were reviewed. Patients with ever ANA positive (titre ⩾1:80) were included and evaluated for the fulfilment of the 2019 EULAR/ACR, 2012 Systemic Lupus International Collaborating Clinics (SLICC) and 1997 ACR criteria for SLE classification. The performance of these criteria in predicting a clinical diagnosis of SLE as judged by an independent panel of rheumatologists was studied and compared in different subgroups.
Results:
A total of 1533 patients (88.2% women; age at first clinic attendance 45.5 ± 15.6 years) were studied and 562 patients were judged to be clinical SLE. The sensitivity and specificity of the EULAR/ACR (⩾10 points), SLICC and ACR criteria for a clinical diagnosis of SLE was 96.1%, 97.9% and 86.1%; and 85.8%, 86.3% and 94.3%, respectively. Applying the attribution rule to the non-SLE controls, the specificity of the three criteria increased to 95.0%, 92.5% and 98.8%, respectively. The specificity of the EULAR/ACR criteria was higher in male patients (97.9%), those aged >50 years (97.0%) and disease duration of ⩽3 years (97.6%). Using a cut-off of 12 points, the specificity of the EULAR/ACR criteria was further increased (96.6%) while a high sensitivity (95.0%) was maintained.
Conclusion:
In Chinese patients with a positive ANA, the EULAR/ACR criteria for clinical SLE perform equally well to the SLICC criteria. Both the EULAR/ACR and SLICC are more sensitive but less specific than the ACR criteria. The specificity of all the three criteria is enhanced by applying the attribution rule to controls. The specificity of the EULAR/ACR criteria is higher in certain patient subgroups or when the cut-off score is raised.
Keywords: classification, criteria, diagnosis, lupus, validation
Introduction
Systemic lupus erythematosus (SLE) is a clinically and serologically heterogeneous multisystem autoimmune disease with variable disease course and prognosis. Although the survival of SLE has improved in the past few decades, further improvement is hampered by uncontrolled disease activity and treatment-related complications that lead to organ damage and comorbidities.1–3 Early diagnosis and prompt treatment are important to improve the outcome of the disease.4,5 However, early clinical manifestations of SLE may be nonspecific and difficult to be differentiated from other rheumatic diseases.6,7
Classification criteria of SLE were developed to differentiate SLE from SLE mimics. They also help identify a more homogeneous group of SLE patients for comparison among different cohorts and recruitment into clinical trials. Over the past two decades, continuous effort has been made to improve the performance of SLE classification criteria. Currently available SLE classification criteria include the 1997 American College of Rheumatology (ACR) criteria, 8 2012 Systemic Lupus International Collaborating Clinics (SLICC) criteria 9 and the more recent 2019 European League Against Rheumatism/American College of Rheumatology (EULAR/ACR) criteria. 10
The ACR criteria consist of nine clinical and two serologic items. 8 Fulfilment of at least 4 of these 11 criteria is necessary for a classification of SLE. All criteria have to be attributed to SLE and can be cumulative. The SLICC group developed another set of criteria in 2012, which comprises 11 clinical and 6 serological criteria. 9 Patients have to fulfil at least 4 (both clinical and serological) of 17 criteria in order to be classified as SLE. The SLICC criteria were found to be more sensitive (97% versus 83%) but less specific (84% versus 96%) than the 1997 ACR criteria in Caucasian patients. 9
The EULAR/ACR criteria were developed to further improve the performance of SLE classification. A weighted approach for each item in the criteria and a positive antinuclear antibody (ANA) (titre ⩾1:80) as the entry criterion was adopted. The criteria included seven clinical domains and three immunological domains. 10 Patients with ⩾10 points are classified as SLE when there is at least one clinical criterion fulfilled. Validation of the 2019 EULAR/ACR criteria showed a similar sensitivity (96% versus 97%) to the 2012 SLICC criteria but higher specificity (93% versus 84%) for SLE. 10 However, the majority of patients in the derivation and validation cohorts were Caucasians (<10% Asians).
As the manifestations, treatment response and outcome of SLE are different between Asian and non-Asian patients,11–14 the performance of 2019 EULAR/ACR criteria for SLE classification may be different in the Asian populations. Table 1 summarizes the recent validation studies of the EULAR/ACR criteria for SLE in adult patients.10,15–24 There are only a few studies involving Asian patients15–17,21,24 and the sample size is variable. There is also no analysis of patient subgroups for the performance of the EULAR/ACR criteria. The current work was undertaken to validate the EULAR/ACR classification criteria for SLE in a cohort of ANA-positive adult Chinese patients followed in our out-patient clinics.
Table 1.
Validation studies of the 2019 EULAR/ACR SLE classification criteria in adult patients.
| Author | Ethnicity | Sample size | Standard of SLE diagnosis | Patient characteristics | Patient subset | Sensitivity | Specificity | ||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 2019 EULAR/ACR | 2012 SLICC | 1997 ACR | 2019 EULAR/ACR | 2012 SLICC | 1997 ACR | ||||||
| Aringer et al. 10 | 74.1% White 9.8% Hispanic 9.4% Asian 5.4% Black |
1270 | From participating centres and verified by three adjudicators | – | – | 96% | 97% | 83% | 93% | 84% | 93% |
| Teng et al. 15 | Chinese | 375 | From three experienced rheumatologists | New-onset SLE | – | 96.5% | 92% | 75.4% | 90.3% | 84% | 96% |
| Suda et al. 16 | Japanese | 100 | From two ACR board-certified doctors | Consecutive cases of SLE visiting study hospital | – | 92% | 99% | 97% | – | ||
| Lee et al. 17 | Korean | 672 | Known cases of SLE followed by rheumatologists for ⩾2 years | – | – | 97.6% | 98.5% | 95.5% | 91.4% | 92.6% | 93.8% |
| Johnson et al. 18 | 74.1% White 5.4% Black 9.8% Hispanic 9.3% Asian |
1270 | From participating investigators and verified by three independent SLE experts | – | Female (n = 1098) | 97% | 97% | 83% | 94% | 82% | 93% |
| Male (n = 172) | 93% | 94% | 78% | 96% | 90 | 94% | |||||
| Asian (n = 118) | 97% | 99% | 77% | 91% | 91% | 93% | |||||
| Pons-Estel et al. 19 | Caucasians, Mestizos, and African Latin Americans | 1047 | Patients fulfilling the 1982/1997 ACR SLE criteria | Latin American patients | – | 91.3% | – | – | – | – | – |
| Dahlström and Sjöwall 20 | NS | 111 | Established by one senior rheumatologist and fulfilled the 1982 ACR criteria | – | – | 93% | 100% | 83% | 73% | 75% | 82% |
| Jin et al. 21 | Chinese | 2097 | Known SLE patients, independently reviewed by two dermatologists | 1865 SLE patients and 232 isolated cutaneous lupus patients | – | 96.8% | 98.3% | 91% | 63.4% | 56.5% | 68.5 |
| Adamichou et al. 22 | Greek | 1091 | From consultant rheumatologists with ⩾5 years practice | Consecutive cases of newly diagnosed SLE | All | 88.6% | 91.3% | 85.7% | 97.3% | 93.8% | 93.0% |
| ANA positive | 94.6% | 94.7% | 87.0% | 92.1% | 84.9% | 88.1% | |||||
| Petri et al. 23 | NS | 716 | Consensus of 32 rheumatologists (with >80% agreement) | – | – | 91% | 97% | 83% | 89% | 84% | 96% |
| Wang et al. 24 | Chinese | 126 | Renal biopsy proven LN based on 2003 ISN/RPS classification | LN patients followed up for >1 year | – | 95.24% | 99.20% | – | – | – | – |
| Present study | Chinese | 1533 | Consensus of a panel of three senior rheumatologists | Consecutive patients in clinics who were ANA positive (⩾1:80) | – | 96.1% | 97.9% | 86.1% | 95.0% | 92.5% | 98.8% |
ACR, American College of Rheumatology; ANA, antinuclear antibody; EULAR, European League of Association of Rheumatology; ISN/RPS, International Society of Nephrology and the Renal Pathology Society; LN, lupus nephritis; NS, not stated; SLE, systemic lupus erythematosus; SLICC, Systemic Lupus International Collaborating Clinics.
Patients and methods
Study population
The medical records of all patients who attended the rheumatology clinics of Tuen Mun and Pok Oi hospital, Hong Kong between August and December 2019 were reviewed. Patients ⩾18 years of age at first clinic attendance and had a positive ANA (titre ⩾1:80) ever tested were included. Patients with ANA titre <1:80, no ANA results or were not Chinese in ethnicity were excluded.
Patients in our rheumatology clinics were regularly followed every 3–6 months and more frequent visits would be arranged for unstable patients or those with complications. The records of the recruited patients from their first clinic attendance to last visits were studied retrospectively by an investigator (Y.K.C.) for the fulfilment of the EULAR/ACR, SLICC and 1997 ACR criteria. The cut-off score for fulfilling the EULAR/ACR criteria was ⩾10, but different cut-off scores were also explored. A random sampling of 400 patients (200 patients fulfilling and 200 patients not fulfilling the EULAR/ACR criteria) were cross-checked by another independent investigator (C.L.). Inconsistency in data entry was discussed to achieve a consensus. Demographic and other clinical data of the recruited patients were also collected.
The clinical diagnosis of SLE was judged by a panel of three senior rheumatologists (L.Y.H., C.H.T., C.C.M.) based on the clinical history, laboratory results and therapeutic decisions shown in the medical records without the knowledge of the classification criteria results. Discrepancy on the clinical diagnosis of SLE was discussed among the panel of rheumatologists via teleconferences. Final agreement was obtained by voting. This study was approved by the Research Ethics Committee (REC) of our hospitals (NTWC/REC/20151). All patient details have been de-identified during data analysis. The reporting of this study conforms to the checklist recommended by EULAR (available as supplemental material) for reporting longitudinal observational registry studies in rheumatology derived from an extension of the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines. 25
Assay of ANA and anti-dsDNA
ANA was performed by indirect immunofluorescence. ANA titre was determined by serial dilution. The reported titre was the last dilution that the indirect immunofluorescence was identified as positive. Anti-double-stranded DNA (anti-dsDNA) was tested by an enzyme-linked immunosorbent assay (ELISA) (DIASTAT anti-dsDNA kit, Euro-Diagnostica, Sweden; upper limit of normal range 50 IU/ml). Anti-dsDNA positivity was defined as a titre 25% above the normal limit, that is, 62.5 IU/ml according to the SLE disease activity index (SLEDAI) definition of increased DNA binding. 26
The sensitivity and specificity of the EULAR/ACR, SLICC and ACR classification criteria for a clinical diagnosis of SLE was calculated and compared. The performance of these three criteria was also tested in different subgroups of SLE patients stratified by sex, age (⩽50 versus >50 years at first attendance) and disease duration (⩽3 versus >3 years). Different cut-off scores were also explored for the EULAR/ACR criteria by receiver operating characteristic curve (ROC) analysis.
Statistical analyses
Descriptive statistics of the patients studied were expressed as either mean ± standard deviation (SD) or median [interquartile range (IQR)]. The sensitivity and specificity of the three criteria were calculated by 2×2 contingency tables according to the panel diagnosis of SLE (‘condition positive’) and criteria positive (‘test positive’) using standard formulas [sensitivity = true positive/(true positive + false negative); specificity = true negative/(true negative + false positive)]. The 95% confidence interval was computed using the online MedCalc Software Ltd. diagnostic test evaluation calculator. 27
ROC analysis was performed to obtain the cut-off score for the best sensitivity and specificity values of the EULAR/ACR criteria for a clinical diagnosis of SLE (Youden’s index, that is, maximum summation of sensitivity and specificity minus one). 28 McNemar’s test was used to compare the sensitivity and specificity among the ACR, SLICC and the EULAR/ACR classification criteria. Statistical significance was defined as a p value of <0.05, two tailed. All statistical analyses were performed using the SPSS software (version 26.0) for Window Xp.
Results
Study population
Among 3967 patients who attended our clinics in the specified period, 2425 patients who had ANA titers <1:80 or no ANA ever done were excluded. Nine non-Chinese patients were also removed from analysis. Finally, 1533 patients (88.2% women) were studied. The age of the patients at their first clinic visit was 45.5 ± 15.6 years. The mean duration of follow-up of the patients was 8.3 ± 6.4 years (74.2% had follow-up >3 years) and 645 (42.1%) patients were above the age of 50 years.
Clinical diagnosis of SLE and classification criteria
A total of 526 patients (34.3%) satisfied all the three criteria. 678, 683 and 539 patients fulfilled the EULAR/ACR, SLICC and ACR criteria, respectively. The consistent rate for scoring of different criteria between the two investigators (Y.K.C. and C.L.) was 93.3%.
The panel of rheumatologists unanimously agreed that 482 patients were clinically SLE. Disagreement on the clinical diagnosis was noted in 135 patients (8.8%) who had overlapping features of other rheumatic diseases and this was resolved by voting through three teleconferences. Finally, 562 patients (36.7%) were judged to be clinical SLE and 971 patients (63.3%) were not [diagnoses included rheumatoid arthritis (42.4%), Sjogren syndrome (13.6%), systemic sclerosis (10.3%), undifferentiated connective tissue disease (5.8%), psoriatic arthritis (2.9%), spondyloarthropathy (2.6%), adult-onset Still’s disease (0.2%), dermatomyositis (2.1%), polymyositis (0.6%), systemic vasculitides (0.8%), polymyalgia rheumatica (0.8%), overlap syndromes (0.7%), antiphospholipid syndrome (0.7%), Behcet’s disease (0.2%), immune thrombocytopenia (0.8%), isolated cutaneous lupus (0.6%) and no rheumatological diseases (21.9%)]. The prevalence of clinical features adopted by the EULAR/ACR criteria in panel judged SLE and non-SLE patients is shown in Table 2. Manifestations not included in the EULAR/ACR criteria in our SLE patients were mainly gastrointestinal and neuropsychiatric: protein losing enteropathy (7.1%), immune hepatitis (0.5%), mesenteric/intestinal vasculitis (0.5%), intestinal pseudo-obstruction (0.2%), myelitis (1.2%), peripheral neuropathy/mononeuritis multiplex (1.2%) and cranial neuropathy (1.1%).
Table 2.
Clinical features of patients judged to be clinical SLE and non-SLE.
| SLE (N = 562) |
Non-SLE (N = 971) |
||
|---|---|---|---|
| Without attribution | With attribution | ||
| Number (%) | |||
| Clinical features | |||
| Constitutional | |||
| Fever (>38.3°C) | 59 (10.5) | 2 (0.3) | 2 (0.3) |
| Mucocutaneous | |||
| Alopecia | 109 (19.4) | 13 (1.3) | 13(1.3) |
| Oral ulcers | 72 (12.8) | 19 (2.0) | 16 (1.6) |
| Subacute cutaneous/discoid lupus | 59 (10.5) | 14 (1.4) | 12 (1.2) |
| Acute cutaneous lupus | 210 (37.4) | 24 (2.5) | 21 (2.2) |
| Musculoskeletal | |||
| Joint involvement | 361 (64.2) | 622 (64.1) | 6 (0.6) |
| Serosal | |||
| Pleural/pericardial effusion | 75 (13.3) | 5 (0.5) | 5 (0.5) |
| Acute pericarditis | 4 (0.7) | 0 (0) | 0 (0) |
| Neuropsychiatric | |||
| Delirium | 8 (1.4) | 0 (0) | 0 (0) |
| Psychosis | 9 (1.6) | 1 (0.1) | 1 (0.1) |
| Seizure | 16 (2.8) | 1 (0.1) | 1 (0.1) |
| Haematological | |||
| Leukopenia (<4000/mm³) | 435 (77.4) | 367 (37.8) | 367 (37.8) |
| Thrombocytopenia (<100,000/mm³) | 139 (24.7) | 28 (2.9) | 28 (2.9) |
| Autoimmune haemolysis | 108 (19.2) | 3 (0.3) | 3 (0.3) |
| Renal | |||
| Proteinuria >0.5 g | 290 (51.6) | 10 (1.0) | 10 (1.0) |
| Class II/V lupus nephritis by biopsy | 124 (22.1) | 0 (0) | 0 (0) |
| Class III/IV lupus nephritis by biopsy | 173 (30.8) | 0 (0) | 0 (0) |
| Serological | |||
| ANA (⩾1:80) | 562 (100) | 971 (100) | 971 (100) |
| Anti-dsDNA antibody | 463 (82.4) | 61 (6.3) | 61 (6.3) |
| Anti-Sm antibody | 131 (23.3) | 20 (2.1) | 20 (2.1) |
| Anti-phospholipid antibodies (lupus anticoagulant, anticardiolipin or anti-β2glycoprotein-1 antibody) | 139 (24.7) | 19 (2.0) | 19 (2.0) |
| Low C3 or C4 | 501 (89.1) | 159 (16.4) | 159 (16.4) |
| Low C3 and C4 | 423 (75.3) | 53 (5.5) | 53 (5.5) |
ANA, antinuclear antibody; SLE, systemic lupus erythematosus.
Specificity of the anti-dsDNA assay
Using a cut-off value of 62.5 IU/ml as positivity for anti-dsDNA, the sensitivity and specificity of anti-dsDNA in the detection of a panel diagnosis of SLE as compared to control non-SLE patients was 80.1% and 94.5%, respectively (data not shown).
Performance of the SLE classification criteria
Table 3 shows the performance of the three SLE classification criteria for a clinical diagnosis of SLE. The sensitivity of the EULAR/ACR (cut-off score ⩾10), SLICC and ACR criteria in our patients were 96.1% (93.9–97.4%), 97.9% (96.3–98.9%) and 86.1% (83.0–88.9%), respectively. The sensitivity of the EULAR/ACR was similar to the SLICC criteria (p = 0.31) and both were more sensitive than the ACR criteria (p < 0.001).
Table 3.
Performance of the EULAR/ACR, SLICC and 1997 ACR criteria in our patients (with attribution rule applied to controls).
| Number | Sensitivity (95% CI) | Specificity (95% CI) | |||||
|---|---|---|---|---|---|---|---|
| EULAR/ACR | SLICC | ACR | EULAR/ACR | SLICC | ACR | ||
| All patients | 1533 | 96.1% (94.2–97.5%) | 97.9% (96.3–98.9%) | 86.1% (83.0–88.9%) | 95.0% (93.4–96.2%) | 92.5% (90.6–94.1%) | 98.8% (97.9–99.4%) |
| Female | 1352 | 96.2% (94.2–97.7%) | 97.9% (96.3–99.0%) | 85.9% (82.7–88.8%) | 94.4% (92.6–95.9%) | 91.9% (89.8–93.7%) | 98.8% (97.8–99.4%) |
| Male | 181 | 94.4% (81.3–99.3%) | 97.2% (85.5–99.9%) | 88.9% (73.9–96.9%) | 97.9% (94.1–99.6%) | 95.9% (91.2–98.5%) | 98.6% (95.1–99.8%) |
| ⩽50 years old | 888 | 96.7% (94.6–98.2%) | 98.2% (96.6–99.2%) | 87.5% (84.1–90.4%) | 92.4% (89.4–94.7%) | 89.1% (85.8–91.9%) | 97.9% (96.1–99.0%) |
| >50 years old | 645 | 93.4% (86.9–97.3%) | 96.2% (90.6–99.0%) | 80.2% (71.3–87.3%) | 97.0% (95.22–98.3%) | 95.2% (93.0–96.8%) | 99.4% (98.4–99.9%) |
| Disease duration ⩽3 years | 396 | 91.8% (81.9–97.3%) | 93.4% (84.1–98.2%) | 83.6% (71.9–91.9%) | 97.6% (95.4–99.0%) | 97.6% (95.4–99.0%) | 99.7% (98.4–99.9%) |
| Disease duration >3 years |
1137 | 96.6% (94.6–98.0%) | 98.4% (96.9–99.3%) | 86.4% (83.1–89.3%) | 93.6% (91.4–95.3%) | 89.8% (87.2–92.0%) | 98.3% (96.9–99.1%) |
ACR, American College of Rheumatology; CI, confidence interval; EULAR, European League of Association of Rheumatology; SLICC, Systemic Lupus International Collaborating Clinics.
The specificity of the EULAR/ACR, SLICC and ACR criteria for a clinical diagnosis of SLE were 85.8% (83.4–87.9%), 86.3% (84.0–88.4%) and 94.3% (92.7–95.7%), respectively. The specificity of EULAR/ACR was comparable to the SLICC criteria (p = 0.60) and both were less specific than the ACR criteria (p < 0.001).
Among the 562 clinical SLE patients, 4 did not meet the SLICC criteria (3 did not meet either the ACR or SLICC) but fulfilled the EULAR/ACR criteria. These were patients with positive anti-dsDNA and arthritis but did not meet the threshold of four criteria in the ACR or SLICC. Conversely, among the 971 non-SLE patients, 29 met either the ACR (n = 8) or SLICC (n = 27) but did not fulfil the EULAR/ACR criteria. Twenty-four (89%) patients who were ‘false positive’ for the SLICC criteria had cytopenia involving ⩾2 lineages or leukopenia with ⩾2 immunological features such that they fulfilled ⩾4 criteria of SLICC but not EULAR/ACR because only the highest score was taken in the same domain of the latter.
Application of an attribution rule to the non-SLE controls
As the investigators who scored the EULAR/ACR criteria were blinded for the panel diagnosis of SLE or non-SLE, the attribution rule was not applied to the non-SLE controls during initial analyses. In a separate model, we adjusted the EULAR/ACR scores in panel judged non-SLE patients if features could be explained by the non-SLE diagnosis. The fulfilment of the EULAR/ACR criteria in the non-SLE patients, with and without the attribution rule, was also shown in Table 2. Of 616 patients with scores adjusted, 506 scored less than 10 points in the original model and the fulfilment of the EULAR/ACR criteria was not affected. With the attribution rule applied to the non-SLE controls, while the sensitivity remained the same, the specificity of the three criteria increased to 95.0%, 92.5% and 98.8%, respectively.
ROC analysis was performed on the EULAR/ACR criteria for the cut-off score which was associated with the best sensitivity and specificity for a clinical diagnosis of SLE (Figure 1). The cut-off score with the highest Youden’s index was 12 [area under the curve (AUC) = 0.994; sensitivity at 95.0%, specificity 96.6%]. The sensitivity and specificity of the EULAR/ACR criteria with cut-off scores from 8 to 14 are shown in Table 4.
Figure 1.
ROC curve analysis of the EULAR/ACR SLE criteria cut-off in our patients (with attribution rule applied to controls). The AUC of the 2019 EULAR/ACR was 0.994. The arrow denotes cut-off point of 12 with a sensitivity of 95.0% and specificity of 96.6%.
Table 4.
Sensitivity and specificity of the EULAR/ACR criteria for clinical SLE using different cut-off scores (with attribution rule applied to controls).
| EULAR/ACR criteria cut-off score | Sensitivity (%) | Specificity (%) | Youden’s index |
|---|---|---|---|
| 8 | 99.1 | 91.1 | 0.902 |
| 9 | 98.2 | 92.2 | 0.904 |
| 10 | 96.4 | 94.7 | 0.911 |
| 11 | 95.2 | 96.2 | 0.914 |
| 12 | 95.0 | 96.6 | 0.916 |
| 13 | 93.1 | 97.9 | 0.910 |
| 14 | 90.6 | 99.2 | 0.898 |
ACR, American College of Rheumatology; EULAR, European League of Association of Rheumatology; SLE, systemic lupus erythematosus.
Subgroups of SLE patients
The performance of the three criteria for a clinical diagnosis of SLE in different subgroups is also shown in Table 3. In female patients (n = 1352), the performance of the three criteria for SLE was comparable to the whole study population. In male patients (n = 181), the sensitivity of the three criteria was similar to the whole population but specificity was higher for the EULAR/ACR and SLICC criteria (97.9%, 95.9% versus 95.0%, 92.5%, respectively).
Regarding the age at first clinic attendance, the EULAR/ACR criteria showed a lower sensitivity (93.4% versus 96.7%) but higher specificity (97.0% versus 92.4%) in patients older than 50 years when compared with patients younger than 50 years. Similarly, the SLICC criteria also exhibited lower sensitivity (96.2% versus 98.2%) but higher specificity (95.2% versus 89.1%) in older patients. The specificity of 1997 ACR criteria for older patients was also higher than that of the younger age group (99.4% versus 97.9%). All the three criteria were less sensitive but more specific for SLE in patients with a shorter disease duration (⩽3 years) than those with a longer disease duration (>3 years) (Table 3).
Forty-eight clinical SLE patients with a disease duration of 4–5 years were further reviewed for the timing of fulfilment of the three criteria (Table 5). Fulfilment of the EULAR/ACR criteria was achieved in 97.9% of these patients at diagnosis and throughout the first 4 years. Fewer patients fulfilled the ACR and SLICC criteria at diagnosis, but the percentages increased over time for these criteria.
Table 5.
Timing of fulfilment of the 3 SLE classification criteria in 48 SLE patients.
| Fulfilment of criteria | Year 0 | Year 1 | Year 2 | Year 3 | Year 4 |
|---|---|---|---|---|---|
| ACR | 42 (87.5%) | 43 (89.5%) | 43 (89.5%) | 44 (91.7%) | 44 (91.7%) |
| SLICC | 46 (95.8%) | 46 (95.8%) | 46 (95.8%) | 47 (97.9%) | 47 (97.9%) |
| EULAR/ACR | 47 (97.9%) | 47 (97.9%) | 47 (97.9%) | 47 (97.9%) | 47 (97.9%) |
ACR, American College of Rheumatology; EULAR, European League of Association of Rheumatology; SLE, systemic lupus erythematosus; SLICC, Systemic Lupus International Collaborating Clinics.
Discussion
Classification criteria of SLE are being refined in recent decades to improve their performance. The latest 2019 EULAR/ACR criteria were developed to maintain the high sensitivity of 2012 SLICC criteria while improving the specificity as compared to the 1997 ACR criteria. In our Chinese patients, the EULAR/ACR criteria had a high sensitivity that is comparable to the SLICC criteria. However, unlike the original validation study in non-Chinese patients, 10 we could not demonstrate improved specificity of the EULAR/ACR as compared to the SLICC criteria. However, with the application of attribution rule to non-SLE controls, the specificity of all the three criteria increased. Both the EULAR/ACR and SLICC were more sensitive than the ACR criteria and their specificity approached that of the ACR criteria. The EULAR/ACR and the SLICC criteria perform equally well in terms of sensitivity and specificity. Our results are in agreement with the recent validation study from Petri et al. 23 using the original physician diagnoses of patient scenarios for the SLICC criteria. It was demonstrated that both the EULAR/ACR and SLICC criteria had higher sensitivity but lower specificity than the 1997 ACR criteria. The specificity of the EULAR/ACR was only slightly higher than the SLICC criteria for SLE (89% versus 84%). Two weighted classification rules of the SLICC criteria did not enhance the performance and researchers may prefer the unweighted criteria. All the three sets of criteria had similar agreement with the physician diagnoses. Hence, both the EULAR/ACR and SLICC criteria were recommended by the authors for SLE classification. 23
Other Asian studies showed heterogeneous results in the performance of the SLE criteria. Teng et al. 15 showed that the EULAR/ACR criteria had improved specificity (90.3%) when compared with SLICC (84%) but was not as good as the ACR criteria (96%) in Chinese patients. Lee et al. 17 showed similar specificity of all three SLE criteria in their Korean patients. Jin et al. 21 studied 2097 Chinese patients with SLE and isolated cutaneous lupus. The sensitivity of the EULAR/ACR criteria for SLE was 96.8%, which was similar to the SLICC criteria (98.3%). However, a low specificity was found across all the three criteria (63.4–68.5%) for distinguishing SLE from cutaneous lupus. As the method of ascertainment of a clinical diagnosis of SLE and the inclusion of non-SLE controls is different in these Asian studies, direct comparison of the results with our study may not be appropriate.
There are several possible explanations for the lower specificity of the EULAR/ACR criteria for SLE in our study as compared to others in our original analyses.10,15–24 We had only included patients who were ANA positive at titres of ⩾1:80, while other studies included clinically diagnosed SLE and non-SLE controls irrespective of their ANA status. The majority, if not all, of patients with SLE is expected to be ANA positive with titers ⩾1:80. Had we included ANA-negative or low-titre ANA-positive patients in the analysis, the calculated specificity would increase because of the greater total number of ‘true negative’ patients. As the sum of weight scores of items instead of the number of items are adopted in the EULAR/ACR criteria, it is essential that features that are more related to an alternative diagnosis should not be scored. Using this attribution rule to score the non-SLE controls, the specificity of all the three criteria increased but the performance of the EULAR/ACR and SLICC criteria was comparable in our patients.
On the contrary, our patients are exclusively Chinese, who are more prone to have renal and gastrointestinal involvement by SLE as compared to the Caucasians.29,30 The prevalence of lupus renal disease was 53% in our study and 8.3% of patients had intestinal and hepatic manifestations. In other Asian series of SLE, gastrointestinal manifestations were reported to occur in up to 18% of patients. 31 However, gastrointestinal manifestations, such as protein losing enteropathy and intestinal vasculitis, are not included in the EULAR/ACR criteria. The difference in the prevalence of these organ manifestations in Asian patients may affect the weighted scores in the EULAR/ACR criteria, leading to a difference in the performance and best cut-off score.
Lee et al. 17 reported higher specificity of the EULAR/ACR criteria for SLE without losing sensitivity when a cut-off score of 12 was used instead of 10 in their Korean patients. This is similar to our finding that the specificity of the EULAR/ACR was enhanced by a cut-off score of 12 while maintaining good sensitivity. Two recent paediatric SLE series from the Middle East and South America also showed higher specificity of the EULAR/ACR criteria for SLE by raising the cut-off score to 13.32,33 Thus, further works are necessary to clarify the weighted scores of renal and gastrointestinal manifestations in Asian patients and the optimal cut-off score in the EULAR/ACR classification criteria.
A higher specificity was observed in our male SLE patients using all the three sets of criteria. This is consistent with the study by Johnson et al. 18 which included 1270 predominantly White SLE patients (172 men) and showed that the specificity of the EULAR/ACR, SLICC and 1997 ACR criteria was numerically higher in male than female patients (96% versus 94% for the EULAR/ACR criteria; 90% versus 82% for the SLICC criteria; 94% versus 93% in the ACR criteria). On the contrary, onset of SLE after the age of 50 years is uncommon, comprising only 9–10% of all SLE patients.34,35 Late-onset SLE patients were reported to be more insidious in onset and clinical presentation was often less aggressive with lower disease activity score. 36 Typical manifestations of SLE, such as malar rash, arthritis and nephritis, are less common 34 and a longer time lag may occur between symptom onset and diagnosis in these patients.35,37 It has been reported that only a small proportion of late-onset SLE patients fulfilled SLE classification criteria at symptom onset. 37 This could explain the lower sensitivity of all the three criteria for classifying SLE in our older patients.
As patients with incomplete SLE may develop full-blown disease eventually, criteria that enhance the sensitivity for SLE classification may allow intervention for early disease. However, our study cannot demonstrate an improved sensitivity of the EULAR/ACR in comparison to the SLICC criteria in patients with shorter disease duration. In fact, all the three SLE criteria were shown to have lower sensitivity but higher specificity in patients with a follow-up duration of less than 3 years. As SLE manifestations cumulate over time, patients with longer disease duration would be more likely to fulfil the SLE classification items, leading to a greater number of ‘true positive’ and hence increased sensitivity. While there is still no standardized definition for early SLE, previous validation studies analysed the performance of the SLE criteria in subgroups of patients with 1–5 years of disease onset. In the study by Johnson et al., 18 the sensitivity of the EULAR/ACR criteria for SLE in patients with <1 year of disease onset was 89%, and the sensitivity increased to 96% in patients with ⩾5 years of disease duration. Similarly, Adamichou et al. 22 reported a lower sensitivity of the EULAR/ACR criteria (87.3%) in patients with <3 years of SLE duration. However, the above studies referred to the overall data in subgroups of patients with different disease duration. In our study, further analysis of 48 patients with a disease duration of 4–5 years showed that more SLE patients fulfilled the EULAR/ACR than the ACR or SLICC during the first 2 years of diagnosis, indicating a higher potential of the EULAR/ACR criteria in picking up early SLE.
The strength of our study is the large sample size as compared with other Asian cohorts. We have included a full range of non-SLE patients from the rheumatology clinics for this validation study. Moreover, all SLE patients in our hospitals, including those with renal and haematological and neuropsychiatric manifestations, are followed in the rheumatology clinics. The inclusion of all consecutive SLE patients from our clinics in the specified time period helps minimize selection bias.
There are several limitations of our study. First, the judgement for a clinical diagnosis of SLE was based on retrospective record review by a limited number of rheumatologists. Clinical features for the SLE criteria that were missed in the medical records were assumed to be absent during data acquisition. Second, we have only included ANA-positive patients. As patients with ANA titers of less than 1:80 or a negative ANA test would not fulfil the EULAR/ACR criteria because of the failure to meet the entry criterion, exclusion of these patients would have lowered the specificity because of the lower numbers of ‘true negative’ cases. The performance of the SLE criteria in our study would apply to patients with positive ANA only. Finally, the exact time of onset of the SLE symptoms might not be fully documented in the medical records. The time of presentation to our clinics was assumed to be that of symptom onset if there was no specification of the duration of individual symptoms. This might affect the performance of the criteria in subgroups of patients with different disease duration.
In conclusion, in this study of ANA-positive Chinese patients in the out-patient rheumatology clinics, the 2019 EULAR/ACR criteria are more sensitive but less specific than the ACR criteria for a clinical diagnosis of SLE. The performance of the EULAR/ACR is similar to the SLICC criteria. The specificity of all the three criteria is increased by the application of the attribution rule to controls. The specificity of the EULAR/ACR criteria in our ANA-positive patients could be enhanced by increasing the cut-off score to 12. As the performance of a classification rule is dependent on the populations from which the cases and non-cases are derived, further analyses are needed to clarify the optimal weighting of each item and cut-off score in the EULAR/ACR criteria for classifying SLE in Chinese patients. At present, both the EULAR/ACR and SLICC criteria are suitable to classify SLE in southern Chinese patients.
Supplemental Material
Supplemental material, sj-pdf-1-tab-10.1177_1759720X221100300 for Validation of the 2019 EULAR/ACR classification criteria for systemic lupus erythematosus in ANA-positive Chinese patients by Yuen Kwan Chung, Ling Yin Ho, Carolyn Lee, Chi Hung To and Chi Chiu Mok in Therapeutic Advances in Musculoskeletal Disease
Footnotes
Ethics approval and consent to participate: This sutdy was approved by the Research Ethics Committee of our hospitals.
Author contribution(s): Yuen Kwan Chung: Data curation; Formal analysis; Investigation; Methodology; Software; Writing – original draft.
Ling Yin Ho: Data curation; Investigation; Methodology; Supervision; Writing – review & editing.
Carolyn Lee: Data curation; Formal analysis; Investigation; Methodology; Writing – review & editing.
Chi Hung To: Conceptualization; Data curation; Investigation; Methodology; Project administration; Supervision; Validation; Writing – review & editing.
Chi Chiu Mok: Conceptualization; Data curation; Investigation; Methodology; Project administration; Supervision; Validation; Writing – review & editing.
ORCID iD: Chi Chiu Mok 
https://orcid.org/0000-0003-3696-1228
Funding: The authors received no financial support for the research, authorship and/or publication of this article.
Conflict of interest statement: The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Supplemental Material: Supplemental material for this article is available online.
Contributor Information
Yuen Kwan Chung, Department of Medicine, Tuen Mun Hospital, Tuen Mun, Hong Kong SAR, China.
Ling Yin Ho, Department of Medicine, Tuen Mun Hospital, Tuen Mun, Hong Kong SAR, China.
Carolyn Lee, Department of Medicine, Pok Oi Hospital, Yuen Long, Hong Kong SAR, China.
Chi Hung To, Department of Medicine, Pok Oi Hospital, Yuen Long, Hong Kong SAR, China.
Chi Chiu Mok, Chief of Rheumatology, Department of Medicine, Tuen Mun Hospital, Tsing Chung Koon Road, Tuen Mun, New Territories, Hong Kong SAR, China.
References
- 1. Mok CC, Kwok RC, Yip PS. Effect of renal disease on the standardized mortality ratio and life expectancy of patients with systemic lupus erythematosus. Arthritis Rheum 2013; 65: 2154–2160. [DOI] [PubMed] [Google Scholar]
- 2. Chen YM, Lin CH, Chen HH, et al. Onset age affects mortality and renal outcome of female systemic lupus erythematosus patients: a nationwide population-based study in Taiwan. Rheumatology 2014; 53: 180–185. [DOI] [PubMed] [Google Scholar]
- 3. Fors Nieves CE, Izmirly PM. Mortality in systemic lupus erythematosus: an updated review. Curr Rheumatol Rep 2016; 18: 21. [DOI] [PubMed] [Google Scholar]
- 4. Fiehn C, Hajjar Y, Mueller K, et al. Improved clinical outcome of lupus nephritis during the past decade: importance of early diagnosis and treatment. Ann Rheum Dis 2003; 62: 435–439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Arnaud L, Tektonidou MG. Long-term outcomes in systemic lupus erythematosus: trends over time and major contributors. Rheumatology 2020; 59(Suppl. 5): v29–v38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Rees F, Doherty M, Lanyon P, et al. Early clinical features in systemic lupus erythematosus: can they be used to achieve earlier diagnosis? A risk prediction model. Arthritis Care Res 2017; 69: 833–841. [DOI] [PubMed] [Google Scholar]
- 7. Leuchten N, Milke B, Winkler-Rohlfing B, et al. Early symptoms of systemic lupus erythematosus (SLE) recalled by 339 SLE patients. Lupus 2018; 27: 1431–1436. [DOI] [PubMed] [Google Scholar]
- 8. Hochberg MC. Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum 1997; 40: 1725. [DOI] [PubMed] [Google Scholar]
- 9. Petri M, Orbai AM, Alarcón GS, et al. Derivation and validation of the Systemic Lupus International Collaborating Clinics classification criteria for systemic lupus erythematosus. Arthritis Rheum 2012; 64: 2677–2686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Aringer M, Costenbader K, Daikh D, et al. 2019 European League Against Rheumatism/American College of Rheumatology classification criteria for systemic lupus erythematosus. Arthritis Rheum 2019; 71: 1400–1412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Boers A, Li Q, Wong M, et al. Differences in SLE disease activity between patients of Caucasian and South-East Asian/Chinese background in an Australian hospital. APLAR J Rheumatol 2006; 9: 43–48. [Google Scholar]
- 12. Kaslow RA. High rate of death caused by systemic lupus erythematosus among U. S. residents of Asian descent. Arthritis Rheum 1982; 25: 414–418. [DOI] [PubMed] [Google Scholar]
- 13. Samanta A, Feehally J, Roy S, et al. High prevalence of systemic disease and mortality in Asian subjects with systemic lupus erythematosus. Ann Rheum Dis 1991; 50: 490–492. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Mok CC, Hamijoyo L, Kasitanon N, et al. The Asia-Pacific League of Associations for Rheumatology consensus statements on the management of systemic lupus erythematosus. Lancet Rheumatol 2021; 3: e517–e531. [DOI] [PubMed] [Google Scholar]
- 15. Teng J, Ye J, Zhou Z, et al. A comparison of the performance of the 2019 European League Against Rheumatism/American College of Rheumatology criteria and the 2012 Systemic Lupus International Collaborating Clinics criteria with the 1997 American College of Rheumatology classification criteria for systemic lupus erythematosus in new-onset Chinese patients. Lupus 2020; 29: 617–624. [DOI] [PubMed] [Google Scholar]
- 16. Suda M, Kishimoto M, Ohde S, et al. Validation of the 2019 ACR/EULAR classification criteria of systemic lupus erythematosus in 100 Japanese patients: a real-world setting analysis. Clin Rheumatol 2020; 39: 1823–1827. [DOI] [PubMed] [Google Scholar]
- 17. Lee EE, Lee EB, Park JK, et al. Performance of the 2019 European League Against Rheumatism/American College of Rheumatology classification criteria for systemic lupus erythematosus in Asian patients: a single-centre retrospective cohort study in Korea. Clin Exp Rheumatol 2020; 38: 1075–1079. [PubMed] [Google Scholar]
- 18. Johnson SR, Brinks R, Costenbader KH, et al. Performance of the 2019 EULAR/ACR classification criteria for systemic lupus erythematosus in early disease, across sexes and ethnicities. Ann Rheum Dis 2020; 79: 1333–1339. [DOI] [PubMed] [Google Scholar]
- 19. Pons-Estel GJ, Ugarte-Gil MF, Harvey GB, et al. Applying the 2019 EULAR/ACR lupus criteria to patients from an established cohort: a Latin American perspective. RMD Open 2020; 6: e001097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Dahlström Ö, Sjöwall C. The diagnostic accuracies of the 2012 SLICC criteria and the proposed EULAR/ACR criteria for systemic lupus erythematosus classification are comparable. Lupus 2019; 28: 778–782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Jin H, Huang T, Wu R, et al. A comparison and review of three sets of classification criteria for systemic lupus erythematosus for distinguishing systemic lupus erythematosus from pure mucocutaneous manifestations in the lupus disease spectrum. Lupus 2020; 29: 1854–1865. [DOI] [PubMed] [Google Scholar]
- 22. Adamichou C, Nikolopoulos D, Genitsaridi I, et al. In an early SLE cohort the ACR-1997, SLICC-2012 and EULAR/ACR-2019 criteria classify non-overlapping groups of patients: use of all three criteria ensures optimal capture for clinical studies while their modification earlier classification and treatment. Ann Rheum Dis 2020; 79: 232–241. [DOI] [PubMed] [Google Scholar]
- 23. Petri M, Goldman DW, Alarcón GS, et al. A Comparison of 2019 EULAR/ACR SLE classification criteria with two sets of earlier SLE classification criteria. Arthritis Care Res 2021; 73: 1231–1235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Wang H, Gao Y, Ma Y, et al. Performance of the 2019 EULAR/ACR systemic lupus erythematosus classification criteria in a cohort of patients with biopsy-confirmed lupus nephritis. Lupus Sci Med 2021; 8: e000458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Zavada J, Dixon WG, Askling J, et al. Launch of a checklist for reporting longitudinal observational drug studies in rheumatology: a EULAR extension of STROBE guidelines based on experience from biologics registries. Ann Rheum Dis 2014; 73: 628. [DOI] [PubMed] [Google Scholar]
- 26. Bombardier C, Gladman DD, Urowitz MB, et al. Derivation of the SLEDAI. A disease activity index for lupus patients. The Committee on Prognosis Studies in SLE. Arthritis Rheum 1992; 35: 630–640. [DOI] [PubMed] [Google Scholar]
- 27. MedCalc Software Ltd. Diagnostic test evaluation calculator, version 20.013, https://www.medcalc.org/calc/diagnostic_test.php
- 28. Martínez-Camblor P, Pardo-Fernández JC. The Youden index in the generalized receiver operating characteristic curve context. Int J Biostat 2019; 15: 20180060. [DOI] [PubMed] [Google Scholar]
- 29. Connelly K, Morand EF, Hoi AY. Asian ethnicity in systemic lupus erythematosus: an Australian perspective. Intern Med J 2013; 43: 618–624. [DOI] [PubMed] [Google Scholar]
- 30. Mok MY, Li WL. Do Asian patients have worse lupus? Lupus 2010; 19: 1384–1390. [DOI] [PubMed] [Google Scholar]
- 31. Chng HH, Tan BE, Teh CL, et al. Major gastrointestinal manifestations in lupus patients in Asia: lupus enteritis, intestinal pseudo-obstruction, and protein-losing gastroenteropathy. Lupus 2010; 19: 1404–1413. [DOI] [PubMed] [Google Scholar]
- 32. Fonseca AR, Gaspar-Elsas MI, Land MG, et al. Comparison between three systems of classification criteria in juvenile systemic lupus erythematosus. Rheumatology 2015; 54: 241–247. [DOI] [PubMed] [Google Scholar]
- 33. Abdwani R, Masroori E, Abdullah E, et al. Evaluating the performance of ACR, SLICC and EULAR/ACR classification criteria in childhood onset systemic lupus erythematosus. Pediatr Rheumatol 2021; 19: 141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Cervera R, Khamashta MA, Hughes GR. The Euro-lupus project: epidemiology of systemic lupus erythematosus in Europe. Lupus 2009; 18: 869–874. [DOI] [PubMed] [Google Scholar]
- 35. Lalani S, Pope J, de Leon F, et al. Clinical features and prognosis of late-onset systemic lupus erythematosus: results from the 1000 faces of lupus study. J Rheumatol 2010; 37: 38–44. [DOI] [PubMed] [Google Scholar]
- 36. Formiga F, Moga I, Pac M, et al. Mild presentation of systemic lupus erythematosus in elderly patients assessed by SLEDAI. Lupus 1999; 8: 462–465. [DOI] [PubMed] [Google Scholar]
- 37. Ho CT, Mok CC, Lau CS, et al. Late onset systemic lupus erythematosus in southern Chinese. Ann Rheum Dis 1998; 57: 437–440. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-pdf-1-tab-10.1177_1759720X221100300 for Validation of the 2019 EULAR/ACR classification criteria for systemic lupus erythematosus in ANA-positive Chinese patients by Yuen Kwan Chung, Ling Yin Ho, Carolyn Lee, Chi Hung To and Chi Chiu Mok in Therapeutic Advances in Musculoskeletal Disease