Comparison of the American-European Consensus Group Sjögren's syndrome classification criteria to newly proposed American College of Rheumatology criteria in a large, carefully characterized sicca cohort

Abstract

Objective

To compare the performance of the American-European Consensus Group (AECG) and the newly proposed American College of Rheumatology (ACR) classification criteria for Sjögren's syndrome in a well-characterized sicca cohort, given ongoing efforts to resolve discrepancies and weaknesses in the systems.

Methods

In a multidisciplinary clinic for the evaluation of sicca, we assessed features of salivary and lacrimal gland dysfunction and autoimmunity as defined by tests of both AECG and ACR criteria in 646 participants. Global gene expression profiles were compared in a subset of 180 participants.

Results

Application of the AECG and ACR criteria resulted in classification of 279 and 268 participants with SS, respectively. Both criteria were met by 244 participants (81%). In 26 of the 35 AECG+/ACR- participants, the minor salivary gland biopsy focal score was ≥1 (74%), while 9 had positive anti-Ro/La (26%). There were 24 AECG-/ACR+ who met ACR criteria mainly due to differences in the scoring of corneal staining. All patients with SS, regardless of classification, had similar gene expression profiles, which were distinct from the healthy controls.

Conclusion

The two sets of classification criteria yield concordant results in the majority of cases and gene expression profiling suggests that patients meeting either set of criteria are more similar to other SS participants than to healthy controls. Thus, there is no clear evidence for increased value of the new ACR criteria over the old AECG criteria from the clinical or biological perspective. It is our contention, supported by this report, that improvements in diagnostic acumen will require a more fundamental understanding of the pathogenic mechanisms than is at present available.

Introduction

Sjögren's syndrome (SS) is a chronic, systemic disease that may be second only to rheumatoid arthritis in prevalence among the rheumatic autoimmune diseases.[1, 2] The principal manifestations of the disease are dry eyes and dry mouth resulting from immune mediated damage and dysfunction of the lacrimal and salivary glands[3, 4] which develop a characteristic lymphocytic infiltrate that can be objectively measured with a focus score.[5] Approximately 67% of patients with SS have circulating autoantibodies to anti-Ro (SSA) and/or anti-La (SSB).[6] Extra-glandular manifestations, which include vasculitis, peripheral neuropathy, renal tubular acidosis, pulmonary involvement, lymphoproliferative disease and/or immunological abnormalities, are present in a subset of patients and found most commonly among those with high levels of anti-Ro and anti-La autoantibodies.[7, 8]

The diagnosis of SS commonly requires a multidisciplinary approach and may be difficult to establish. Sicca symptoms are common, non-specific, and there is no gold standard diagnostic test. For research purposes, 11 sets of classification criteria have been proposed since the mid-1960's.[9-19] The last of these, the 2002 revised American-European Consensus Group (AECG) Classification Criteria, have had widespread acceptance and adoption in clinical and research studies of SS, having been cited >1500 times.[20] They consist of six criteria, two subjective and four objective (Table 1).[19] In 2012, the American College of Rheumatology (ACR) endorsed a new set of preliminary criteria proposed by the Sjögren's International Collaborative Clinical Alliance.[21, 22] These criteria are centered around three objective features (Table 1).

Table 1.

Comparison of the Revised American-European Consensus Group (AECG) Classification criteria and the American College of Rheumatology (ACR) Classification criteria for Sjögren's syndrome.

AECG Classification1	ACR Classification2
Inclusion criteria
I. Ocular symptoms: a positive response to at least one of the following questions: Have you had daily, persistent, troublesome dry eyes for more than 3 months? Do you have a recurrent sensation of sand or gravel in the eyes? Do you use tear substitutes more than 3 times a day?	None
II. Oral symptoms: a positive response to at least one of the following questions: Have you had a daily feeling of dry mouth for more than 3 months? Have you had recurrently or persistently swollen salivary glands as an adult? Do you frequently drink liquids to aid in swallowing dry food?	None
III. Ocular signs - that is, objective evidence of ocular involvement defined as a positive result for at least one of the following two tests: Schirmer's I test, performed without anesthesia (≤5 mm in 5 minutes) Rose Bengal score or other ocular dye score (≥4 according to van Bijsterveld's scoring system)	Keratoconjunctivitis sicca with ocular staining score ≥3 (assuming that individual is not currently using daily eye drops for glaucoma and has not had corneal surgery or cosmetic eyelid surgery in the last 5 years)
IV. Histopathology: in minor salivary glands (obtained through normal appearing mucosa) focal lymphocytic sialoadenitis, evaluated by an expert histopathologist, with a focus score ≥ 1, defined as number of lymphocytic foci (which are adjacent to normal-appearing mucous acini and contain more than 50 lymphocytes) per 4 mm2 of glandular tissue	Labial salivary gland biopsy exhibiting focal lymphocytic sialadenitis with a focus score ≥1 focus/4 mm2
V. Salivary gland involvement: objective evidence of salivary gland involvement defined by a positive result for at least one of the following diagnostic tests: Unstimulated whole salivary flow (≤1.5 ml in 15 min) Parotid sialography3 showing the presence of diffuse sialectasias (punctate, cavitary or destructive pattern), without evidence of obstruction in major ducts Salivary scintigraphy4 showing delayed uptake, reduced concentration and/or delayed excretion of tracer	None
VI. Autoantibodies: presence in the serum of the following autoantibodies: 1. Antibodies to Ro (SSA) or La (SSB) antigens, or both	Positive serum anti-SSA/Ro and/or anti-SSB/La or (positive rheumatoid factor and ANA titer ≥1:320)
Classification rules
For Primary SS:	For Sjögren's Syndrome:
In patients without any potentially associated disease, primary SS may be defined as follows: The presence of any 4 of the 6 items is indicative of primary SS, as long as either item IV (Histopathology) or VI (Serology) is positive The presence of any 3 of the 4 objective criteria items (that is, items III, IV, V, VI) The classification tree procedure represents a valid alternative method for classification, although it should be more properly used in clinical-epidemiological survey	The classification of SS, which applies to individuals with signs/symptoms that may be suggestive of SS, will be met in patients who have at least 2 of the 3 objective features previously described
For Secondary SS:
In patients with a potentially associated disease (for instance, another well defined connective tissue disease), the presence of item 1 or item II plus any 2 from among items III, IV and V may be considered as indicative of secondary SS	Eliminated the distinction between primary and secondary forms of SS
Exclusion criteria
Past head and neck radiation treatment Hepatitis C infection Acquired Immunodeficiency Syndrome (AIDS) Pre-existing lymphoma Sarcoidosis Graft versus host disease Use of anticholinergic drugs (since a time shorter than 4-fold the half life of the drug)	Prior diagnosis of any of the following conditions would exclude participation in SS studies or therapeutic trials because of overlapping clinical features or interference with criteria tests: History of head and neck radiation treatment Hepatitis C infection Acquired Immunodeficiency Syndrome Sarcoidosis Amyloidosis Graft versus host disease IgG4-related disease

¹Revised AECG classification criteria[19];

²ACR criteria[21];

³Sialography and

⁴Scintigraphy are tests that were not performed in the current study.

We undertook this study to compare the new ACR criteria to the revised AECG criteria in a cohort of participants with sicca symptoms that have been carefully evaluated for Sjögren's syndrome.

Methods

Participant Recruitment

The participating subjects were evaluated in the Sjögren's Research Clinic at Oklahoma Medical Research Foundation or at a similar clinic at the University of Minnesota. Participants were self or physician-referred. Each potential clinic participant was interviewed by phone by trained personnel who assessed the presence of ocular and oral symptoms by asking the six standardized and validated [17] questions in the subjective criteria of the revised AECG Classification Criteria (Table 1).[19] In order to be eligible for an appointment at the clinic, at least one ocular and one oral question had to be answered affirmatively. The exclusion criteria for evaluation at the clinic were also based on the recommendations of the AECG (Table 1).[19] In addition, we excluded individuals that presented with known current pregnancy or inability to provide informed consent.

With very few exceptions, participants were evaluated in a single morning clinic visit using standardized protocols. Patients underwent an oral exam consisting of measurement of stimulated and timed whole unstimulated salivary flow (WUSF), a lip biopsy and collection and storage of saliva. Participant evaluation did not include sialography or scintigraphy. The ocular specialist performed ocular surface staining with lissamine green and fluorescein, a unanesthesized Schirmer's I test, and collection and storage of tears. The ocular vital dye score was determined using the quantitative dot-counting method [23] rather than by descriptive features [24] and the score for each section was recorded independently before generating a final score for each eye. Blood samples were collected for general laboratory tests and extraction of DNA, RNA, and serum. A physician completed a detailed history and physical examination, including general medical, rheumatological and neurological evaluations. If patients gave a history of a past diagnosis of rheumatoid arthritis, mixed connective tissue disease, systemic sclerosis, myositis, primary biliary cirrhosis, multiple sclerosis, or systemic lupus erythematosus, classification criteria for these illnesses were specifically ascertained by history, medical record review and testing for the corresponding autoantibodies.

All procedures were approved by the Oklahoma Medical Research Foundation and University of Minnesota Institutional Review Boards. Each participant provided written informed consent prior to entering the study.

Biopsy

The dentist performed lip biopsies to obtain minor salivary glands in all patients, unless slides from a previous biopsy were available and contained sufficient tissue for re-examination by our pathologists. A portion of each specimen was formalin-fixed and paraffin-embedded, sections were cut and stained with hematoxylin-eosin while other fragments were cryologically preserved. Two dental pathologists reviewed the specimens independently; the results were compared and a consensus reading was generated. The lymphocytic infiltration of the glands was graded by focus score.[5]

Clinical laboratory and serology

Anti-Ro/SSA and anti-La/SSB autoantibodies were determined by multiple methods. In addition, all patients were tested for rheumatoid factor (RF), ANA, precipitins for autoantibodies associated with other connective tissue disorders, hepatitis C serology, CBC with differential, immunoglobulin profile and urinalysis (see online supplementary text).

Classification

Each study participant was classified according to both the revised AECG,[19] and to the newly proposed ACR criteria.[21] We eliminated from analysis the participants that did not have results for all the features of both classification systems with the exception of sialography and scintigraphy (Table 1).

Peripheral Blood mRNA Transcript Measurements

Global gene expression profiles comprising transcript levels for >15,000 loci were compared in a subset of 180 participants (see online supplementary text).

Statistical Analysis

Performance of the tests was assessed via sensitivity, specificity, positive predictive value and negative predictive value estimated by considering the AECG criteria as the “gold standard”, and summarizing the results with exact binomial 95% confidence intervals (95% CI). McNemar's Test of paired samples was used to assess whether the two sets of criteria were significantly different with respect to dichotomous variables. The Kappa statistic was used to quantify the degree of agreement between the new classification criteria and the AECG criteria. Details of the statistical analyses for the gene expression data are available in the see online supplementary text.

Results

The initial cohort of participants evaluated at either the Sjögren's Research Clinic at Oklahoma Medical Research Foundation or the Sjögren's Clinic in the University of Minnesota comprised 837 individuals. Of these, 646 had all data points of both AECG and ACR classification criteria and thus constitute the study cohort. The demographic characteristics of both cohorts are comparable in makeup with respect to age, sex, race and ethnicity (supplementary Table S1).

We tabulated the presence or absence of each of the 6 AECG classification criteria for SS and each of the 3 ACR criteria (summary in Table 2; details in supplementary Table S2). Of the 646 study participants, 279 and 268 patients were classified as SS according to AECG and ACR criteria, respectively. Of the 303 participants classified by either system as SS, 244 (81%) individuals met both sets of criteria.

Table 2.

Comparison of classification by AECG criteria versus ACR criteria for Sjögren's Syndrome.

		AECG criteria				*ACR criteria	**AECG criteria
		SS	DNMC	Total		Estimated value (95% CI)	Estimated value (95% CI)
ACR	SS	244	24	268	Sensitivity	0.87 (0.83-0.91)	0.91 (0.87-0.94)
	DNMC	35	343	378	Specificity	0.93 (0.90-0.96)	0.91 (0.87-0.93)
Total		279	367	646	PPV	0.91 (0.87-0.94)	0.87 (0.83-0.91)
					NPV	0.90 (0.87-0.93)	0.93 (0.90-0.96)

AECG=American European Consensus Group Classification Criteria for Sjögren's syndrome; ACR=American College of Rheumatology Criteria Classification Criteria for Sjögren's syndrome; SS= Sjögren's syndrome; DNMC=did not meet criteria; p = 0.19 by Mc Nemar's Test; Kappa = 0.81 (95% confidence intervals 0.77 – 0.86).

^*Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the ACR criteria considering the AECG criteria as the “gold standard”

^**Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the AECG criteria considering the ACR criteria as the “gold standard”

The comparison of the new ACR classification criteria to the AECG criteria (Table 2) shows that they are not significantly different (McNemar's test of paired samples: p=0.19) and there was a concordance rate of 0.81 (95% CI=0.77-0.86) based on the Kappa statistic. The analysis of the sensitivity, specificity, positive predictive value and negative predictive value of each set of criteria was done using the other criteria as the gold standard and was similar for both classification systems. The sensitivity of the ACR criteria was 87.5 (95% CI=82.9-90.9) with a specificity of 93.4 (95% CI=90.3-95.7); the positive predictive value was 91.0 (95%CI=86.8-94.0) and the negative predictive value was 90.7 (95%CI=87.2-93.4). Thus, 12.5% (35 of 279) of participants classified as SS under the AECG criteria were not considered SS when evaluated by the ACR criteria; conversely, 8.9% (24 of 268) met only the ACR criteria.

The differences between how the two systems classified the sicca participants revolved around which objective measures of ocular and oral involvement were included in addition to the histology and Ro/La serology. Namely, the van Bijsterveld (vBS) grading system of ocular staining, Schirmer's I test, and WUSF volume for the AECG criteria, and a different version of the first of these (the SICCA ocular staining score or OSS) for the ACR criteria plus positive ANA (≥1:320) and positive RF as an alternative measure of serological activity.

We compared the performance of these measures in the participants that met and did not meet either or both sets of criteria. When analyzing the characteristics of the SS patients as defined by AECG criteria that were excluded by ACR criteria (ACR-/AECG+ participants; Table 3), the most striking feature was that 26 of the 35 (74.3%)of them had a minor salivary gland biopsy with a focus score ≥1, while the other 9 (25.7%) had positive Ro and/or La auto-antibodies. These patients met AECG criteria by having, in addition to either histopathological or serological criteria, subjective ocular and oral symptoms plus either an abnormal Schirmer's test and/or an abnormal WUSF test (Table 3). They did not meet the ACR criteria because they had only one of the histopathology or serology criteria but did not have an abnormal ocular staining examination.

Table 3.

Summary of subjects classified as Sjögren's Syndrome by only one set of criteria and not the other. The top of the table shows subjects classified under the revised AECG criteria but not under the newly proposed criteria; the bottom shows patients classified as primary Sjögren's Syndrome by the new ACR criteria but not by the modified AECG criteria.

AECG criteria only:
Histology	vB score	OSS	Ro/La	ANA+RF	Schirmer's	WUSF	n=
+	-		-		+	-	8
+	-		-		+	+	10
+	-		-		-	+	8
-	-		+		+	-	0
-	-		+		+	+	4
-	-		+		-	+	5
26	0		9		14	27	35
ACR criteria only:
+		+	-	-			13
+		-	+	-			0
+		-	-	+			2
+		-	+	+			0
-		+	+	-			4
-		+	-	+			5
-		+	+	+			0
15		22	4	7			24

Gray shading = the criterion is not evaluated for the corresponding set of criteria: AECG in the top half of the table; ACR in the bottom half.

Histology=A positive consists of a focus score ≥1 on minor salivary gland biopsy

vB score = van Bijsterveld Score. AECG scoring system for keratoconjunctivitis sicca; applies only for AECG criteria. It is positive ≥ 4[24]

OSS = Ocular Staining Score (lissamine green + fluorescein) for evaluation of keratoconjunctivitis sicca according to the ACR criteria. It is positive ≥ 3[23]

Ro/La = A positive consists of the presence of anti-Ro or anti-La in the serum

ANA+RF= Antinuclear antibodies ≥ 1:360 AND above normal range for laboratory

Schirmer's = Schirmer's I test of tear production with <5 mm in 5 minutes considered abnormal (+). Does not apply for ACR criteria

WUSF = Whole unstimulated salivary flow with <1.5 mls in 15 minutes considered abnormal (+). Does not apply for ACR criteria

Alternatively, there were 24 participants classified as SS by ACR criteria but not AECG criteria (ACR+/AECG- participants; Table 3) and they met criteria mainly due to differences in the scoring of the ocular staining (n=17): the ACR criteria use the OSS[23] which is abnormal at a ≥3 score out of 12 possible points while the AECG criteria utilize the van Bijsterveld score[24] that is abnormal with a score ≥4 out of 9 possible points (Figure 1). Seven of the ACR+/AECG- participants met the ACR criteria but not the AECG criteria by having positive ANA plus RF.

Figure 1.

Comparison of the scoring of keratoconjunctivitis sicca by the van Bijsterveld method, used in the AECG classification criteria, and the OSS (Ocular Staining Score), used in the ACR classification criteria. The van Bijsterveld score evaluates a maximum of 9 points per eye and is considered positive if at least one eye shows a score of ≥4;[24] the OSS scores a maximum of 12 points per eye evaluated and is considered positive if at least one eye shows a score of ≥3.[23] The combination of a lower cutoff level for the OSS and a higher number of possible points makes it significantly less stringent in determining that a patient has keratoconjunctivitis sicca, particularly in true Sjögren's syndrome patients (p=1×10^-6).

The performance of each individual test was assessed in three subsets of participants: 1) classified by ACR criteria, 2) classified by AECG criteria, and 3) classified as having SS by either one or both sets of criteria (supplementary Table S3). As expected, the tests performed consistently across all groups and in summary, the Schirmer's I test had a low sensitivity (range 0.49-0.54) with higher specificity (0.71-0.73) while the WUSF had both low sensitivity (0.59-0.65) and specificity (0.52-0.57). On the other hand, the serology and histopathology performed well, with sensitivities of 0.63-0.64 and 0.84-0.86, respectively, and specificities of 0.94-0.96 and 0.89-0.95 respectively. The most important difference in individual test performance was in the evaluation of keratoconjunctivitis sicca by ocular surface staining. The use of the AECG scoring system by vBS resulted in a sensitivity of 0.57-0.61 with a specificity of 0.70-0.71. The ACR OSS scoring very significantly improved the sensitivity (0.80-0.90) but at the expense of the specificity (0.45-0.51) (supplementary Table S3). We compared the number of participants that had a positive score by one method versus the other and found highly significant differences (Table 4). When assessing this difference in any patient that was classified as having SS (by either or both sets of criteria, n=303), 23% of those having a positive OSS did not have a positive vBS (p<1×10^-6). This difference was 24% (p<1×10^-6) if all participants were included, irrespective of whether they were classified as SS or not. An intermediate result was obtained if the OSS was considered abnormal at a cutoff of ≥4 rather than ≥3: 58 participants (13.5%) went from being OSS (+) to OSS (−), (p=0.001); 53 of these 58 were AECG (−).

Table 4.

Comparison of the performance of the van Bijsterveld and OSS scores for evaluation of keratoconjunctivitis sicca in all participants with complete data (n=646) and all patients classified as having Sjögren's syndrome by either or both sets of criteria.

		van Bijsterveld score (+)1		Δ
All Participants (n=646)		Positive	Negative
OSS2	Positive	277	152	152 (23.5%) p < 1 × 10-6
	Negative	0	217
All SS (n=303)
OSS2	Positive	173	69	69 (22.7%) p < 1 × 10-6
	Negative	0	61

¹van Bijsterveld score = AECG scoring system for keratoconjunctivitissicca (lissamine green + fluorescein) which is positive if ≥ 4 out of 9 possible points[24]

²OSS = Ocular Staining Score (lissamine green + fluorescein) for evaluation of keratoconjunctivitis sicca according to the ACR criteria. It is positive if ≥ 3 out of 12 possible points[23]

When evaluating gene expression profiles for participants meeting only one set of criteria, those that met both sets of criteria, and healthy controls, we found that the participants that met criteria for SS by one or more sets of criteria tended to cluster together and were distinct from controls (Figure 2). Furthermore, using low stringency criteria designed to maximize determination of differences, we found no gene expression difference between the participants meeting both sets of criteria versus those meeting only one set of criteria.

Figure 2.

Assessment of the AECG and ACR criteria using whole-blood gene expression profiling in SS. Heat maps are displayed using fold changes for differentially expressed transcript; overexpressed transcripts (FC>0) are bright yellow, while underexpressed transcripts (FC<0) are light blue. Data is displayed in rows and columns, with rows defined by transcripts and columns defined by individual samples. Colors above each column header denote healthy controls (blue; n=73) and cases (red, yellow, or green). For cases, the represents the criteria used to define SS: red denotes cases meeting only ACR criteria (n=4); yellow denotes cases meeting only AECG criteria (n=25); and green denotes cases meeting both ACR and AECG criteria (n=127). A Sjögren's-specific set of differentially expressed (DE) transcripts was defined by comparing all SS cases regardless of classification criteria to healthy controls. Hierarchical clustering was performed with respect to transcripts and samples, and dendrograms generated to visualize sample clustering. In Panel A, all samples are displayed, with cases and controls generally segregating. In Panel B, hierarchical clustering was performed using the DE transcripts defined in Panel A, but removing cases meeting both AECG and ACR criteria. Panel C further limits the clustering to those individuals meeting only ACR or AECG. Distinct clustering of patients meeting only ACR or AECG criteria was not observed, suggesting molecular similarity between cases defined by either ACR or AECG criteria.

Discussion

The pathophysiological mechanisms underlying SS are still poorly understood and accurately determining who does and does not have SS is difficult.[28] In the clinical setting, the diagnosis of SS relies on interpreting and integrating all aspects of the patient's history, test results and the expert opinion of the clinician. For research purposes, many classification systems have been proposed in the last few decades, and the co-existence of more than one system may lead to heterogeneity and confusion in the interpretation of research studies. As has recently been highlighted by Vitali et al.[20] the SS community should be striving for the common goal of reaching a final agreement on classification criteria for the disease. The first steps in this direction are to evaluate the performance of the new criteria and compare them to the currently used AECG criteria in external cohorts of patients and controls. Such comparison in our uniformly evaluated cohort of subjects presenting with sicca helps serve this purpose.

This cohort has been evaluated in a homogeneous and standardized manner at two research clinics in Oklahoma and Minnesota by a multidisciplinary team of experts. The evaluation includes all the exams and laboratory procedures detailed in the AECG and ACR classification criteria, including the subjective components of the AECG criteria, and both ocular staining scoring systems. [19, 21] We did not assess the revised Japanese Ministry of Health criteria, [12, 29] because they are intended as an aid for clinical diagnosis and not for research classification, which is the aim of our SS clinics. In addition, they have not been tested in non-Japanese population and include additional invasive procedures which we felt were not justified for our participants.

It is relevant to note that there are differences in the enrollment strategy of our cohort in comparison to the SICCA cohort. The most important difference is that while in the SICCA cohort only 79% of the participants had both subjective dry eyes and dry mouth, [22] the totality of our participants responded affirmatively to at least one ocular and one oral dryness question of the AECG criteria as evidence of symptoms of oral and ocular dryness.

The two clinics have so far evaluated 837 individuals, but only 646 for whom we had all the data points for both sets of criteria were included in the current analysis. Of these, 303 participants were classified as having SS by either one or both sets of criteria but almost 20% of the participants met only one set of criteria for SS, and not the other. This level of disagreement between the two classification systems is similar to that reported by Shiboski et al.[21] These patients would have been excluded from any study based on only one of the classification methods; thus, knowing their characteristics becomes relevant to future research.

In the case of patients classified as having SS by the AECG criteria only (n=35), two thirds of them had a minor salivary gland lip biopsy consistent with SS and the remaining one third had positive Ro/La serology. They did not meet ACR criteria because they did not have keratoconjunctivitis sicca and their objective measures of dryness were confined to either the biopsy or the serology but not both (Table 3). While we have not used formal expert consensus methodology, we believe that most experts would agree that SS is present in a person presenting with subjective dry eyes, subjective dry mouth, a confirmatory minor salivary gland biopsy or positive Ro/La serology plus at least one additional objective measure of dryness, be it a positive Schirmer's I test or an abnormal WUSF test.

Conversely, the subjects that met only the ACR criteria (n=24) did so because of alternative positive serology status or differences in the evaluation of keratoconjunctivitis sicca. Seven of them (29%) met criteria by having positive ANA/RF but negative anti-Ro/La as one of two criteria. Again, we have not done formal expert testing, but it is unlikely that these subjects would be considered to have SS based on expert opinion, especially without information about sicca symptoms. The remaining 71% met criteria for keratoconjunctivitis sicca by ACR but not by AECG criteria.

It has been proposed that it would be useful to know if the OSS developed for the ACR criteria can be substituted by the AECG vBS.[21] Few already established cohorts that we are aware of, if any, are currently able to directly compare the performance of the vBS with the OSS. In cohorts that were evaluated before the publication of the OSS in 2010, determining the OSS would require access to the breakdown of the scoring of each eye: individual scores for medial and lateral bulbar conjunctiva and cornea plus the description of patches of confluent staining, staining in the papillary area and presence of filaments (Figure 1). The vBS does not take into consideration these last three features,[24] which add 3 possible points to the score of each eye in the case of the OSS. The vBS is considered abnormal with a score of ≥4 out of 9 possible points[24] while the OSS is positive at ≥3 out of 12 points.[23]

We are in the unique position of having recorded separately each of the 12 possible scoring points for each eye in all our cohort participants. Thus, we were able to determine both their vBS and OSS scores and compare the performance of each system. To reduce inter-observer and intra-observer variability inherent to the van Bijsterveld scoring system,[24] the vital dye score for each section of the ocular surface was determined using the SICCA dot counting method. While there are no studies validating the conversion of this scoring method with the traditional van Bijsterveld technique, the two are similar; we felt that an objective scoring method would be more meaningful and reproducible in the context of multiple observers. As expected, participants were more likely to have an abnormal OSS score than vBS, resulting in ∼25% of subjects having a positive OSS but negative vBS. This difference was highly significant both in all cohort participants and in patients that were classified as having SS by one or both sets of criteria (p<1 × 10^-6). The OSS is superior in including true positive cases but has a poor performance ruling out those that do not have SS (i.e. it is very sensitive but has poor specificity); the opposite is the case for the vBS. It is pertinent to note that only a minor proportion of cases of keratoconjunctivitis sicca are due to SS.[30] It remains to be seen how other prospective cohorts evaluate these two scoring systems vis à vis, in order to determine what the optimal threshold should be. It is noteworthy that one of the main goals of the development of new classification criteria by the SICCA consortium was to come up with a system that has high specificity to avoid exposing unaffected subjects to the potentially serious adverse effects of novel investigational therapies.[21]

The two tests that performed the best across all comparison groups were the minor salivary gland biopsy and anti-Ro/La serology, which performed similarly to reports in previous studies.[21, 31] The Schirmer's and WUSF tests while less useful in distinguishing true SS patients from participants with non-Sjögren's sicca syndrome, are easy to perform and non-invasive. It has recently been suggested that more emphasis should be given to tests that in addition to identifying true cases and excluding unaffected subjects, can be done at early stages, multiple times, and with minimal distress to the participant.[32] In the future, we may see salivary gland ultrasonography playing this role in SS.[33] But with the current sets of criteria, there is an interesting difference in terms of accessibility; while the ACR criteria require evaluation by a practitioner specializing in eyes and by a practitioner who can perform a lip biopsy, both the Schirmer's and the WUSF tests can be performed in a standard medical office without the need for sophisticated equipment or medical specialists. Thus, subjects can be assessed for subjective dry eyes and dry mouth, the presence of autoantibodies along with Schirmer's and WUSF testing by a rheumatologist. If AECG criteria are not met with such an assessment, then biopsy and eye examination can be pursued. In some clinical care settings or research situations that do not include exposing the selected participants to the risk of significant adverse events (such as some therapeutic trials), a stepwise approach such as this may be useful and cost-effective.

The comparison of the AECG criteria with the proposed ACR classification demonstrates that neither system is clearly superior to the other when classifying a patient with SS; a finding already reported in the initial publication of the ACR criteria.[21] The lack of highly sensitive, specific, and reproducible criteria may in part be due to our current limited understanding of SS physiopathology; such knowledge would provide the most rational basis for disease classification. In the current setting, the ACR criteria may be best suited for stricter studies focused on high specificity to reduce the risk of drug-related toxicity, while the AECG criteria may be applicable to broader use, particularly in less risky medical research, or in non-treatment clinical or translational research settings. Moreover our findings of similar gene expression profiles across all possible subjects affected by Sjögren's syndrome, which is different from what is observed in healthy controls, supports our notion that modifying classification using only clinical criteria is not likely to lead to consequential improvements in our ability to identify patients with SS. We believe that such improvements in diagnostic acumen will require a more fundamental understanding of the pathogenic mechanisms than is at present available.