Development of New International Antiphospholipid Syndrome Classification Criteria Phase I/II Report: Generation and Reduction of Candidate Criteria

Abstract

Objectives:

An international multi-disciplinary initiative, jointly supported by American College of Rheumatology (ACR) and European League Against Rheumatism (EULAR), is underway to develop new rigorous classification criteria to identify patients with high likelihood of Antiphospholipid Syndrome (APS) for research purposes. We applied an evidence- and consensus-based approach to identify candidate criteria and develop a hierarchical organization of criteria within domains.

Methods:

During Phase I, the APS classification criteria Steering Committee used systematic literature reviews and surveys of international APS physician scientists to generate a comprehensive list of items related to APS. In Phase II, we reviewed the literature, administered surveys, formed domain subcommittees, and used Delphi exercises and nominal group technique to reduce potential APS candidate criteria. Candidate criteria were hierarchically organized into clinical and laboratory domains.

Results:

Phase I generated 152 candidate criteria, expanded to 261 items with the addition of subgroups and candidate criteria with potential negative weights. Using iterative item reduction techniques in Phase II, we initially reduced these items to 64 potential candidate criteria organized into ten clinical and laboratory domains. Subsequent item reduction methods resulted in 27 candidate criteria, hierarchically organized into six additive domains (laboratory, macrovascular, microvascular, obstetric, cardiac, and hematologic) for APS classification.

Conclusion:

Using data- and consensus-driven methodology, we identified twenty-seven APS candidate criteria in six clinical or laboratory domains. In the next phase, the proposed candidate criteria will be used for real-world case collection and further refined, organized, and weighted to determine an aggregate score and threshold for APS classification.

INTRODUCTION

Antiphospholipid Syndrome (APS) is a systemic autoimmune disease characterized by thrombosis and/or pregnancy morbidity in patients with persistent antiphospholipid antibodies (aPL). Classification of antiphospholipid syndrome (APS) for clinical trials and studies is currently based on clinical and laboratory criteria described in the “Sapporo Classification Criteria” published in 1999 (1), validated in 2000 (2), and revised in 2006 (3), now known as the Revised Sapporo APS Classification Criteria (or Sydney Criteria).

The original Sapporo APS classification criteria included clinical (vascular thrombosis or pregnancy morbidity) and laboratory items (persistent lupus anticoagulant [LA] test and/or anticardiolipin antibody [aCL] IgG/M positivity with at least two tests performed at least six weeks apart)(1). In the revised criteria (3), modifications were made including: anti-β₂-glycoprotein-I antibodies (aβ₂GPI) IgG/M as a new laboratory test, wider time interval between serologic testing (at least 12 weeks instead of six weeks), clarification of a time interval between serology and clinical manifestations (maximum of five years), and specification of laboratory assay titer threshold and “non-criteria” aPL-manifestation definitions. Over the past decade substantial evidence has accumulated describing additional clinical and laboratory manifestations associated with aPL (4–7). Additionally, new methodologically rigorous and data-driven approaches to address biases and develop a robust set of classification criteria have been published (8–12).

In 2016, the 15th International Congress on aPL Task Force on APS Classification conducted a needs assessment survey of international physicians with APS expertise. Respondents indicated the need for the following components in a new APS classification criteria system, including: a) capturing the full spectrum of clinical and laboratory manifestations of disease; b) distinguishing APS from other co-morbidities; c) weighting certain clinical factors more than others; and d) including a strong evidence basis for definitions of aPL positivity or pregnancy morbidity.

Thus, an international effort was initiated with the overall goal of developing a new APS classification system, which is jointly supported by the American College of Rheumatology (ACR) and EULAR (European League Against Rheumatism). The objective was to develop a new APS Classification Criteria, which will demonstrate excellent face, discriminant, and construct validity. We aimed to incorporate the use of bias reduction strategies to arrive at a system with the highest sensitivity, specificity, and positive predictive value for the likelihood of APS, against the gold standard of expert consensus, while retaining face validity for classification. Here we summarize the overall methodology and results from the first two phases of our four-phase APS classification criteria development initiative.

METHODS

The APS classification criteria development involves a four-phase methodology, as previously used in other rheumatic disease classification systems. Details of the methodology overview as well as the APS steering committee development, and experts and stakeholders involved in Phase I/II are provided in the Supplement.

Phase I Item Generation

During Phase I, the APS Classification Criteria Steering Committee used surveys and literature review to generate all potential candidate criteria associated with APS. First, we asked 54 collaborators in an e-mailed survey to list all features that, in their experience, occur as part of aPL/APS laboratory and clinical spectrum. The purpose of this question was to help identify potential candidate criteria with “positive weight”. Respondents were encouraged to consider their real-life experience with aPL-positive patients, rather than focusing only on the current APS Classification Criteria. Responses were systematically clustered by organ system to avoid duplication and improve interpretability. Second, we asked participants to provide potential criteria that, if present, would lead a physician to question the diagnosis of APS, i.e., candidate criteria with “negative weight”, whose presence points away from APS but does not rule it out. These potential negatively weighted items were also organized into separate categories. Third, we asked whether physicians consider APS patients in different subpopulations (Figure 1, Table 1). Three of the Core Planning Group members (MB, SZ, DE) additionally compiled and screened an article reference library (n:26) including mostly recent APS-related systematic reviews and meta-analyses (Supplement) to include additional items related to aPL/APS.

Figure 1.

Phase I/II Methodology of New Antiphospholipid Syndrome Classification Criteria Development

Table 1.

Phase I/II Item Generation and Item Reduction Surveys During New Antiphospholipid Syndrome Classification Criteria Development

	Survey Question and Response Option	Goal	Response Rate	Respondents	Results Summarized	Final Number of Candidate Criteria
Phase I
Question #1	Question: ‘Describe all features (historical, clinical, laboratory, radiological, and pathological) that in your experience can occur as part of aPL/APS spectrum” Response Option: Open-Ended	Identify potential candidate criteria with “positive weight”	41/54 (76%)	Of 41, 18 were rheumatologists, 5 hematologists, 5 clinical immunologists, 5 nephrologists/cardiologists/neurologists, 4 internists, 2 pediatric rheumatologists, and 2 obstetricians.	Distribution of Potential aPL/APS Spectrum Candidate Criteria by system (#):  • Neurologic (25)  • Laboratory (aPL) (23)  • Obstetric (16)  • Dermatologic (15)  • Renal (12)  • Vascular (10)  • Cardiac (9)  • Laboratory (non-aPL) (9)  • Other (7)  • Hematology (5)  • Pulmonary (5)  • Gastrointestinal (4)  • Musculoskeletal (4)  • Endocrinologic (3)  • Ophthalmology (2)  • Auditory (2)  • Family History (1)	• 152 items (based on question 1) expanded to 261 to include negatively weighted criteria and subgroups  • Of note, some items were reported as both positive and negative candidate criteria; we included those items in the expanded list.
Question #2	Question: ‘Describe all features (historical, clinical, laboratory, radiological, and pathological) or concomitant diseases that, if present, would make you question the diagnosis of APS even if aPL tests are positive’ Response Option: Open-Ended	Identify potential candidate criteria with “negative weight”			Distribution of Potential Negative Weight Criteria (categories):  • Laboratory (aPL vs. non-aPL)  • Other Thrombosis Risk Factors  • Infections  • Traditional CVD Risk Factors  • Autoimmune Diseases  • Malignancy  • Neurological  • Histological  • Elderly  • Thrombotic Microangiopathies  • Radiological  • Medications  • Lupus Manifestations  • Strong Family History  • Bleeding  • Other
Question #3	Question: ‘When you consider the diagnosis of APS, do you think of APS patients in different subpopulations? If so, please describe how you categorize these patients.” Response Option: Open-Ended	Categorize candidate criteria			Subpopulations of APS patients categorized by:  • Age  • Clinical Manifestations  • aPL Profile  • Risk Level
Phase II
Survey A	Question: “Consider two patients who are exactly the same except that one has the clinical feature presented and the other does not. Please rate each feature in terms of how strong this feature is in differentiating APS from other similar conditions, i.e., specific for APS” using a Likert scale (−5 to +5). Response Option: Please rate each item based on a scale of −5 to +5, with +5 being “extremely specific" for APS.	Item Reduction by elimination of low specificity items and organizing higher specificity items into separate domains	43/61 (71%)	• Of 43, 22 were rheumatologists, 4 hematologists, 4 nephrologists/cardiologists/neurologists/vascular specialists, 4 internists, 3 clinical immunologists, 3 pediatric rheumatologists, 2 pediatric hematologists, and 1 obstetrician.	• See Table 2	132 items (reduced to 64 items and 10 domains when overlapping items were eliminated)
Survey B	“Consider two patients who are exactly the same except that one has the clinical feature presented and the other does not. Please rate each feature in terms of how strong this feature is in differentiating APS from other similar conditions, i.e., specific for APS” using a Likert scale (−5 to +5). Response Option: Please rate each item based on a scale of −5 to +5, with +5 being “extremely specific" for APS.	Further item reduction to goal of approximately 30 candidate criteria	19/19 (100%)	Of 19, 8 were rheumatologists (adult), 2 hematologist, 2 cardiologists/vascular medicine specialists, 2 immunologists, 2 obstetrician, 1 pediatric rheumatologist, 1 neurologist, and 1 classification criteria methodologist	• See Table 3	• 27 items and 6 domains

We analyzed the comprehensive list of potential aPL-associated manifestations as follows: first, we clustered the generated candidate criteria into meaningful categories by organ system; second, each potential criterion included in the list was checked for reliability and precision in its measurement; and third, similar or highly correlated criteria were eliminated to avoid redundant data.

We then used the following strategies to further expand the list of potential criteria: a) for items that were submitted with subcategories during the phase I survey (e.g., age below or above 55 years old; stroke with or without hypertension; aCL titer below or above 40U), we expanded the list to consider all potential items within subgroups individually; and b) the steering committee systematically reviewed the results of the additional Phase I survey designed to identify candidate criteria with potential negative weight and aPL/APS subgroups.

Phase II Item Reduction

Item reduction was guided by the following principles: a) the criteria remaining after phase II should demonstrate good face, construct, and discriminant validity; b) items with low sensitivity or specificity, poor reliability, redundancy, or insufficient feasibility should be removed; and c) items should be organized into separate domains. This phase was also an iterative process, guided by literature reviews, surveys, Delphi exercises, Steering Committee communications (e-mail/teleconference/in-person meetings), and the development of “Domain Subcommittees”.

Literature Reviews:

To aid in their decision-making, the Core Planning Group provided Phase II survey respondents and Steering Committee members with the article reference library (n:26) (Supplement). Additionally, five different teams (under the guidance of SZ and DE) worked on meta-analyses of selected other manifestations of aPL, i.e., livedo, thrombocytopenia, hemolytic anemia, aPL-nephropathy, and cardiac valve disease. These meta-analyses, together with others, helped guide the Steering Committee and Collaborators during the new classification criteria development efforts. To better clarify arterial and venous thrombosis risk factors, the Core Planning Group evaluated major recent international thrombosis and cardiovascular disease (CVD) guidelines, as well as conferring with North American and European cardiologists. In addition, our pathology domain subcommittee (renal pathologist [SS], rheumatologists [MB, DE, MT]), and vascular medicine specialist (SZ) in collaboration with a nephrologist have been leading an effort to better characterize the definition of aPL-associated nephropathy, a poorly defined but specific and clinically relevant pathologic manifestation related to APS.

Item Reduction Surveys A/B and Delphi Exercises:

We administered two surveys for item reduction in Phase II (A and B), summarized in Table 1. In survey A, 61 Collaborators and selected Steering Committee members were emailed a survey to assess each item generated in Phase I based on the specificity of each feature in differentiating APS from other similar conditions, using a Likert scale (−5 to +5), with (−5) extremely strongly against APS, i.e., more likely related to another disease entity; (0) not for or against APS; and (+5) extremely specific for APS. Mean survey scores for each item (±SD) were calculated. Items were ranked from highest to lowest mean score. Low specificity items (Likert score <1) were eliminated when also agreed upon by the Steering Committee. In Survey B, the Steering Committee members (n: 20) were asked to score previously identified items from Survey A, using a Likert scale (−5 to +5), based on how specifically each item differentiated APS from other similar conditions. Items were then re-ranked by their mean (±SD) survey scores and again low specificity items (Likert score <2 were eliminated after discussion and agreement by the Steering Committee. The threshold for elimination of low specificity items by Likert score was intentionally slightly higher for survey B (score <2) compared to survey A (score <1) to improve specificity.

Domain Identification, Organization, and Assessment of Face and Content Validity:

After completion of each survey (A and B), Core Planning Group and Steering Committee members reviewed survey responses, survey scores, and literature reviews; nominal group technique (via in-person, email, and teleconference meetings) was used to organize and group highest scoring items based into separate “domains”. We created domains under the guidance of classification criteria methodology experts (RN and KC), using basic principles for domain identification, consistent with previous classification criteria (13), suggesting the need for: a) no more than 8–10 domains; b) separate domains, allowing for an additive classification criteria system; c) hierarchy of candidate criteria within each domain based on item specificity; d) the highest specificity item within each domain to be scored; and e) scores from each domain to be summed (i.e. additive domains) to produce a final APS classification criteria score.

In-person Meeting preceding the 2018 ACR Annual Congress:

Proposed domains and candidate criteria within domains were presented and further refined by the Steering Committee before the ACR annual congress (October 20, 2018, Chicago, IL, USA), which also included patient participation. Changes to the number and content of domains, and criteria within domains, were made in response to group consensus. Literature was also presented to discuss sensitivity, prevalence, and specificity of items. Challenges with drawing conclusions related to APS from the general population literature were also discussed, as were the relevance of definitions used in past classification criteria. To more precisely define, cluster and organize items, the Steering Committee also recommended forming “Domain Subcommittees” to review the literature and come to consensus on the definition, reliability, and precision of each potential criterion in its measurement. Additionally: a) the Core Planning Group proposed classification “entry criteria”, i.e., minimum criteria required for consideration of APS classification, to the Steering Committee; and b) the concept of risk stratifying venous and arterial thrombosis in aPL-positive patients in the context of venous thromboembolism and CVD risk factors was discussed to increase APS specificity, based on Phase I and II survey results.

Other Important Considerations Relevant to Phase I/II

Following this Steering Committee meeting, we administered a separate survey asking the Steering Committee to assess and rate proposed entry criteria and risk factors for thrombosis. Additional questions were included regarding the definitions of aPL “persistence”, aPL “positivity”, whether LA test in the setting of anticoagulation should be included in the classification criteria scoring, which aPL laboratory tests to include in the entry criteria, and whether a separate classification system should exist for primary APS versus APS-associated with other systemic autoimmune diseases.

RESULTS

Phase I Item Generation

Based on our phase I item generation survey and literature review, we generated a comprehensive list of potential aPL-associated manifestations, in particular those occurring at the time of APS diagnosis (Table 1). Survey responses were originally clustered based on organ systems (n: 152) (supplement Table 1), and then expanded to incorporate negatively weighted responses and aPL/APS subgroups (n: 261) (supplement Table 2). We checked each potential criterion for reliability, redundancy, and precision in its measurement. For example, thrombotic stroke was defined as a focal clinical neurologic event confirmed by neuroimaging studies such as computed tomography or magnetic resonance imaging (14). Additionally, similar criteria (e.g., deep vein thrombosis (DVT) and thromboembolism), or highly correlated criteria (e.g., DVT and pulmonary embolism) were merged to avoid redundant data. The reported subpopulations of APS patients, grouped based on age, clinical manifestations, aPL profile, and risk level are shown in Table 1.

The Core Planning Group subsequently restructured Phase I results. 152 candidate criteria, identified during Phase I, were expanded to 261 with the addition of subgroups and candidate criteria with potential negative weight (Figure 1). The Phase I literature screening of the article reference library did not yield additional candidate criteria that were not already included in the comprehensive list generated by the surveys. Potential criteria with subgroups (e.g., age less than 55 years old; stroke without hypertension; or aCL titer above 40U), were separated into individual criterion. We also added potential negatively weighted criteria to the list of 152 candidate criteria, including but not limited to infections (e.g., HIV, Hepatitis B or C, or syphilis), other thrombosis risk factors (i.e. surgery, immobilization, or infective endocarditis, and items that overlapped with positive criteria (e.g., false positive venereal disease research laboratory [VDRL] test for syphilis).

Phase II Item Reduction

Literature Reviews and Additional Studies:

1. Thrombosis Risk Factors. Based on Phase I survey results and Steering committee consensus to consider arterial and venous events in the presence or absence of thrombosis risk factors, we reviewed major recent international guidelines that emphasized the need to incorporate cardiovascular risk factors (e.g. hypertension, hypercholesterolemia) for arterial thrombosis events (15–20) and venous thromboembolism risk factors (e.g. oral contraceptive use) for venous events (21–24). The committee agreed that these risk factors will be further defined and structured during Phase III.

2. Meta-analyses. Meta-analyses performed in systemic lupus erythematosus (SLE) patients with and without aPL demonstrated the increased odds ratios (OR) of various clinical manifestations, including cardiac valvular disease (OR 3.13 [95% CI 2.31–4.24]), livedo (OR 3.36 [95% CI 2.49–4.55]), thrombocytopenia (OR 2.48 [95% CI 2.10–2.98]), hemolytic anemia (OR 3.22 [95% CI 2.40–3.42]), and renal impairment (OR 2.9 [95% CI 1.9–4.3]) in aPL -positive SLE patients compared to aPL-negative SLE patients (25–27). These studies provided additional evidence that the presence of aPL may increase the likelihood of the above clinical manifestations.

3. Additional Studies. The pathology domain subcommittee evaluated real-world renal biopsies to better characterize ‘aPL nephropathy’. The committee concluded that biopsy reports inconsistently use aPL-nephropathy related terminology and the use of aPL-nephropathy related terms varies among pathologists while reporting the biopsy findings of kidney involvement in aPL-positive patients (data to be published separately).

Item Reduction Surveys A/B and Delphi Exercises:

Phase II survey A mean Likert scale scores for item specificity ranged from −2.29 to 4.65; items of low specificity (score less than 1) were eliminated. The higher specificity items (n=132) were organized into seven clinical (cardiopulmonary, dermatologic, hematologic, neurologic, obstetric, renal/abdominal, and vascular), two laboratory (aPL immunoassays and lupus anticoagulant test), and one pathologic domain (Tables 1 and 2). Items overlapping or describing similar concepts were combined to arrive at n=64. Phase II survey B mean item scores ranged from −1.05 to 4.79; items of low specificity (score less than 2) were eliminated (Tables 1 and 2). Each item was considered by the Steering Committee, and in certain situations the committee agreed to retain or eliminate criteria regardless of their survey score for further evaluation in Phase III, as described in Table 2. Both survey A and B results demonstrated very different weights for macrovascular domain outcomes with or without additional thrombosis risk factors.

Table 2.

Phase II Survey A Results and Subsequent Steering Committee and Domain Subcommittee Decisions/Rationale for Item Reduction During New Antiphospholipid Syndrome Classification Criteria Development

Candidate Criteria and Domains	Committee Consensus Supported by the Literature Review for Items Eliminated (Despite Score >2) or Retained (Despite Score <2)
I: Cardiopulmonary • Retained1: Cardiac Valve (CV) Vegetation, CV Thickening, Alveolar Hemorrhage (AH)*  -------------------------------------------------------------------------------------------------- • Eliminated2: Pulmonary HTN (thromboembolic [TE])*, Heart Failure Due to Ischemia	AH: Considered distinct entity associated with aPL(37);retained for Phase 3 ‘microvascular’ domain. ----------------------------------------------------------------------------- Pulm HTN (TE): Commonly due to pulmonary embolism in APS which is redundant with the item VTE (chronic TE pulmonary hypertension[Group 4])(38, 39).
II: Dermatologic • Retained1: Livedo Racemosa (LRa), Livedoid Vasculopathy  -------------------------------------------------------------------------------------------------- • Eliminated2: Livedo Reticularis (LRe)*, Atrophie Blanche de Milan, Pyoderma Gangrenosum-like Skin Ulcers, Anetoderma	---------------------------------------------------------------------------- LRe: Relatively common in general population; not as specific as LRa for APS classification(40, 41).
III: Hematologic: • Retained1: Thrombocytopenia (mild), Thrombocytopenia (severe)*  -------------------------------------------------------------------------------------------------- • Eliminated2: Microangiopathic Hemolytic Anemia, Coombs+ without Hemolytic Anemia	Thrombocytopenia (severe): Given that other more likely explanations than aPL exist, committee proposed evaluating “cut-off” thresholds (mild-moderate-severe) in Phase 3 to improve specificity(42, 43). ----------------------------------------------------------------------------
IV: Neurology: • Retained1: Transient Ischemic Attack (TIA) without additional risk factor (ARF) (age <55), Transient Ischemic Attack (TIA) without additional risk factor (ARF) (age >55)*, Transient Ischemic Attack (TIA) with additional risk factor (ARF) (age <55)*, Transient Ischemic Attack (TIA) with additional risk factor (ARF) (age >55)*, Acute Ischemic Encephalopathy (age<55y)  -------------------------------------------------------------------------------------------------- • Eliminated2: Seizure/Epilepsy*, Atypical Multiple Sclerosis-like Disease, Migraine Responsive to Anticoagulation, Chorea, Cerebral White Matter Lesions, Cognitive Dysfunction, Longitudinal Extensive Transverse Myelitis, Multi-Infarct Dementia (<65y)	TIA: Retained for Phase 3 independent of age and ARF, with plan to further analyze in context of age and ARF during case collection. ----------------------------------------------------------------------------- Seizure/Epilepsy: Rarely associated with aPL; mainly secondary to stroke in aPL positive patients; poor definability(5, 44).
V: Obstetric: • Retained1: Fetal Loss (Single*/Recurrent/Consecutive), Stillbirth, Early Pregnancy Loss (Recurrent*), Pre-eclampsia (early/severe/mild/late*), HELLP Syndrome*, Eclampsia, Intrauterine Growth Restriction*  --------------------------------------------------------------------------------------------------- • Eliminated2: Chorea Gravidarum, Pregnancy-induced Hypertension	Obstetric Morbidity: Re-structured domain for evaluation of individual items during Phase 3 case collection. (45, 46). -----------------------------------------------------------------------------
VI: Vascular • Retained1: Arterial Thrombosis (AT) without ARF (age <55), AT with ARF (age< 55)*, AT without ARF (age >55)*, AT with ARF (age >55)*, Venous Thromboembolism (VTE) without ARF (age <55y), VTE with ARF (age < 55y)*, VTE without ARF (Age > 55)*, VTE with ARF (age>55)*, Superficial Venous Thrombosis (SVT) without ARF (age <55y), SVT with ARF (age < 55y)*, SVT without ARF (age > 55)*, SVT with ARF (age>55)*	AT/VTE/SVT: SVT considered to have lower specificity for aPL than AT or VTE(5, 47). However, all three retained for Phase 3 independent of age and risk factors; committee agreed to include age thresholds and additional thrombosis risk factors during case collection.
VII: Renal/Abdominal: • Retained1: Nephrotic Syndrome, Adrenal hemorrhage
VIII: Pathology: • Retained1: Thrombosis/infarction without Vasculitis, Pulmonary Capillaritis (PC)*, Fibrous Interstitial Hyperplasia (FIH)*, Focal Cortical Atrophy (FCA)	PC/FIH: PC increasingly associated with aPL(37). FIH considered the most frequent chronic lesion in primary aPL nephropathy(48). Retained and restructured both items for Phase 3 data collection under ‘microvascular’ domain.
IX. Laboratory Part I • Retained: Lupus Anticoagulant Test
X. Laboratory Part II • Retained1: Anticardiolipin Antibodies (aCL) IgG/IgM*, Anti-β2Glycoprotein-I Antibodies (aβ2GPI) IgG/IgM*  ------------------------------------------------------------------------------------------------- • Eliminated2: Anti-Domain-I Antibodies (aDI)*, Anti-Phosphatidylserine-Prothrombin Antibodies (aPS/PT)*, aCL IgA, aβ2GPI IgA, Anti-Prothrombin antibodies	aCL & aβ2GPI IgM: Wide variation in survey scores for aPL ELISA IgM based on titer level and single/persistent positivity. aPL ELISA IgM noted to have lower specificity than IgG for APS. Final decision was to collect detailed aPL ELISA isotype and titer information during phase 3 (49, 50). ---------------------------------------------------------------------------- aDI & aPS/PT: limited commercial availability; additional research needed to define feasibility, clinical correlation, and standardization(4).

¹“Reason for retention” was based on committee agreement, supported by survey Likert score ≥2. Any item scoring <2 but retained based on committee discussions is marked with an asterisk (*), and the reason is explained in the notes section.

²“Reason for elimination” was based on committee agreement, supported by survey Likert score <2 (low specificity). Any item scoring ≥2 but eliminated based on committee discussions is marked with an asterisk (*) explained in the notes section

Domain Identification, Organization, and Assessment of Face and Content Validity:

After Phase II survey A and B completion, based on subsequent literature reviews, nominal group technique, Steering Committee and Domain Subcommittee discussions, and one in-person face-to-face Steering Committee meeting (see below), 64 items identified in Phase II Survey A were reduced to 27 candidate criteria (Table 3). Through literature review and Domain Subcommittee expert-based discussions, face and content validities of each candidate criterion were assessed and determined. Specificity of a candidate criterion for APS was determined to be contingent on assessment in a non-APS population, using literature review, when available, and/or expert opinion. Consensus was achieved among the Steering Committee for the following: a) only score a particular criterion if no other more likely cause exists; b) clinical criteria can occur on one occasion and need not to occur simultaneously with other clinical criteria; and c) only the highest weighted criterion in each domain would be counted towards the total APS score. The Core Planning Group and Steering Committee members adhered to the recommendations outlined by the classification criteria methodology experts and hierarchically organized the highest scoring candidate criteria into six separate and additive domains (Table 3).

Table 3.

Phase II Survey B Results with Proposed Domains and Items During New Antiphospholipid Syndrome Classification Criteria Development

Proposed Domains (n:6)	Proposed Items (n:27)
Candidate Laboratory Criteria
1A: Antiphospholipid Antibody Testing-Coagulation-based Functional Assays	• Lupus Anticoagulant Test
1B:Antiphospholipid Antibody Testing-Solid Phase Assays	• Anticardiolipin Antibody IgG, Anticardiolipin Antibody IgM • Anti-β2-glycoprotein-I IgG, Anti-β2-glycoprotein-I IgM
Candidate Clinical Criteria
2: Macrovascular	• Superficial Vein Thrombosis, Venous Thromboembolism, Arterial Thrombosis, Transient Ischemic Attack
3: Microvascular	• Livedo Racemosa, Livedoid Vasculopathy, Adrenal Hemorrhage or Plexus Thrombosis, Acute Ischemic Encephalopathy, Cardiac Microvascular Disease, Pulmonary Hemorrhage, Acute aPL Nephropathy, Chronic aPL Nephropathy
4: Obstetric	• Pregnancy Loss <10 weeks (w) of Gestation, Fetal Death Between 10w to <16w of Gestation, Fetal Death Between 16w to 34w of Gestation, Pre-eclampsia with Severe Features <34w of Gestation, Placental Insufficiency with Severe Features <34w of Gestation
5: Cardiac Valve Disease	• Non-infectious Valve Vegetation, Thickening
6: Hematologic	• Platelet count <20 × 109 per liter, Platelet count 20 – 130 × 109 per liter, Platelet count 131–150 × 109 per liter

In-person Meeting prior to the 2018 ACR Annual Congress:

During this face-to-face meeting, the Steering Committee agreed that candidate clinical criteria must be interpreted in the context of a “clinically acceptable” aPL profile. Thus, all members voted in favor of proposed permissive, rather than restrictive, entry criteria requiring: fulfillment of at least one laboratory criterion (aPL positivity [LA test, aCL IgG/M, or aβ₂GPI IgG/M positivity above the normal laboratory range] at any time) and one clinical criterion (identified from the clinical domains). The group also discussed and eliminated items with low specificity (i.e., Phase II survey A Likert score <1) or poor definability; items overlapping or describing similar concepts were combined (Table 2). The committee also agreed with the need for further evaluation of: a) the definition of aPL laboratory ‘persistence’ and titer level cut-offs; b) use of an age cut-off; c) and Phase I and Phase II survey A results suggesting that thrombotic events with other risk factors should be weighted differently from those without risk factors. The Steering committee agreed that there are not enough data supporting the inclusion of the IgA isotype in the classification criteria until there is better understanding of the pathogenic and prognostic significance. This decision was based on: a) our Phase II survey B results demonstrating low scores for aCL or aβ2GPI IgA regardless of titer or persistence (Likert score <1); b) multiple studies demonstrating the lack of predictive value of isolated IgA positivity for APS manifestations including a study led by members of our laboratory domain subcommittee team (28, 29); and c) the lack of feasibility and standardization (30–32). During and after the pre-ACR 2018 in-person meeting, the proposed candidate criteria within domains were further refined by Domain Subcommittees.

Other Important Considerations Relevant to Phase I/II

Based on Phase II survey C results and additional Steering Committee discussions, the following consensus decisions were also made: a) additional thrombosis risk factors with low specificity were eliminated; (72–89% agreed with proposed thrombosis and CVD risk factors and definitions); b) “entry criteria” would be used to identify the relevant patient population to whom the classification criteria would be applied; c) 84% voted in favor of introducing a time restriction between the a positive aPL test and aPL-related clinical event in the entry criteria; d) 89% agreed that persistence of laboratory testing should be defined as at least “12 weeks apart”; e) 79% agreed that timing of aPL testing with respect to known active infections and malignancy should be included in the assessment; and f) 100% agreed with including LA test based on International Society on Thrombosis and Hemostasis (ISTH) guidelines (33).

DISCUSSION

The first two phases of the APS Classification Criteria development support the concept that clinical manifestations associated with aPL are heterogeneous and complex. We used a data-driven, consensus-based approach to identify and organize 27 candidate criteria for classification of APS into six separate domains using previously established methodology and incorporating bias reduction techniques. Phase I/II involved a dedicated, multi-disciplinary team of approximately 80 physicians, investigators, epidemiologists, classification criteria methodologists, and patients worldwide, allowing for diversity of opinions and experiences. In the next phase, these proposed candidate criteria will be used for real-world case collection and further refined, hierarchically organized, and weighted so that an aggregate score and threshold for APS classification can be determined.

Our methodology builds on recently published rheumatic disease classification criteria including that of SLE, systemic sclerosis, rheumatoid arthritis, and IgG-4 related disease, but also includes novel concepts (34–36). Given our findings that physicians view aPL-positive patients in different subpopulations, e.g., by thrombosis provoking risk factors, age, and aPL profile, consideration of these factors provides a unique risk stratification aspect to the new classification criteria development, which differs from previous rheumatic disease or prior APS classification criteria (3). Additionally, the need to evaluate candidate clinical criteria in the context of a clinically significant aPL profile, emphasizes the important role of entry criteria, and may alter the structure of the final threshold determination by suggesting the need for separate clinical and laboratory summation scores.

Classification criteria, in contrast to diagnostic criteria, are used to identify relatively homogeneous groups of patients for inclusion in clinical trials and observational studies; thus, the goal of the new APS classification criteria is to better standardize patients for APS research. Given the heterogeneity of aPL-related clinical manifestations and laboratory tests, our methodology ultimately aims to develop a weighted scoring system to increase the validity and performance of APS classification criteria. Through expert and data-driven methods, we captured the wide spectrum of APS manifestations, which were eventually refined to more specific manifestations. In particular, inclusion of microvascular, hematologic, and cardiac valve disease domains will improve the sensitivity and generalizability of the new classification criteria.

Given the major morbidity and mortality risk related to APS and currently limited treatment options, the development and validation of stringent, high-performing classification criteria is critical to improve understanding of APS epidemiology and clinical outcomes. Additionally, research on the immunologic, genetic, and diagnostic aspects of APS has been evolving rapidly. Fortunately, our new classification criteria methodology will allow us to modify the domains in the future, e.g., to include new commercially available laboratory tests or clinical criteria shown to be highly specific for APS.

Our study has a number of potential biases and limitations. First, a major source of potential bias in classification criteria development is one of circularity of reasoning, which may result when the same experts who contribute to development of classification criteria also provide patient data (in Phase III) and participate in validation (Phase IV). Additionally, we asked experts to consider the clinical spectrum of aPL/APS during Phase I/II, and not to use the previously published Sapporo criteria as a decision-making guide. As one bias reduction strategy, we ensured that the majority of collaborators involved in Phase I/II will be mutually distinct from those who will be involved in future phases. Second, our literature reviews revealed a relative dearth of studies evaluating the performance characteristics (e.g., sensitivity and specificity) or prevalence of certain candidate criteria. However, by using expert-based surveys, nominal group technique, and Delphi exercises, forming Domain Subcommittees, and conducting our own meta-analyses and additional studies, we aimed to address these gaps in knowledge and to minimize its effect on the validity and reliability of the data.

Following the work described above, Phase III will refine the definitions of entry criteria and relative candidate criteria, and collect real-world, international potential APS cases. This next phase aims to use Multi-Criteria Decision Analysis methodology and 1000Minds software. We will test our criteria for reliability and applicability, employ a systematic methodology for weighting and threshold determination, test the performance of the scoring system at the margin between “included” and “excluded” cases, and use cohorts for testing and validation, which deliberately include the grey zone where classification is difficult.

In conclusion, substantial advances in the methodology for classification criteria development have occurred since formulation of the original APS Sapporo Classification Criteria. By employing a new methodology used for the development of classification criteria for other rheumatic diseases, and incorporating novel disease-specific strategies, we anticipate that the new APS Classification Criteria will have excellent face validity, criterion validity, and performance.

SIGNIFICANCE AND INNOVATION.

• During the ‘Item Generation’ and ‘Item Reduction’ phases of an international multi-disciplinary initiative that is underway to develop new rigorous classification criteria to identify patients with high likelihood of Antiphospholipid Syndrome (APS), we generated 261 items, which were reduced to 27 potential APS candidate criteria

• Using data- and consensus-driven methodology, we identified 27 APS candidate criteria organized into six domains (laboratory, macrovascular, microvascular, obstetric, cardiac, and hematologic), which will be further refined, organized, and weighted to determine an aggregate score and threshold for APS classification.

• This work demonstrates a potential role for clinical components not previously included in the APS Sapporo Classification Criteria.