Abstract
Objectives
To validate the preliminary criteria of global flare for childhood-onset SLE (cSLE).
Methods
Pediatricians experienced in cSLE care (n=268) rated unique patient profiles (PP); results of standard cSLE laboratory testing and information about the cSLE flare descriptors were presented: global assessment of patient well-being, physician global assessment of disease activity (MD-global), disease activity index score, protein/creatinine ratio (PCR), and ESR. Using rater interpretation of the course of cSLE (baseline vs. follow-up as the gold standard), performance [sensitivity, specificity, area under the receiver operating characteristic curve (AUC)] of the preliminary flare criteria (Arthritis Care & Research, 2011) was tested. An international consensus conference was held to rank the preliminary flare criteria as per the ACR-recommendations and delineate threshold scores for minor, moderate and major flares.
Results
The accuracy of the two highest ranked candidate criteria which consider absolute changes (∆) of the SLEDAI or BILAG (numeric scoring: A=12; B=8; C=1; D/E=0), MD-global, PCR, and ESR were confirmed (both AUC > 0.93). For the SLEDAI-based criteria [0.5x ∆SLEDAI + 0.45x ∆PCR + 0.5x ∆MD-global + 0.02x ∆ESR] flare scores ≥6.4/3.0/0.6 constituted major/moderate/minor flares. For the BILAG-based algorithm [0.4x ∆BILAG + 0.65x ∆PCR+0.5x ∆MD-global + 0.02x ∆ESR] flare scores ≥7.4/3.7/2.2 delineated major/moderator/minor flares. These threshold values (SLEDAI, BILAG) were all >82% sensitive and specific for capturing flare severity.
Conclusions
Provisional criteria for global flares in cSLE are available to identify patients who experienced a flare. These criteria also allow for discrimination of the severity of cSLE exacerbations.
Keywords: lupus, childhood-onset SLE, SLE, pediatric SLE, juvenile SLE, flare, criteria, children, cSLE
INTRODUCTION
Systemic lupus erythematosus is a complex, chronic multi-system autoimmune inflammatory disease, with up to 20% of patients diagnosed during childhood (cSLE) (1, 2). When disease commences early in life rather than during adulthood, it has a less favorable prognosis, particularly due to multi-organ and kidney involvement (3, 4). The course of cSLE is characterized by episodes of disease flares; followed by periods of improvement, generally due to more intensive drug therapy. There is international consensus that a flare of cSLE is “a measurable worsening of disease activity in at least one organ system, involving new or worse signs of disease that may be accompanied by new or worse SLE symptoms; depending on the severity of the flare, more intensive therapy may be required” (5). Further, using consensus formation techniques, agreement has been achieved regarding preliminary criteria of global flare of cSLE based on changes of the erythrocyte sedimentation rate (ESR), the protein/creatinine ratio (PCR), physician global assessment of cSLE activity (MD-global), and the score of the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) (6, 7) or the British Isles Lupus Activity Group index (BILAG) (8). Moreover, there is consensus around the need to discriminate flares as per their severity: mild/minor, moderate, and major/severe flares (5). However, there are no generally accepted criteria or algorithms to determine how to measure the severity of cSLE flares, nor have the preliminary cSLE flare criteria been validated in an independent dataset. Thus, the objectives of this phase of the project were to validate the preliminary criteria of global flare of cSLE and to apply consensus formation methodology to define flare threshold levels for minor, moderate and major flares. These criteria were created to define cSLE flares and their severity for use in clinical trials.
PATIENTS AND METHODS
The overall approach to this project was based on the methodological framework successfully employed in pediatric rheumatology criteria measures in the past (9–11), aligned with recommendations of the ACR Criteria Subcommittee and the Quality of Care Committee (QOC) (12). The initial results of the consensus process resulting in preliminary cSLE flare criteria have been described elsewhere (5, 13). Briefly, previous research demonstrated that the scores of a disease activity measure alone are inadequate for identifying flares (5). International agreement was reached regarding preliminary criteria to measure global flares of cSLE. Pediatric rheumatologists participated in Delphi surveys that yielded consensus around a common definition of cSLE global flares, and the delineation of cSLE flare descriptors. This was followed by exploration of candidate flare criteria (5) and the identification of preferred algorithms of global cSLE flares (14). Notably, data and analyses all suggested that uniform percentage changes of the cSLE flare descriptors are insufficient to capture cSLE flares with high sensitivity. Further, inclusion of the MD-global assessment of cSLE activity in highly accurate cSLE candidate flare algorithms proved necessary (5, 15). During the first Consensus Conference the top-performing candidate flare algorithms, derived either from multinomial logistic regression modeling or classification tree analysis (CART) were established.
We now present the phase of the project aimed at validating the preferred preliminary flare algorithms (14) via testing in an independent validation data set (Figure 1). These encompassed Patient Profiles PP ratings that were requested from 503 pediatric rheumatologists from Australia, Africa, Asia, Europe and the Americas who were members of at least one of the following organizations: Pediatric Rheumatology Collaborative Study Group, Childhood Arthritis Rheumatology Research Alliance, Pediatric Rheumatology European Society Juvenile Lupus Working Group, and Pan-American League of Arthritis & Rheumatology [Step 1].
Figure 1. Flow diagram of the entire process used to develop and validate the approved criteria of global flare of cSLE.
The steps 1–5 have been summarized in Brunner & Klein-Gitelman: Arthritis Care Res (Hoboken). 2010;62(6):811–20; and Brunner & Mina: Arthritis Care & Research. 2011;63(9):1213–23. The current report commences at step 5 and focuses on steps 6 – 8.
The interpretation of the flare or ‘true’ disease course of a given PP was determined using two approaches, which resulted in two distinct datasets for the subsequent validation exercises [Step 2]. Using the PP ratings, the preliminary criteria for cSLE global flares were tested for their ability to discriminate patients who experienced different levels of flares (minor, moderate, major) [Step 3]. Subsequently, during a Consensus Conference, the validity of the criteria was critically reviewed, taking into consideration information from the medical literature, statistical performance, reliability, feasibility, and face validity as per the ACR guidance document and the OMERACT filter (16) [Step 4].
Preliminary cSLE Flare Algorithms
We considered the top four preliminary flare algorithms (identified in the first Consensus Conference) based on feasibility, truthfulness and discrimination (17). Two of the four preliminary cSLE flare algorithms [SLEDAI-based criteria: 0.5x ∆SLEDAI + 0.45x ∆PCR + 0.5x ∆MD-global + 0.02x ∆ESR; BILAG-based criteria: 0.4x ∆BILAG + 0.65x ∆PCR+0.5x ∆MD-global + 0.02x ∆ESR] were derived by multinomial logistic regression that considered several of the cSLE flare descriptors, and yield “flare scores” (or log odds of flare), with higher score representing a higher likelihood of a flare to have occurred. The other two of the top preliminary flare criteria were derived from CART [SLEDAI-CART: Score=4 if 3 ≤ SLEDAI; Score=3 if 0.7 ≤ PCR and 3 > SLEDAI; Score=2 if 2 ≤ MD and 0.7 > PCR and 3 > SLEDAI; and Score=1 Otherwise; BILAG-CART: Score=4 if 2 ≤ BILAG; Score=3 if 0.7 ≤ PCR and 2 > BILAG; Score=2 if 2 ≤ MD and 0.7 > PCR and 2 > BILAG; Score=1 Otherwise]. Similar to algorithms derived by multinomial logistic regression, CART-based criteria yield ‘CART-scores’ that can be used to decide on the presence of a flare, including its severity (18).
Step A: Patient Profiles & Ratings of Disease Course of a Patient Profile
Two of the authors (MH, HIB) conducted a pilot study to test the format of the PP. Built on this pilot study, we generated over 2,996 unique PPs, using prospectively collected data of cSLE patients from the CCHMC Lupus Registry (19), the PRINTO Lupus Cohort (20), the United Kingdom Juvenile-onset SLE Cohort Study (21), and the APPLE trial (22). Missing observations in the datasets were imputed using multiple imputation methods and expectation–maximization algorithms in computation (23–25).
Each PP provided data about a patient at the time of a baseline visit and a follow-up visit. For each PP visit, the cSLE flare descriptors were provided (5): [1] MD-global, measured on a visual analog scale (VAS) (0 = inactive disease; 10 = very active disease); [2] parent assessment of patient overall well-being, measured on a VAS with a range from 0 to 10 (0 = very poor; 10 = very well); [3] proteinuria, measured by timed urine collection or spot PCR; [4] ESR; [5] levels of complement C3 and C4; [6] item and summary scores of the SLEDAI, version 2k (7), or the domain and summary scores of the BILAG using the following numeric conversion: A=12; B=8; C=1; D/E=0 (8). Information on complete blood counts and differential, serum chemistry, urinalysis and anti-dsDNA antibodies were also provided. Details on PP formats are provided in Appendix 1.
Disease Course
PP raters were randomly assigned to assess the disease course of a maximum of 51 PP. Response options offered were: major flare; moderate flare; minor flare; unchanged; improved; or “I do not have enough information to make this assessment”. A global flare was considered as “present” whenever the disease course was rated as minor, moderate, or major flare.
Step B: Adjudication of Disease Course of the PP
A randomization scheme was pre-planned to ensure that each PP was sent to about 13 raters, with the ratio of American and international raters matching that of the PP raters’ pool (about 1:1). PP with fewer than 4 ratings were regarded as “invalid” or “unqualified” and excluded from further consideration. Only “qualified” PP with successful adjudication were considered in Step 3.
Adjudication of the (true) disease course
Given that PP raters may not necessarily agree on the disease course, the “true” overall course of cSLE for a given PP was adjudicated using two approaches; (a) 67%-Rule: at least 2/3 of the raters agreed on a given disease course, (b) Majority-Rule: the majority of the raters of a PP agreed on a given disease course. Other Rules (50%-Rule and 75%-Rule) were also explored and results were similar to the Majority-Rule and the 67%-Rule, respectively; hence they are not presented herein.
Step C: Assessment of Performance
Statistical analysis in preparation of the testing of preliminary flare criteria
Considering the intended widespread use of the cSLE flare criteria (14), we tested whether there were systematic differences in the ratings provided by raters (a) from different geographic regions, or (b) with varying professional experience as measured by the duration of medical practice. Agreement among raters was assessed using intra-class correlation coefficients (ICC) and/or Kappa (κ) statistics. An ICC or a κ value can be interpreted as follows: poor agreement: ICC or κ< 0.4; fair to good agreement: ICC or κ≥ 0.4– 0.75; substantial to excellent agreement: ICC or κ> 0.75 (26).
Performance & Accuracy
Each of the four flare algorithms (SLEDAI-based criteria, BILAG-based criteria, SLEDAI-CART, BILAG-CART) was assessed for diagnostic accuracy using receiver’s operating characteristic (ROC) curve analysis. Specifically, the area under the ROC curve (AUC) was calculated, and the diagnostic accuracy was considered outstanding, excellent, good, fair, and poor if the AUC was in the range of 0.9–1.0, 0.81–0.90, 0.71–0.80, 0.61–0.70, and < 0.60, respectively (18, 27). Different from flare criteria derived from multinomial regression models (SLEDAI-based criteria, BILAG-based criteria), CART-based flare algorithms (SLEDAI-CART and BILAG-CART) result in a single discrete value for sensitivity and specificity, respectively. Considering all possible flare scores, the overall diagnostic accuracy of an algorithm can be estimated.
Threshold score candidates for algorithms derived by multinomial logistic analysis
In the absence of strong guidance from the ACR, we used two statistical methods to define potential threshold scores: (a) in an earlier phase of the project, consensus had been achieved that “flare score threshold” for a given algorithm should reflect the highest conditional AUC among all candidate thresholds on a ROC curve. Hence, these flare score thresholds represent the point on the ROC curves with the highest precision of correctly classifying the severity of a cSLE flare. (b) We also explored a distribution-weighted approach in which the flare score threshold was calculated based upon the average of means of scores in two neighboring flare states weighted by the standard deviations of the scores. The performance of the candidate thresholds from both statistical analyses (a, b) was calculated and average accuracies for the correct identification of minor, moderate and major flares for the SLEDAI-based and BILAG-based algorithms.
Step D: Ranking of Candidate Flare Criteria &Thresholds Score
To support decision making, Consensus Conference participants reviewed a syllabus that provided the results of the preceding Delphi surveys, relevant published medical literature and the results of the statistical analyses prior to the Consensus Conference (see Step 3). Participants in the Consensus Conference were 13 experienced pediatric rheumatologists and nephrologists from South America, North America, Asia, and Europe with substantial clinical and research experience in cSLE (HIB, MWB, SPA, SA, CAS, FF,BG, SEW, DML, AR, RK, TA, and MKG). A priori, the consensus level at the consensus conference was set at 75%, i.e. comparable or even somewhat higher than that chosen for similar studies in the past (15–19). Using nominal group technique guided by an experienced moderator (BMF), the expert panel assessed each of the four top candidate flare algorithms (14) and potential flare score thresholds according to [1] feasibility, i.e. practicability: can the items be measured easily?; [2] reliability, i.e. reproducibility: can the items be measured precisely?; [3] redundancy: are there two or more items included in the candidate criteria measuring the same aspect of the disease?); [4] face validity, i.e. credibility: are the criteria sensible?; [5] content validity, i.e. comprehensiveness: do the criteria sample all of the domains of the disease?; [6] criterion validity: based on AUC, do the criteria accurately approximate the “gold standard”, i.e. the adjudicated disease course as per 67%-Rule or Majority-Rule?; [7] sensitivity and specificity: do the criteria effectively identify patients with cSLE flares and distinguish them from patients who do not have a flare of their cSLE?; and [8] discriminant validity: do the criteria detect the smallest clinically important change? (i.e. discriminate patients with one of the following disease courses: minor flare, moderate flare, major flare, no flare). Based on the above considerations, the Consensus Conference experts were asked to rank the candidate flare criteria from 1 (lowest) to 4 (highest criterion).
The survey source data were batch-processed, and open source online survey software, Limesurvey, was used for response management and as a presentation layer (see http://www.limesurvey.org/).
All analyses were done using SAS 9.4 (SAS, Cary, NC) software and SYSTAT 12 (Systat Software, Inc, Chicago, IL) software. P-values < 0.05 were considered statistically significant.
Ethics Review
The study was approved by the institutional review boards of the participating pediatric rheumatology centers. Informed consent was obtained from all parents and, as appropriate, assent was given by the participants prior to the study procedures.
RESULTS
Patient Profile Raters and Validation Dataset (Steps A and 2)
A total of 2,996 ratings were provided to 503 pediatric rheumatologists and used for Step 2. The response-rate of the pediatric rheumatologists to the PP was 54% (274/503; locations: 30% from the U.S. and Canada; 8% from Australia/Asia, 3% Africa/Middle East, 40% South and Central America, and 19% Europe). The majority (69%) of PP raters had over 10 years of experience in treating cSLE. There were 1860 PP (1860/2996= 62%) that were rated by at least 4 raters, hence considered “qualified” for inclusion in Step 3. There were no significant differences of distribution of flares between qualified and unqualified PP (Fisher’s exact test, p=0.62).
When the Majority-Rule was applied to the “qualified” PP, there were 1318 PP representing global flares (510 minor flares, 483 moderate flares and 325 major flares) and 542 unchanged/improved (29% of 1860 PP). When applying the 67%-Rule to the 1860, only 818 PP remained available for analysis, among them 484 representing a flare (194 minor flares, 146 moderate flares and 144 major flares) and 334 PP without cSLE flare. The patient characteristics reflected in these PP are summarized in Table 1. PP raters from different geographic locations did not differ systematically in the disease course assignment for a given PP (North America vs. other countries: ICC = 0.658). Similarly, there was fair to good agreement among PP raters with different duration of medical experience (3–5, 6–10, 10–15, >15 years) for the interpretation of the disease courses (ICC = 0.656). Additionally, we explored other selection criteria (50% Rule, 75% Rule) and found no systematic differences with the 50% Rule and 75% resulting in similar adjudication of the PP compared to the Majority-Rule and the 67%-Rule, respectively [data not shown].
Table 1.
Baseline Characteristics of Validation Cohort
| Values are % from N, unless stated otherwise | Majority Rule (N=1860) | 67% Rule (N=818) |
|---|---|---|
|
| ||
| Mean age (Years) | 15.0 | 15.1 |
|
| ||
| Gender (% Of Females) | 81.7% | 82.5 % |
|
| ||
| Protein-Creatinine Ratio* | 0.39 | |
| ≤ 0.2 | 63.8% | 67.5% |
| > 0.2 | 36.2% | 32.5% |
| > 0.5 | 14.5% | 13.0% |
| > 2.0 | 3.4% | 2.7% |
|
| ||
| Organ Involvement With Active cSLE At Baseline | 2.7% | 7.0% |
| Neuropsychiatric | 12.4% | 8.67% |
| Musculoskeletal | 21.7% | 22.6% |
| Mucocutaneous | 15.4% | 12.7% |
| Hematologic | 24.1% | 20.5 % |
| Renal | 1.2% | 1.0% |
| Cardiopulmonary | 2.7% | 8.1% |
| Constitutional symptoms | ||
either from 24 hour urine or random urine sample; (mg protein/mg urine creatinine)
Performance of Preliminary Algorithms of cSLE global flares (Step C)
The absolute baseline-to-follow-up changes of the parameters considered in the preliminary flare algorithms by flare severity and rule are provided in Table 2. Irrespective of the dataset (67%-Rule; Majority-Rule), most of the cSLE flare descriptors included in the preliminary cSLE flare criteria (ESR, PCR, MD-global, SLEDAI, BILAG) significantly changed between the baseline and follow-up visit, by flare severity.
Table 2.
Change of Descriptors in Relationship to cSLE Disease Course †
| Flare Descriptors | Rule | Mean ± SE | Adjusted p-value | |||||
|---|---|---|---|---|---|---|---|---|
| (1) Improved/No change |
(2) Minor Flare |
(3) Moderate Flare |
(4) Major Flare |
(1) vs. (2) | (2) vs. (3) | (3) vs. (4) | ||
| ESR | Majority rule | −0.02 ± 1.30 | 8.81 ± 1.34 | 22.80 ± 1.38 | 28.99 ± 1.68 | <0.0001 | <0.0001 | 0.023 |
| 67% rule | 0.54 ± 1.58 | 7.28 ± 2.07 | 31.95 ± 2.39 | 35.34 ± 2.41 | 0.048 | 0.000 | 0.749 | |
| MD global of disease activity | Majority rule | 0.66 ± 0.50 | 3.05 ± 0.52 | 5.92 ± 0.53 | 7.95 ± 0.65 | 0.005 | 0.001 | 0.075 |
| 67% rule | 0.76 ± 0.60 | 2.70 ± 0.79 | 7.74 ± 0.91 | 9.79 ± 0.92 | 0.210 | <0.0001 | 0.392 | |
| Protein-creatinine ratio | Majority rule | 0.02 ± 0.07 | 0.10 ± 0.07 | 0.66 ± 0.07 | 1.44 ± 0.08 | 0.843 | <0.0001 | <0.0001 |
| 67% rule | 0.03 ± 0.07 | 0.02 ± 0.09 | 0.64 ± 0.11 | 1.61 ± 0.11 | 1.000 | <0.0001 | <0.0001 | |
| SLEDAI | Majority rule | 1.81 ± 0.26 | 4.58 ± 0.28 | 8.45 ± 0.29 | 16.00 ± 0.36 | 0.000 | <0.0001 | <0.0001 |
| 67% rule | 1.56 ± 0.35 | 4.63 ± 0.48 | 9.98 ± 0.56 | 19.88 ± 0.55 | 0.000 | <0.0001 | <0.0001 | |
| BILAG | Majority rule | 3.12 ± 1.08 | 7.76 ± 0.93 | 15.19 ± 0.95 | 24.19 ± 1.15 | 0.007 | <0.0001 | <0.0001 |
| 67% rule | 1.79 ± 1.34 | 8.63 ± 1.50 | 15.64 ± 1.61 | 28.71 ± 1.75 | 0.005 | 0.010 | <0.0001 | |
| SLEDAI-based Flare Algorithm | Majority rule | −0.23 ± 0.17 | 1.66 ± 0.18 | 4.79 ± 0.19 | 9.88 ± 0.23 | <0.0001 | <0.0001 | <0.0001 |
| 67% rule | −0.34 ± 0.21 | 1.67 ± 0.29 | 5.84 ± 0.34 | 12.34 ± 0.34 | <0.0001 | <0.0001 | <0.0001 | |
| BILAG-based Flare Algorithm | Majority rule | 0.40 ± 0.56 | 3.00 ± 0.48 | 7.10 ± 0.49 | 11.96 ± 0.60 | 0.003 | <0.0001 | <0.0001 |
| 67% rule | −0.11 ± 0.66 | 3.49 ± 0.76 | 8.23 ± 0.79 | 15.05 ± 0.88 | 0.003 | <0.0001 | <0.0001 | |
Values presented are changes in means (standard deviation) adjusted for multiple comparisons using the Tukey’s method.
Notably, the accuracy of the SLEDAI-based algorithm was outstanding [AUC= 0.93; 95% confidence interval (CI): 0.91– 0.95] as was that of the BILAG-based algorithm [AUC= 0.93; 95% CI: 0.89– 0.98]. The CART-SLEDAI algorithm had an excellent accuracy for identifying patients with global flare of cSLE (any severity) [AUC= 0.89; sensitivity= 88.8%; specificity= 87.1%]. The same was true for the CART-BILAG criteria [AUC= 0.84; sensitivity= 93.9%; specificity= 72.9%]. Comparisons of accuracies in the development data set in 2010 (18) and this validation data set are summarized in Table 3.
Table 3.
Comparison of the Performance of the Preliminary Flare Algorithm in the Development and Validation Dataset
| Algorithm details | Flare Category | Area under the ROCˆ curve | ||
|---|---|---|---|---|
| 2010 data | 2017 data | |||
| SLEDAI-based flare score$ | Score=0.5 × SLEDAI + 0.45 × PCR** + 0.5 × MD + 0.02 ESR | Major flare | 0.95 | 0.93 |
| At least moderate flare | 0.85 | 0.94 | ||
| At least minor flare | 0.86 | 0.93 | ||
| BILAG-based flare score$ | Score=0.4 × BILAG + 0.65 × PCR + 0.5 × MD + 0.02 ESR | Major flare | 0.93 | 0.91 |
| At least moderate flare | 0.85 | 0.92 | ||
| At least minor flare | 0.85 | 0.93 | ||
| SLEDAI-based CART rule | Score=4 if 3 ≤ SLEDAI; Score=3 if 0.7 ≤ PCR and 3 > SLEDAI; Score=2 if 2 ≤ MD and 0.7 > PCR and 3 > SLEDAI; Score=1 Otherwise. |
Major flare | 0.85 | 0.76 |
| At least moderate flare | 0.80 | 0.80 | ||
| At least minor flare | 0.84 | 0. 89 | ||
| BILAG-based CART rule | Score=4 if 2 ≤ BILAG; Score=3 if 0.7 ≤ PCR and 2 > BILAG; Score=2 if 2 ≤ MD and 0.7 > PCR and 2 > BILAG; Score=1 Otherwise. |
Major flare | 0.86 | 0.71 |
| At least moderate flare | 0.80 | 0.75 | ||
| At least minor flare | 0.82 | 0.84 | ||
Details about algorithm development are provided in Brunner, H. I., Mina, R. “Preliminary criteria for global flares in childhood-onset systemic lupus erythematosus.” Arthritis Care Res (Hoboken) 2011, 63(9): 1213–1223.
Algorithm considers for the change (baseline – follow-up) of each of the flare descriptors included
Values presented represent the area under the ROC curve considering PP with consensus as defined by the 67%-Rule
PCR: Urine protein/creatinine ratio from random urine sample
MD-global: Physician global assessment of disease measured on a visual analog scale (range: 0–10; 0= inactive disease)
Numeric values larger than or equal to the flare score signify a flare; higher scores are seen with more severe flare.
Receiver operating characteristic
Flare Thresholds
Figure 2, Panel A and B depict potential thresholds for defining minor, moderate and major flares. In this final Consensus Conference, again consensus (100%) was reached to use the statistically optimal threshold from logistic models to define all threshold scores for the both SLEDAI-based and the BILAG-based algorithms. As shown in Figure 3, Panel A and B using these threshold cut-off scores allows for the discrimination of minor from moderate or severe flares, all with sensitivities and specificities of ≥ 82%. Neither of the CART-based algorithms was suited to discriminate between mild and moderate cSLE flares (Figure 3, Panel C and D).
Figure 2. Potential flare thresholds to define cSLE flare severity. Panel A: SLEDAI-based algorithm, Panel B: BILAG-based algorithm.
Flare threshold values based on multinominal logistic regression models and distribution-weighted strategies for each flare category (minor, moderate, major flare) were presented to the experts participating in the final concensus conference. There was 100% agreement to use threshold values derived from multinomial logistic regression, i.e. thresholds with the best statisticial performance in receiver-operating characteristic curve analysis. Each threshold had the largest summation of sensitvity and specificity on the ROC curve. Blue bars represent threshold scores from multinomial logistic regrssion models and yellow bars depict those derived from distribution-weighted approaches. Red bars indicate the scores using each algorithm to assess the 2010 data (14).
Distribution-weighted
Logistic
Figure 3. Flare score interpretation.
Flare scores represent the cut-off score on the Receiver Operating Characteristic (ROC) curves that provide the best discrimination between adjacent disease states (no flare, minor or mild flare, moderate flare, major or severe flare) with cSLE. Sensitivities and specificities are shown for the SLEDAI-based algorithm in Panel A, and the BILAG-based algorithm in Panel B. As shown in Panel C, the SLEDAI-CART algorithm [Score= 4 if 3 ≤ SLEDAI; Score= 3 if 0.7 ≤ PCR and 3 > SLEDAI; Score= 2 if 2 ≤ MD and 0.7 > PCR and 3 > SLEDAI; and Score=1 Otherwise] and in Panel D the BILAG-CART algorithm [Score= 4 if 2 ≤ BILAG; Score= 3 if 0.7 ≤ PCR and 2 > BILAG; Score= 2 if 2 ≤ MD and 0.7 > PCR and 2 > BILAG; Score= 1 Otherwise] are only able to distinguish major flares from other cSLE disease courses. Thus the other two of the top preliminary flare criteria (SLEDAI-CART, BILAG-CART) were unable to discriminate minor from moderate cSLE flare.
Ranking of the Preliminary cSLE Flare Algorithms (Step D)
Consensus Conference participants achieved consensus that the BILAG-based (92%) and SLEDAI-based (100%) flare algorithms have both construct validity for measuring global flares of cSLE. There was consensus (100%) to recommend both measures to be collected in future cSLE clinical trials and that either one may be chosen as the primary endpoint. Consistent with their performance in the validation data set, no consensus was reached whether one of these two algorithms was preferable to the other. Consensus was achieved that CART-based algorithms are not suited for use in clinical trials, given that these algorithms cannot be used to discriminate minor from moderate cSLE flares. The results of this study were reviewed by the ACR Criteria Subcommittee and the ACR Quality of Care Committee.
DISCUSSION
The need to develop internationally agreed upon criteria for disease flares has become more urgent since the introduction of randomized withdrawal trials in pediatric rheumatology, in which time to flare or the proportion of patients who experience a flare are used as primary efficacy measures (28). We confirm the outstanding accuracy of the previously developed preliminary criteria of global flares of cSLE, based on large international datasets used for validation. Consensus has been achieved on how to interpret flare scores. The preferred cSLE global flare algorithms for use in clinical trials were derived from multinomial logistic regression models. These algorithms consider the differential and complementary contribution of select cSLE flare descriptors in identifying disease flares in this disease with highly variable multi-organ involvement. Despite consensus that CART-based algorithms are potentially of value when used in clinical care settings, there was agreement that they should not be used in clinical trials.
As for SLE in adulthood, measures of the overall course are especially relevant because not all cSLE features improve or worsen in parallel. Current drugs used in cSLE therapy are not equally effective in reducing disease activity in the various organ systems. Thus it is reasonable to assume that the same holds true for new or emerging drugs for cSLE. In clinical trials aimed at reducing cSLE-mediated inflammation in certain organ systems, it appears mandatory to ensure that global disease, i.e. disease manifestations in other than the target organ systems, is not worsening. The results of this study support that the SLEDAI-based and the BILAG-based flare scores are both highly suited to provide such information.
Based on the current evidence about these algorithms they are similarly sensitive, specific and accurate. Hence, Consensus Conference experts considered both algorithms equally valuable and suitable for use in clinical trials. Different from what is currently used to gauge response to therapy in juvenile idiopathic arthritis (29), flare algorithms derived from regression models allow for consideration of the differential importance of changes in individual cSLE flare descriptors when recognizing cSLE flares. The SLEDAI-based and BILAG-based flare scores are reminiscent of the disease activity score (DAS) used in rheumatoid arthritis (30). However, the DAS score considers the natural logarithm of the ESR and square roots of the number of swollen or tender joints, while the preliminary cSLE flare criteria require at most simple arithmetic maneuvers to calculate a cSLE flare score, supporting their ease of use (18).
All flare score algorithms consider changes in proteinuria, despite the inclusion of proteinuria assessment in the SLEDAI and BILAG scores. This allows for detection of renal SLE flares that occur in patients with existing proteinuria and also allows for the consideration of increases in proteinuria that would otherwise not be captured given the item definition used in the SLEDAI and BILAG, respectively. As reported previously, exclusion of changes in proteinuria from the flare algorithms resulted in inferior accuracy in predicting cSLE flares (14).
In line with our earlier studies (5, 8) both cSLE flare criteria from CART and multinomial logistic regression analysis showed excellent or even outstanding accuracy. Statistically, they were superior to algorithms that considered equally weighted percentage changes from a statistical point of view in the past.
Given the simplicity of CART-based criteria, they appear particularly suited for clinical settings but a potential short-coming of CART-based criteria include so-called ‘over-fitting of the mathematical model’ which can make them prone to less favorable statistical performance in subsequent validation studies (14). Mild cSLE flares often do not prompt clinicians to change therapy, whereas moderate cSLE generally require more intensive anti-inflammatory therapy. Although CART-based flare algorithms were highly accurate for discriminating any kind of global flare when tested in this validation data set, they were unable to distinguish minor from moderate cSLE flares. This limitation prompted the agreement among the Consensus Conference experts to not recommend CART-based algorithms for use as outcome measures in clinical trials.
We chose two approaches to adjudicate the disease course (67%-Rule, Majority-Rule) presented in the various PPs, which might have introduced bias. However, both approaches yielded comparable results.
The ACR has outlined a series of validation steps necessary before new criteria are to be widely used for clinical care or research (12). Among others, one step is to use data from clinical trials for developing response criteria. However, clinical trial data from interventions that impact cSLE activity are unavailable at present. In our study, the presence of a flare was based on the PP raters’ perception of the course of cSLE instead. Given their prospective character and the expertise of the PP raters, we consider the quality of our data to be high and the number of PPs per flare severity category yielded robust provisional cSLE flare criteria.
We would like to point-out that PP raters from different parts of the world and different degrees of experience showed all excellent concordance (inter-rater agreement) in their assessment of the cSLE course. This supports the robustness of this validation study. A limitation might be that only 54% of those physicians approached to provide PP ratings provided feed-back. Nonetheless, responses from 274 pediatric rheumatologists were obtained, which is a much larger number than for many similar validation exercises (9–11).
In addition to criteria for global flare and improvement, criteria for changes of cSLE in specific organ systems are likely needed. Depending on the proposed effect of a cSLE drug candidate, the Cutaneous Lupus Activity and Severity Index (31), pediatric lupus nephritis response measures (32) and standardized joint assessments for children (29), have already been validated to adequately capture the proposed therapeutic effects. To further provide support for the accuracy of the provisional criteria of global flare of cSLE data from clinical trials will be needed.
Taken together a methodologically stringent validation process has been employed to calculate a flare score that can be used to interpret the course of cSLE over time with respect to the degree of worsening that might have occurred. Based on the data available these algorithms cannot be used to quantify potential improvement over time.
SIGNIFICANCE & INNOVATION.
Results of the preliminary validation of criteria of global flare for childhood-onset SLE are provided.
Based on the flare scores mild flares, moderate flares and severe flares can be defined.
Acknowledgments
CCHMC: Kasha Wiley (overall study coordination), Susan Priest (consensus conference logistics), Pinar Avar (consensus conference support and data management), Carly Muller, Malea Rolfsen, Allen Watts, Gaurav Gulati and Jamie Meyers-Eaton (patient profile testing); CCHMC Biomedical Informatics (Web-based data management application development).
A special thanks to Drs. Laura Schanberg and Christy Sandberg and CARRA for provision of the data from the APPLE clinical trial.
A special thanks to the UK JSLE Study Group, for provision of the data from the UK JSLE Cohort Study
We are indebted to the members of the External Scientific Advisory Committee of this study for their advice in the study implementation, conduction and its statistical analysis: Drs. Tuhina Neogi, Ian Bruce, David Isenberg, Nicola Ruperto and James Witter.
Grant Support:
The study is supported by NIH grants 5U01-AR51868, P30-AR AR47363 and 2UL1RR026314.
This study is also supported by grants from Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP 2015/03756-4 to CAS), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq 303422/2015-7 to CAS) and by Núcleo de Apoio à Pesquisa “Saúde da Criança e do Adolescente” da USP (NAP-CriAd) to CAS.
This study is also supported by LUPUS UK, who supports the UK JSLE Cohort Study, along with the National Institute of Health Research (NIHR) Clinical Research Network (CRN), NIHR CRN Children’s Specialty Group and NIHR Alder Hey Clinical Research Facility.
APPENDIX 1
A. Patient Profiles considering the SLEDAI
B. Patient Profiles considering the BILAG
| Gerd | Ganser | St. Josef Stift Sendenhorst | Ganser@St-Josef-Stift.De |
|---|---|---|---|
| Claas | Hinze | University Hospital Muenster | Claas.Hinze@Gmail.Com |
| Markus | Hufnagel | University Children’s Hospital Freiburg | Markus.Hufnagel@Uniklinik-Freiburg.De |
| Thomas | Lutz | University Hospital Heidelberg | Thomas.Lutz@Med.Uni-Heidelberg.De |
| Ralf | Trauzeddel | Helios Klinikum Berlin Buch | Ralf.Trauzeddel@Helios-Kliniken.De |
| Greece: | |||
| Sorina | Boiu | Attikon University Hospital, Athens | Sboiu@Med.Uoa.Gr |
| Maria | Trachana | Hippokration Hospital, Aristotle University of Thessaloniki | Mtrachan@Auth.Gr |
| Elena | Tsitsami | Agia Sofia University Children’s Hospital | Elena@Tsitsamis.Gr |
| Guatemala: | |||
| Mayra | Cifuentes | Hospital General San Juan de Dios | Mayra.Cifuentes@Gmail.Com |
| Hungary: | |||
| Ilonka | Orbán | National Institute of Rheumatology and Physiotherapy | Orbanilon@Freemail.Hu |
| India: | |||
| Amita | Aggarwal | Sanjay Gandhi Postgraduate Institute, Lucknow | Aa.Amita@Gmail.Com |
| Sharath | Kumar | Columbia Asia Hospital, Bangalore | Arthritisdoctor.In@Gmail.Com |
| Sujata | Sawhney | Sir Ganga Ram Hospital, New Delhi | Drsujatasawhney@Gmail.Com |
| Israel: | |||
| Yonatan | Butbul Aviel | Meyer Children’s Hospital, Haifa | Yonatanbutbul@Gmail.Com |
| Yosef | Uziel | Meir Medical Center, Tel Aviv University | Uziely@Clalit.Org.Il |
| Italy: | |||
| Rolando | Cimaz | AOU Meyer Hospital Florence Italy | R.Cimaz@Meyer.It |
| Valeria | Gerloni | Azienda Ospedaliera Istituto Ortopedico Gaetano Pini, Milano | Valeria_Gerloni@Yahoo.It |
| Maria Cristina | Maggio | Università degli Studi di Palermo, Palermo | Sbenfratello@Libero.It |
| Serena | Pastore | IRCCS Burlo Garofolo, Pediatric Clinic | Serena.Pastore@Burlo.Trieste.It |
| Latvia: | |||
| Ingrida | Rumba-Rozenfelde | Children’s Hospital, University of Latvia | Rumba@Lu.Lv |
| Libya: | |||
| Soad | Hashad | Tripoli Children’s Hospital | Khaines@Hackensackumc.Org |
| Malaysia: | |||
| Sern Chin | Lim | Hospital Selayang | Sernchin@Gmail.Com |
| Mexico: | |||
| Carlos | Abud | Hospital Central Ignacio Morones Prieto | C_Abud@Hotmail.Com |
| Ruben | Burgos-Vargas | Hospital General de México | R.Burgos.Vargas@Gmail.Com |
| Roberto | Carreño-Manjarrez | Hospital Infantil De Mexico | Rcarrenom63@Gmail.Com |
| Ivonne | Cespedes | UMAE Hospital General Centro Medico Nacional “La Raza” | Ivonneadri@Yahoo.Com.Mx |
| Sandra | Enciso | Hospital Infantil de Mexico Federico Gomez | Sandiaenciso@Gmail.Com |
| Hayde | Hernandez-Huirache | Hospital Regional de Alta Especialidad del Bajio | Little_Hayde@Hotmail.Com |
| Jorge | Jaimes-Hernández | Centro Médico ISSEMYM, Toluca | Jorjaimes@Yahoo.Com |
| Rocio | Maldonado | Hospital Infantil de Mexico | Irama.Maldonado@Gmail.Com |
| Rogelio | Martinez | Hospital Infantil de México, Federico Gómez | Rexmtz@Hotmail.Com |
| Floricela | Olivera | Unidad de Especialidades Medicas, Sedena | Floricela_14@Hotmail.Com |
| Javier | Orozco | Universidad de Guadalajara, Guadalajara | jaorozcoa44@yahoo.com.mx |
| Ana Luisa | Rodriguez-Lozano | Instituto Nacional de Pediatria, Mexico City | Anarlozano@Yahoo.Com.Mx |
| Omar Ernesto | Rojas Pacheco | Cancun | Rojaspacheco@Gmail.Com |
| Araceli | Arellano | UMAE, Pediatría, CMNO, IMSS, Guadalajara | enanara@yahoo.com.mx |
| Luz Maria | Suárez Larios | Hospital infantil del Estado de Sonora | Marysuhil@Hotmail.Com |
| Alfonso Ragnar | Torres Jimenez | Hospital General Centro Medico Nacional La Raza, IMSS | Dr-Poncho@Hotmail.Com |
| Gabriel | Vega | Hospital México Americano, Guadalajara | Gvc81@Hotmail.Com |
| Manuel | Vera | Hospital Infantil de Especialidades del Estado de Chihuahua,Chihuahua | Drhectorvera@Hotmail.Com |
| Julia Verónica | Ramírez Miramontes | Instituto Mexicano del Seguro Social, Mexico City | Juliver_Rami05@Hotmail.Com |
| Ivon Karina | Ruíz | Hospital General de México, Durango | Voniruiz75@Yahoo.Com.Mx |
| New Zealand | |||
| Anthony | Concannon | Starship Children’s Health, Auckland City | Anthony.Concannon@Middlemore.Co.Nz |
| Jacqueline | Yan | Starship Children’s Health, Auckland City | Jyan@Adhb.Govt.Nz |
| Nicaragua: | |||
| Martha | Jarquin Jaime | Hospital Infantil MJR La Mascota, Managua | Jorjaimes@Yahoo.Com |
| Oman: | |||
| Safiya | Al Abrawi | Royal Hospital, Muscat | Safiassm@Yahoo.Com |
| Paraguay: | |||
| Cynthia | Vega | Hospital General Pediatrico Acosta Ñu, San Lorenzo | Cynvegab@Hotmail.Com |
| Jorge | Lopez-Benitez | La Costa Medical Center, San Lorenzo | Jmlopezbenitez@Gmail.Com |
| Zoilo | Morel | Universidad Nacional de Asunción, San Lorenzo | Zoiloma@Hotmail.Com |
| Peru: | |||
| Eliana | Gastañaga | Instituto Nacional de Salud del Niño, Lima | Fcuepaz@Infonegocio.Net.Pe |
| Tatiana | Guzmán | Hospital Rebagliati, Lima | Tmiraval@Hotmail.Com |
| Amparo | Iban͂ez | Instituto Nacional de Salud del Nin͂o, Lima | Amparoies@Yahoo.Com |
| Poland: | |||
| Violetta | Opoka-Winiarska | Medical University of Lublin | Viola.Winiarska@Gmail.Com |
| Lidia | Rutkowska-Sak | National Institute of Geriatrics, Reumatology & Rehabilitation, Warsaw | Lidia.Rutkowska-Sak@Spartanska.Pl |
| Elzbieta | Smolewska | Medical University, Department of Pediatric Rheumatology | E.Smolewska@Wp.Pl |
| Portugual: | |||
| Marta | Conde | Hospital de Dona Estefânia, Lisboa | Marta.C.Conde@Gmail.Com |
| Margarida | Guedes | Porto Hospital Center, Porto | Margguedes@Gmail.Com |
| Puerto Rico: | |||
| Enid | del Valle | San Jorge Children’s Hospital, San Juan | Enidjdelvalle@Gmail.Com |
| Elivette | Zambrana | HIMAHealth, San Juan | Ezambrana@Hotmail.Com |
| Romania: | |||
| Constantin | Ailioaie | II Paediatric Clinic, Children’s Emergency Hospital “St. Mary” | Laserail_Mail@Yahoo.Com |
| Calin | Lazar | Pediatric Clinic, no.1, University of Medicine and Pharmacy Iuliu Hatieganu Cluj-Napoca | Calinlazar2004@Yahoo.Com |
| Russian Federation: | |||
| Ekaterina | Alekseeva | Scientific Centre of Children’s Health,Russian Academy of Medical Sciences | Alekatya@Yandex.Ru |
| Vladimir | Keltsev | Samara Regional Clinical Hospital, Samara | Keltsev@Mail.Ru |
| Saudi Arabia: | |||
| Sulaiman | Al-Mayouf | King Faisal Specialist Hospital & Research Center, Riyadh | Mayouf@Kfshrc.Edu.Sa |
| Abdurhman | Asiri | Prince Sultan Military Medical City (PSMMC) | Asiri1000@Yahoo.Com |
| Wafaa | Sewairi | King Fahad National Guard Hospital, King Abdulaziz Medical City | Sewairiw@Ngha.Med.Sa |
| Serbia: | |||
| Gordana | Susic | Belgrade Institute of Rheumatology, Belgrade | Susic.Gordana@Gmail.Com |
| Gordana | Vijatov-Djuric | Institute for Child and Youth Health Care of Vojvodina, Belgrade | Vijatovg@Sbb.Rs |
| Singapore: | |||
| Elizabeth | Ang | National University Children’s Medical Institute, Singapore | rheum_kids@nuhs.edu.sg |
| Thaschawee | Arkachaisri | KK Women’s and Children’s Hospital and Duke-NUS Medical School | Thaschawee.Arkachaisri@Kkh.Com.Sg |
| Spain: | |||
| Alina | Boteanu | Hospital Universitario Ramón y Cajal, Madrid | al_boter@yahoo.com |
| Juan Carlos | Lopez-Robledillo | Hospital Infantil Universitario Niño Jesús, Madrid | Jclrobledillo@Gmail.Com |
| Marta | Medrano San Ildefonso | Hospital Miguel Servet Zaragoza, Zaqragoza | Mmedrano@Unizar.Es |
| Consuelo | Modesto | Hospital Vall D’Hebron, Barcelona | Cmodesto@Vhebron.Net |
| Calvo | Penades | Hospital Universitario Politecnico, La Fe | Calvo_Inm@Gva.Es |
| Sweden: | |||
| Anders | Fasth | Queen Silvia Children’s Hospital, Gothenburg | Anders.Fasth@Gu.Se |
| Jorge | Sotoca-Fernandez | Medicinkliniken Mälarsjukhuset, Sörmland | Yosoysoti@Yahoo.Es |
| Switzerland: | |||
| Isabel | Bolt | Kinderspital, Universität Zürich | Isabel.Bolt@Insel.Ch |
| Singapore: | |||
| Soamarat | Vilaiyuk | Faculty of Medicine Ramathibodi Hospital, Mahidol University | Soamarat21@Hotmail.Com |
| The Netherlands: | |||
| Sylvia | Kamphuis | Sophia Children’s Hospital, Rotterdam | S.Kamphuis@Erasmusmc.Nl |
| Dieneke | Schonenberg-Meinema | Emma’s Children Hospital, Academical Medical Center, Amsterdam | D.Schonenberg@Amc.Nl |
| The Philippines: | |||
| Leonia | Dans | University of the Philippines-Philippine General Hospital, Manila | Leonila.Dans@Gmail.Com |
| Karen Joy | Kimseng | Chong Hua Hospital and Cebu Institute of Medicine, Cebu City, Cebu | Yenkimseng@Gmail.Com |
| Turkey: | |||
| Seza | Ozen | Hacettepe University, Ankara | Sezaozen@Hacettepe.Edu.Tr |
| United Kingdom: | |||
| Eileen | Baildam | Alder Hey Children’s Hospital, Liverpool | Eileen.Baildam@Alderhey.Nhs.Uk |
| Clare | Pain | Alder Hey Children’s NHS Foundation Trust, Liverpool | clare.pain@alderhey.nhs.uk |
| Eslam | Al-Abadi | Birmingham Children’s Hospital, Birmingham | Eslam.al-abadi@bch.nhs.uk |
| Lampros | Fotis | Queens Medical Center, Nottingham | Labfotis@Hotmail.Com |
| Clarissa | Pilkington | The Hospital for Sick Children, London | clarissa.pilkington@gosh.nhs.uk |
| Uruguay: | |||
| Juan | Cameto | Centro Hospitalario Pereira Rossell, Montevideo. | Cametojuan@Gmail.Com |
| Rosario | Jurado | Centro Hospitalario Pereira Rossell, Montevideo. | Jurado@Femi.Com.Uy |
| Puerto RicoL | |||
| Ana | Quintero-Del Rio | Santurce, PR | Anaquintero8293@Msn.Com |
| United States of America | |||
| Bryce | Binstadt | Masonic Children’s Hospital, Minneapolis,MN | Binstadt@Umn.Edu |
| David D. | Sherry | Children’s Hospital of Philadelphia, PA | Sherry@Email.Chop.Edu |
| Robert | Sundel | Boston Children’s Hospital, Boston, MA | Robert.Sundel@Childrens.Harvard.Edu |
| Leslie | Abramson | University of Vermont Medical Center, Burlington, VT | Leslie.Abramson@Vtmednet.Org |
| Khalid | Abulaban | Helen Devos Children’s - Spectrum Health, Green Bay, MI | Khalid.Abulaban@Helendevoschildrens.Org |
| Cassyanne | Aquiar | Children’s Hospital of the King’s Daughters, Norfolk, VA | cassyanne.aquiar@chkd.org |
| Bita | Arabshahi | Inova Childrne’s Hospital, Fairfax,VA | Barabshahi@Psvcare.Org |
| Fatima | Barbar-Smiley | Nationwide Children’s Hospital, Columbus, OH | Fatima.Barbar-Smiley@Nationwidechildrens.Org |
| John | Bohnsack | Primary Childrens’ Hospital, University of Utah Health Care, Salt Lake City, UT | John.Bohnsack@Hsc.Utah.Edu |
| Alexis | Boneparth | Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ | Adb177@Rwjms.Rutgers.Edu |
| Amanda | Brown | Texas Children’s Hospital, Houston, TX | amanda.brown2@bcm.edu |
| Diane | Brown | Children’s Hospital of Los Angeles, CA | Debrown@Chla.Usc.Edu |
| Jon | Burnham | Children’s Hospital of Philadelphia, PA | Burnhams@Email.Chop.Edu |
| Elizabeth C. | Chalom | Childrens Hospital of New Jersey, Newark, NJ | Echalom@Barnabashealth.Org |
| Peter | Chira | Rady Children’s Hospital, Los Angeles, CA | Pchira@Ucsd.Edu |
| Dominic | Co | American Family Children’s Hospital, Madison, WI | Dco@Mcw.Edu |
| Colleen | Correll | Masonic Children’s Hospital, Minneapolis,MN | Corr0250@Umn.Edu |
| Randy | Cron | University of Alabama, Birmingham, AL | Rcron@Peds.Uab.Edu |
| Fatma | Dedeoglu | Boston Children’s Hospital, Boston, MA | Fatma.Dedeoglu@Childrens.Harvard.Edu |
| Marietta | Deguzman | Texas Children’s Hospital, Houston, TX | Mmdeguzm@Texaschildrens.Org |
| Anne | Eberhard | Cohen Children’s Medical Center, New Hyde Park, NY | Beberhard@Nshs.Edu |
| Kaleo | Ede | Phoenix Children’s Hospital, Phoenix, AZ | Kede@Phoenixchildrens.Com |
| Cuoghi | Edens | MetroHealth Medical Center, Cleveland, OH | Cuoghi.Edens@Uhhospitals.Org |
| Andrew | Eichenfield | New York-Presbyterian Morgan Stanley Children Hospital, New York, NY | Ahe2101@Columbia.Edu |
| Abraham | Gedalia | Children’s Hospital of New Orleans, New Orleans, LA | Agedal@Lsuhsc.Edu |
| Alexei | Grom | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | Alexi.Grom@Cchmc.Org |
| Kathleen | Haines | Hackensack University Medical Center, Hackensack, NJ | Margguedes@Gmail.Com |
| Michael | Henrickson | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | Soadhashad@Hotmail.Com |
| Christine | Hom | Maria Fareri Children’s Hospital, Valhalla, NY | Christine_Hom@Nymc.Edu |
| Jennifer | Huggins | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | jennifer.huggins@cchmc.org |
| Norman | Ilowite | The Children’s Hospital at Montefiore, New York, NY | Nilowite@Montefiore.Org |
| Rita | Jerath | CHOG, Augusta University, Augusta, GA | Rjerath@Gru.Edu |
| Jordan | Jones | Children’s Mercy Hospital, Kansas, MI | Jtjones@Cmh.Edu |
| Lawrence | Jung | Children’s National Medical Center, Washington, DC | Ljung@Cnmc.Org |
| Daniel | Kingsbury | Randall Children’s Hospital at Legacy Emanuel | Dkingsbu@Lhs.Org |
| Jamie | Lai | Palo Alto, California | Jamietlai@Gmail.Com |
| Daniel | Lovell | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | Daniel.Lovell@Cchmc.Org |
| Jay | Mehta | Children’s Hospital of Philadelphia,PA | Mehtaj@Chop.Edu |
| Terry | Moore | Cardinal Glennon Children’s Hospital, Saint Louis, MO | Mooretl@Slu.Edu |
| Lakshmi | Moorthy | Rutgers-Robert Wood Johnson University, New Brunswic, NJ | Moorthln@Rwjms.Rutgers.Edu |
| Kabita | Nanda | Seattle Children’s Hospital, Seattle, WA | Kabita.Nanda@Seattlechildrens.Org |
| James | Nocton | Children’s Hospital of Wisconsin, WI | Jnocton@Mcw.Edu |
| Judyann | Olson | Children’s Hospital of Wisconsin, WI | Jcolson@Mcw.Edu |
| Kathleen | O’neil | Riley’s Childen’s Hopital, Indianapolis, IN | Kmoneil@Iu.Edu |
| Karen | Onel | Hospital for Special Surgery, New York, NY | onelk@HSS.EDU |
| Amir | Orandi | St Louis Children’s Hospital, MI | Orandi_A@Kids.Wustl.Edu |
| Janet | Orrock | The Children’s Hospital at Montefiore, New York, NY | Jorrock@Montefiore.Org |
| Lynn | Punaro | Children’s Medical Center/Texas Scottish Rite Hospital, Dallas, TX | Marilynn.Punaro@Tsrh.Org |
| Egla | Rabinovich | Duke Children’s Hospital, Raleigh, NC | Egla.Rabinovich@Duke.Edu |
| Andreas | Reiff | Children’s Hospital Los Angeles, Los Angeles, CA | Areiff@Chla.Usc.Edu |
| Kelly | Rouster-Stevens | Emory University, Atlanta, GA | Krouste@Emory.Edu |
| Tamar | Rubinstein | The Children’s Hospital at Montefiore, New York, NY | Tamar.Rubinstein@Gmail.Com |
| Natasha | Ruth | Medical University of South Carolina, Charleston, SC | Ruthn@Musc.Edu |
| Lisabeth | Scalzi | Penn State Health Milton S. Hershey Medical Center, Hershey, PA | Lscalzi@Hmc.Psu.Edu |
| Ken | Schikler | Norton Children’s Hospital, Louisville, LY | Knschi01@Louisville.Edu |
| Kara | Schmidt | Norton Children’s Hospital, Louisville, LY | Kara.Schmidt@Louisville.Edu |
| Grant | Schulert | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | Grant.Schulert@Cchmc.Org |
| Bracha | Shaham | Children’s Hospital Los Angeles, Los Angeles, CA | bshaham@chla.usc.edu |
| Nora | Singer | MetroHealth Medical Center, Cleveland, OH | Nora.Singer@Case.Edu |
| Judith | Smith | American Family Children’s Hospital, Madison,WS | Jsmith27@Pediatrics.Wisc.Edu |
| Leonard | Stein | Levine Children’s Specialty Center, Chapel Hill, NC | Ldstein@Med.Unc.Edu |
| Sara | Stern | Primary Childrens’ Hospital, Salt Lake City, UT | Sara.Stern@Hsc.Utah.Edu |
| Anne | Stevens | Seattle Children’s Hospital, Seattle, WA | Anne.Stevens@Seattlechildrens.Org |
| Grant | Syverson | American Family Children’s Hospital, Madison, WS | Gsyverson@Medicine.Wisc.Edu |
| Melissa | Tesher | Comer’s Children’s Hospital, Chicago, IL | Mtesher@Peds.Bsd.Uchicago.Edu |
| Tracy | Ting | Cincinnati Children’s Hospital Medical Center, Cincinnati, OH | Tracy.Ting@Cchmc.Org |
| Heather | Tory | Connecticut Children’s Specialty Group, Hartford, CT | Htory@connecticutchildrens.org |
| Patricia | Vega-Fernandez | Children Hospital of Atlanta, Atlanta, GA | Pvegafe@Emory.Edu |
| Richard | Vehe | Masonic Children’s Hospital, Minneapolis,MN | Vehex001@Umn.Edu |
| Linda | Wagner-Weiner | Comer’s Children’s Hospital, Chicago, IL | lww@uchicago.edu |
| Heather | Walters | Cohen Children’s Medical Center, New Hyde Park, NY | Hwalters@Nshs.Edu |
| Andrew | White | St Louis Children’s Hospital, St Louis, MS | White@Kids.Wustl.Edu |
| Venezuela: | |||
| Eillen | Macias | Hospital Vargas de Caracas, Caracas | Macias.Eillen@Gmail.Com |
| Irama | Maldonado | Complejo Hospitalario Universitario Ruiz y Paez | Rmaldonadov64@Hotmail.Com |
Footnotes
DR. SCOTT E WENDERFER (Orcid ID : 0000-0002-8991-8277)
Contributors relevant to this work are:
Important contributions to this work were provided by the physicians providing their expertise when rating the patient profiles. They are summarized in Appendix 2
LITERATURE
- 1.Silva CA, Avcin T, Brunner HI. Taxonomy for systemic lupus erythematosus with onset before adulthood. Arthritis Care & Research. 2012;64(12):1787–93. doi: 10.1002/acr.21757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Hochberg MC. Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum. 1997;40(9):1725. doi: 10.1002/art.1780400928. [DOI] [PubMed] [Google Scholar]
- 3.Brunner HI, Gladman DD, Ibanez D, Urowitz MD, Silverman ED. Difference in disease features between childhood-onset and adult-onset systemic lupus erythematosus. Arthritis Rheum. 2008;58(2):556–62. doi: 10.1002/art.23204. [DOI] [PubMed] [Google Scholar]
- 4.Hiraki LT, Benseler SM, Tyrrell PN, Hebert D, Harvey E, Silverman ED. Clinical and laboratory characteristics and long-term outcome of pediatric systemic lupus erythematosus: a longitudinal study. J Pediatr. 2008;152(4):550–6. doi: 10.1016/j.jpeds.2007.09.019. [DOI] [PubMed] [Google Scholar]
- 5.Brunner HI, Klein-Gitelman MS, Higgins GC, Lapidus SK, Levy DM, Eberhard A, et al. Toward the development of criteria for global flares in juvenile systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2010;62(6):811–20. doi: 10.1002/acr.20126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Brunner HI, Feldman BM, Bombardier C, Silverman ED. Sensitivity of the Systemic Lupus Erythematosus Disease Activity Index, British Isles Lupus Assessment Group Index, and Systemic Lupus Activity Measure in the evaluation of clinical change in childhood-onset systemic lupus erythematosus. Arthritis Rheum. 1999;42(7):1354–60. doi: 10.1002/1529-0131(199907)42:7<1354::AID-ANR8>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
- 7.Gladman DD, Ibanez D, Urowitz MB. Systemic lupus erythematosus disease activity index 2000. J Rheumatol. 2002;29(2):288–91. [PubMed] [Google Scholar]
- 8.Isenberg DA, Rahman A, Allen E, Farewell V, Akil M, Bruce IN, et al. BILAG 2004. Development and initial validation of an updated version of the British Isles Lupus Assessment Group’s disease activity index for patients with systemic lupus erythematosus. Rheumatology (Oxford) 2005;44(7):902–6. doi: 10.1093/rheumatology/keh624. [DOI] [PubMed] [Google Scholar]
- 9.Wallace CA, Ravelli A, Huang B, Giannini EH. Preliminary validation of clinical remission criteria using the OMERACT filter for select categories of juvenile idiopathic arthritis. J Rheumatol. 2006;33(4):789–95. [PubMed] [Google Scholar]
- 10.Ruperto N, Ravelli A, Oliveira S, Alessio M, Mihaylova D, Pasic S, et al. The Pediatric Rheumatology International Trials Organization/American College of Rheumatology provisional criteria for the evaluation of response to therapy in juvenile systemic lupus erythematosus: prospective validation of the definition of improvement. Arthritis Rheum. 2006;55(3):355–63. doi: 10.1002/art.22002. [DOI] [PubMed] [Google Scholar]
- 11.Giannini EH, Ruperto N, Ravelli A, Lovell DJ, Felson DT, Martini A. Preliminary definition of improvement in juvenile arthritis. Arthritis Rheum. 1997;40(7):1202–9. doi: 10.1002/1529-0131(199707)40:7<1202::AID-ART3>3.0.CO;2-R. [DOI] [PubMed] [Google Scholar]
- 12.Singh JA, Solomon DH, Dougados M, Felson D, Hawker G, Katz P, et al. Development of classification and response criteria for rheumatic diseases. Arthritis Rheum. 2006;55(3):348–52. doi: 10.1002/art.22003. [DOI] [PubMed] [Google Scholar]
- 13.Brunner H, Baker A, Cedeno A, Huggins JL, Sagcal-Gironella ACP, Ying J, et al. The Pediatric Automated Neuropsychological Assessment Metrics Has Reproducibility and Criterion Validity in Childhood-Onset Lupus. Arthritis and Rheumatism. 2011;63(10):S780–S1. [Google Scholar]
- 14.Brunner HI, Mina R, Pilkington C, Beresford MW, Reiff A, Levy DM, et al. Preliminary Criteria for Global Flares in Childhood-Onset Systemic Lupus Erythematosus. Arthritis Care & Research. 2011;63(9):1213–23. doi: 10.1002/acr.20507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Brunner HI, Higgins GC, Klein-Gitelman MS, Lapidus SK, Olson JC, Onel K, et al. Minimal clinically important differences of disease activity indices in childhood-onset systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2010;62(7):950–9. doi: 10.1002/acr.20154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Boers M, Kirwan JR, Gossec L, Conaghan PG, D’Agostino MA, Bingham CO, 3rd, et al. How to choose core outcome measurement sets for clinical trials: OMERACT 11 approves filter 2.0. J Rheumatol. 2014;41(5):1025–30. doi: 10.3899/jrheum.131314. [DOI] [PubMed] [Google Scholar]
- 17.Lassere MN. A users guide to measurement in medicine. Osteoarthritis Cartilage. 2006;14(Suppl A):A10–3. doi: 10.1016/j.joca.2006.02.021. [DOI] [PubMed] [Google Scholar]
- 18.Brunner HI, Mina R, Pilkington C, Beresford MW, Reiff A, Levy DM, et al. Preliminary criteria for global flares in childhood-onset systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2011;63(9):1213–23. doi: 10.1002/acr.20507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mina R, Harris JG, Klein‐Gitelman MS, Appenzeller S, Centeville M, Eskra D, et al. Initial Benchmarking of the Quality of Medical Care in Childhood‐Onset Systemic Lupus Erythematosus. Arthritis care & research. 2016;68(2):179–86. doi: 10.1002/acr.22666. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Ruperto N, Ravelli A, Pistorio A, Ferriani V, Calvo I, Ganser G, et al. The provisional Paediatric Rheumatology International Trials Organisation/American College of Rheumatology/European League Against Rheumatism Disease activity core set for the evaluation of response to therapy in juvenile dermatomyositis: a prospective validation study. Arthritis Rheum. 2008;59(1):4–13. doi: 10.1002/art.23248. [DOI] [PubMed] [Google Scholar]
- 21.Watson L, Leone V, Pilkington C, Tullus K, Rangaraj S, McDonagh JE, et al. Disease activity, severity, and damage in the UK Juvenile-Onset Systemic Lupus Erythematosus Cohort. Arthritis Rheum. 2012;64(7):2356–65. doi: 10.1002/art.34410. [DOI] [PubMed] [Google Scholar]
- 22.Schanberg L, Sandborg C, Barnhart H, Ardoin S, Yow E, Evans G, et al. Use of atorvastatin in systemic lupus erythematosus in children and adolescents. Arthritis & Rheumatism. 2012;64(1):285–96. doi: 10.1002/art.30645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Little RJ. Pattern-mixture models for multivariate incomplete data. Journal of the American Statistical Association. 1993;88(421):125–34. [Google Scholar]
- 24.Schafer JL. Multiple imputation: a primer. Statistical methods in medical research. 1999;8(1):3–15. doi: 10.1177/096228029900800102. [DOI] [PubMed] [Google Scholar]
- 25.Rubin DB. multiple imputation for nonresponse in surveys. Hoboken, NJ: Wiley-Interscience; 2004. [Google Scholar]
- 26.Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–8. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
- 27.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 28.Caldwell JR, Furst DE, Smith AL, Clark JA, Bonebrake RA, Gruhn WB, et al. Flare during drug withdrawal as a method to support efficacy in rheumatoid arthritis: amiprilose hydrochloride as an example in a double blind, randomized study. J Rheumatol. 1998;25(1):30–5. [PubMed] [Google Scholar]
- 29.Giannini EH, Ruperto N, Ravelli A, Lovell DJ, Felson DT, Martini A. Preliminary definition of improvement in juvenile arthritis. Arthritis & Rheumatology. 1997;40(7):1202–9. doi: 10.1002/1529-0131(199707)40:7<1202::AID-ART3>3.0.CO;2-R. [DOI] [PubMed] [Google Scholar]
- 30.Balsa A, Carmona L, Gonzalez-Alvaro I, Belmonte MA, Tena X, Sanmarti R. Value of Disease Activity Score 28 (DAS28) and DAS28-3 compared to American College of Rheumatology-defined remission in rheumatoid arthritis. J Rheumatol. 2004;31(1):40–6. [PubMed] [Google Scholar]
- 31.Albrecht J, Taylor L, Berlin JA, Dulay S, Ang G, Fakharzadeh S, et al. The CLASI (Cutaneous Lupus Erythematosus Disease Area and Severity Index): an outcome instrument for cutaneous lupus erythematosus. Journal of Investigative Dermatology. 2005;125(5):889–94. doi: 10.1111/j.0022-202X.2005.23889.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Mina R, von Scheven E, Ardoin SP, Eberhard BA, Punaro M, Ilowite N, et al. Consensus treatment plans for induction therapy of newly diagnosed proliferative lupus nephritis in juvenile systemic lupus erythematosus. Arthritis Care Res (Hoboken) 2012;64(3):375–83. doi: 10.1002/acr.21558. [DOI] [PMC free article] [PubMed] [Google Scholar]