Preliminary test of the LFA rapid evaluation of activity in lupus (LFA-REAL): an efficient outcome measure correlates with validated instruments

Abstract

Objective

Current disease activity measures for systemic lupus erythematosus (SLE) are difficult to score or interpret and problematic for use in clinical practice. Lupus Foundation of America (LFA)-Rapid Evaluation of Activity in Lupus (REAL) is a pilot application composed of anchored visual analogue scores (0–100 mm each) for each organ affected by lupus. This study evaluated the use of LFA-REAL in capturing SLE disease activity.

Methods

In a preliminary test of LFA-REAL, this simplified, organ-based system was compared with the most widely used outcome measures in clinical trials, the British Isles Lupus Assessment Group 2004 Index (BILAG), the SLE Disease Activity Index (SLEDAI) and the Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA) SLEDAI Physician's Global Assessment (SS-PGA). The level of agreement was analysed using Spearman rank correlations.

Results

91 patients with SLE with mild to severe disease activity were evaluated, their median SLEDAI score was 4.0 (range 0–28) and BILAG score 8.0 (0–32). The median SS-PGA was 38 mm (4–92) versus the total REAL 50 mm (0–268), which expands in range by additive organ scores. Thirty-three patients had moderate to severe disease activity (≥1.5 on SS-PGA landmarks). The median SS-PGA score of this group was 66 mm (50–92) versus median REAL score of 100 mm (59–268), confirming ability to detect a wider distribution of scores at higher disease activity. Total REAL correlated with SLEDAI, BILAG and SS-PGA (correlation coefficient=0.816, 0.933 and 0.903, respectively; p<0.001 for all). Individual LFA-REAL organ scores for musculoskeletal and mucocutaneous also correlated with corresponding BILAG domain scores (correlation coefficient=0.925 and 0.934, p<0.001).

Conclusions

In this preliminary exercise, there were strong correlations between LFA-REAL and validated lupus disease activity indices. Further development may be valuable for consistent scoring in clinical trials, grading optimal assessment of change in disease activity and reliable monitoring of patients in practice.

Key messages.

• LFA-REAL is a simplified SLE outcome measure that highly correlates with validated SLE disease activity instruments.

• LFA-REAL is being developed for use in clinical trials and in routine care.

Introduction

Systemic lupus erythematosus (SLE) is a heterogeneous, waxing and waning, multisystem autoimmune disease.¹ The complexity and clinical unpredictability of this illness challenge the assessment of disease activity in populations over time.² Multiple disease monitoring instruments have been developed; the most widely used in multicentre trials today are the SLE Disease Activity Index (SLEDAI), the British Isles Lupus Assessment Group Index (BILAG 2004) and the Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA) SLEDAI Physician's Global Assessment (SS-PGA).^3–7 Derived from clinician consensus, the SLEDAI includes 24 common manifestations that each receives a numerical point reflecting the usual severity of the descriptor as it occurs in most patients.

The BILAG 2004 Index includes nine organ systems and 97 descriptors divided among these organs with each descriptor scored as having been ‘not present’, ‘improving’, ‘same’, ‘worse’ or ‘new or recurrence’ during the past month. A complex algorithm considers all items scored within a given organ and assigns the organ to have no activity, mild disease, moderate disease or severe disease.

The SS-PGA was originally a 3-inch scale but has been normalised to a 100 mm scale in many clinical trials for physicians to assess patient's overall disease activity in a 0–3 range with landmarks at one for mild and two for moderate disease activity.^{8 9}

A number of pitfalls exist in these measures. Each descriptor of the SLEDAI is scored if the minimal definition is met and receives the same numerical weight regardless of its severity. The SLEDAI was not designed to detect worsening over time, or anything less than virtually complete resolution. Two corrections for those issues have been devised in the SELENA SLEDAI Flare Index and the SRI-50 (improvement index) but neither addresses the fact that patients with severe disease often cannot be differentiated from those with mild disease.^{10 11} However, the BILAG does differentiate mild, moderate and severe activity and is more sensitive to changes in a large range of indicators. However, it is complicated to learn and is not accepted as a useful or efficient measure for routine clinical care.^{8 12} In addition, another pitfall in scoring the BILAG is that if a person met two descriptors for a given score within one organ, only one score can be assigned for that organ. Thus, a person with severe rash and severe vasculitis only receives one severe ‘A’ score, which would be indistinguishable from having only one of these manifestations. The SS-PGA is simple and intuitive, but its compression of a broad spectrum of moderate to severe disease severity into a small region of the scale can be problematic in accurately evaluating disease progress, particularly when multiple organs are involved. Finally, despite incremental improvements in training with each new multicenter study, these outcome measures are still frequently misunderstood and scored incorrectly, even by experienced clinical trialists.

The current study provides a preliminary assessment of an efficient but scalable SLE disease activity measure, the Lupus Foundation of America-Rapid Evaluation of Activity in Lupus (LFA-REAL), for potential use both as a reliable outcome measure in clinical trials and in real world clinical practices. We believe that this system can be learnt by a clinician skilled in the care of lupus patients within minutes, and in practice it can be scored rapidly for most patients. Since it is constructed as an expanded version of PGA (based on completing a separate SS-PGA for each active organ system), it is designed to be sensitive to change and allow for precise and accurate measurement of disease severity and treatment progress. Clinically meaningful change and the optimal sensitivity (versus discriminatory capacity) for use in clinical trials have never been derived from patient data. This instrument is constructed so that exercise can be performed. Additionally, a tool that can be used in both clinical trials and practice would allow practicing clinicians to more easily interpret, evaluate and apply the findings from clinical trials.

Methods

Description of instrument

Two physicians (ADA and JTM) established the prototype for the LFA-REAL through extensive literature review and discussions with lupus specialists whose comments were incorporated in the current version of the instrument.

The LFA-REAL (figure 1) has six or more anchored Visual Analogue Scales (VASs; 0–100 mm each): the first six assess the most commonly affected organs (mucocutaneous, musculoskeletal, cardiorespiratory, neuropsychiatric, renal and haematological); ‘other’ scales can be added to record features that do not fit the six categories above or to separately score each descriptor in organs with two or more manifestations (eg, discoid rash and cutaneous vasculitis or pleurisy and pericarditis). It is a rapid system because only active disease requires scoring and it is unlikely to find more than three or four active manifestations at once in most patients. The landmarks are identical to the SS-PGA: 0=none, 1=mild, 2=moderate and 3=severe (figure 1). An anchor at 1.5 was added to serve as a potential cut-off for the initiation of immunosupressants. Each investigator checked the length of the instrument before use. Copies of the instrument with scales not equalled to 100 mm were discarded. Each scale was measured with a study-specific 10 cm ruler. In every case, when used in the study, the instrument was scored in <60 s. However, this needs to be confirmed with other users.

Figure 1.

Lupus Foundation of America-Rapid Evaluation of Activity in Lupus (LFA-REAL) comprised seven anchored Visual Analogue Scales (0–100 mm each) and can describe both organ-specific and overall disease progress. SLEDAI, Systemic Lupus Erythematosus Disease Activity Index.

Study description

Ninety-one consecutive patients with SLE were cross-sectionally evaluated during routine clinical visits with their physicians (ADA or JTM), who are experienced in scoring the SLEDAI, BILAG and SS-PGA. Hybrid SLEDAI, a version of the SLEDAI, was used for this study. All patients met at least four of the 11 1997 American College of Rheumatology revised criteria for SLE.^{13 14} This study was approved by the Institutional Review Board of New York University School of Medicine (NYU) and Oklahoma Medical Research Foundation (OMRF). All patients provided informed consent. Information on demographics and current medications were collected. Laboratory assessments required to score the disease activity instruments were also recorded, including complete blood count with differential, urinalysis with microscopic examination, C3, C4, anti-double stranded DNA and chemistry panel. Protein/creatinine ratio was obtained for patients with known or suspected renal involvement.

Statistical analysis

Descriptive analysis was performed for demographic and medication data. For calculating the global BILAG score, BILAG A was graded at 12 points, BILAG B was eight points and BILAG C was one point.⁷ Statistical analyses were performed using SigmaStat software. Spearman rank correlation coefficients (SRCC) between LFA-REAL scores and scores of the validated disease activity measures were evaluated, defining a low, moderate and high correlation to have a coefficient of 0.10–0.29, 0.30–0.49 and ≥0.50, respectively.¹⁵

Results

Demographics and medications

Ninety-one patients participated in the study. Forty-seven patients were enrolled at the NYU site and 44 at the OMRF site. The mean age was 42.1 years. Of these patients, 86 were women; 54% were Caucasian, 31% were of African descent, 14% were Asian and 24% were Hispanic. Seventy-seven per cent of the patients were taking anti-malarials, 58% were taking immune suppressants and 44% were taking prednisone at the time of the study.

Disease activity comparisons between instruments

Median disease activity scores are compared in table 1. As expected, the LFA-REAL had a much wider range of scores than the BILAG, SLEDAI or PGA, and the range was increased at higher levels of disease activity, consistent with a wider potential to detect differences between patients.

Table 1.

Disease activity of participating patients

Instrument scores	Median (range)
SLEDAI score	4.0 (0–28)
BILAG 2004 score	8.0 (0–32)
PGA (mm)	38 (4–92)
LFA-REAL total score (sum)	50 (0–268)
33 patients ≥1.5 on SLEDAI landmarks
PGA (mm)	66 (50–92)
LFA-REAL total clinician score	100 (59–268)

BILAG, British Isles Lupus Assessment Group; LFA-REAL, Lupus Foundation of America-Rapid Evaluation of Activity in Lupus; PGA, Physician's Global Assessment; SLEDAI, Systemic Lupus Erythematosus Disease Activity Index.

The total REAL score (sum of each organ-based SS-PGA) was correlated with the SS-PGA, SLEDAI and BILAG scores, with SRCC values of 0.903, 0.816 and 0.933, respectively (p<0.001 for all analyses) (figures 2–4). Additionally, individual LFA-REAL scores for the most commonly involved organs (musculoskeletal and mucocutaneous) correlated well with the corresponding BILAG domain scores (coefficients=0.925 and 0.934, respectively; p<0.001; figures 5 and 6).

Figure 2.

British Isles Lupus Assessment Group (BILAG) vs Rapid Evaluation of Activity in Lupus (REAL). Spearman rank correlation coefficient=0.933 (p<0.001).

Figure 3.

Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) vs Rapid Evaluation of Activity in Lupus (REAL). Spearman rank correlation coefficient=0.816 (p<0.001).

Figure 4.

Physician's Global Assessment (PGA) vs Rapid Evaluation of Activity in Lupus (REAL). Spearman rank correlation coefficient=0.903 (p<0.001).

Figure 5.

British Isles Lupus Assessment Group (BILAG, musculoskeletal analogue) vs Rapid Evaluation of Activity in Lupus (REAL). Spearman rank correlation coefficient=0.925 (p<0.001).

Figure 6.

British Isles Lupus Assessment Group (BILAG, mucocutaneous analogue) vs Rapid Evaluation of Activity in Lupus (REAL). Spearman rank correlation coefficient=0.934 (p<0.001).

Discussion

The LFA-REAL system is in development to include both a clinician assessment and a similarly designed patient reported assessment tool (currently being vetted through patient focus groups). The project reported here was a preliminary evaluation of the clinician measure. This instrument includes seven landmarked VASs for individual organs/descriptors affected by SLE with the potential for adding additional scales as needed when there is more than one symptom in a given organ or a rare manifestation occurs that is not covered in the described scales. Since most patients present with at most two or three manifestations at a time, we believe that the instrument can be completed rapidly, with all other possible descriptors defaulting to zero. Precise timing evaluation remains to be performed in a wider group of physicians. Formal validation and modification of this instrument are part of an extensive, multicentre development programme that is currently ongoing.

A weakness of this preliminary test is that both clinicians involved are skilled in using the more complex disease activity instruments, thus their knowledge of those outcome measures could bias the scoring on the LFA-REAL. Additionally, we acknowledge that scoring of multiple instruments by the same well-trained evaluators can introduce bias and plan to address this in the next stage of development. The need of guidelines or limitations to adding more scales will also be explored and discussed among lupus specialist. A second preliminary study is currently underway at five clinical sites, comparing the results completed by a clinician skilled in SLEDAI, BILAG and SS-PGA with those completed by a clinician inexperienced in the aforementioned instruments, both of whom assessed the same patient at the same time. On the basis of the results for both studies, as well as feedback from the investigators, further modifications of the instrument and/or the instructions will be considered. A patient outcome measure will also be incorporated into the complete LFA-REAL system. A pilot study will then be performed to compare the LFA-REAL clinician and patient measures. Final changes to the instrument will be discussed in a community-wide forum and followed by formal validation studies.

The preliminary LFA-REAL global and organ-specific scores evaluated in the present study showed strong statistically significant correlations with the SLEDAI, BILAG and SS-PGA scores. The broader score range of the LFA-REAL suggests that it may have a far greater potential to differentiate grades of disease activity than the SLEDAI, BILAG and SS-PGA. Sensitivity to change may be increased; however, this is not always the optimal way to distinguish between an effective treatment and placebo. However, given the scalable construction of this instrument, both discriminatory capacity and clinically significant change can be optimised, derived from real-world data. Landmarks for clinically significant change have only been previously derived for lupus studies through consensus meetings or clinical studies of instruments with prespecified landmarks.¹⁶

With this efficient tool in everyday care, we believe that clinicians who are experienced in the clinical care of lupus patients could be trained in minutes and render a sophisticated opinion rapidly without having to master the complex definitions and rules of the BILAG and SLEDAI. While the SS-PGA can be learnt and scored quickly, a weakness of the SS-PGA is that the assessments of all manifestations are compressed into a single linear scale, making it ineffective in differentiating disease activity in individual organs. The restricted landmark-dependent scaling of this instrument can also be problematic: a change from no disease to moderate disease activity takes up half the scale; the range of moderate disease activity can be represented at most by a third of the scale; severe disease activity is squeezed into a small region at the top of the scale. This means that major changes reflecting more active disease have significantly less room to show gradations than changes in mild disease activity. Although the current LFA-REAL system does not change this progressively compressed scaling, it does allow separate measures for each organ manifestation, and the data here confirm that this increased the range of discrimination in patients with a greater level of active disease activity and more than one symptom or manifestation. Furthermore, unlike the SS-PGA, organ-specific scoring allows different manifestations and the changes in their severity over time to be examined and compared separately on the LFA-REAL.

It is planned for the LFA-REAL to be developed into a versatile mobile application that can increase speed and accuracy of assessment by observing or even relying on previous disease activity from each past visit when a new form is to be scored. Similar to the SS-PGA, the instruction will be to compare disease activity to the previous visit and move the marker left, right or not at all, depending on the comparison of current disease with previous disease. We hope that LFA-REAL will be used in all clinical settings, providing accessible data over time and improving clinical assessments and clinical care.