Abstract
Objectives:
We sought to develop and test an interviewer-administered measure of damage in SLE, the Brief Index of Lupus Damage (BILD), for use in epidemiological studies in which administration of the SLICC/ACR Damage Index (SDI) by trained physicians is not possible or feasible. In addition, we compared the BILD to another recently developed patient-reported measure, the Lupus Damage Index Questionnaire (LDIQ), which was designed as a written survey.
Methods:
A sample of 81 patients from two university-affiliated SLE clinics was used to test the criterion validity of the BILD and the LDIQ. A second sample, the Lupus Outcomes Study (LOS, n=728) was used to ascertain the construct validity of the BILD.
Results:
We found good agreement between most BILD items and corresponding SDI items, and moderately high overall Spearman rank correlations for SDI with BILD (0.64 and with LDIQ (0.54). BILD scores were higher among older individuals, those with longer disease duration, and those with higher mean disease activity in the preceding four years. In addition, higher BILD scores were associated with poorer self-rated health and functional status, greater unemployment and work disability, and increased health care utilization.
Conclusions:
We developed and performed a preliminary validation study demonstrating content, criterion and construct validity of a new practical patient-reported instrument of SLE disease damage. Although further studies are needed to examine reliability and to document psychometric properties in other populations, the BILD appears to represent a promising tool for studies of SLE outside the clinical setting.
As survival in patients with systemic lupus erythematosus (SLE) continues to improve, measuring outcomes beyond mortality has become a focus of epidemiologic research. In addition to disease activity, the concept of cumulative damage has emerged as an important outcome in SLE. Damage predicts not only mortality in SLE (1-5), but also a wide range of other outcomes, such as physical function (6, 7), health care utilization (8), and disability (9, 10).
Studies examining disease damage in SLE have traditionally relied on a validated physician-assessed measure, the Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index (SDI) (11). The SDI has been used widely over the last decade, with studies supporting its criterion, discriminant, and construct validity (1-5, 11, 12), as well as its reliability (13). Because the SDI requires a trained physician to complete a 41-item questionnaire, it has largely been used in research conducted in centers with resources and expertise to study SLE. In recent years, several large community-based studies in SLE using alternative data collection strategies, such as surveys administered by mail or by telephone, have been launched (14, 15). These studies have attempted to broaden clinical research in SLE to include individuals cared for outside of specialty centers. In doing so, a need has arisen to expand the tools available to measure important SLE-related outcomes to include patient-reported measures. A patient-reported measure of SLE disease activity, the Systemic Lupus Activity Questionnaire (SLAQ), has been developed and initially validated for this purpose (16, 17).
Similarly, researchers have recently developed a patient-reported proxy for the SDI, the Lupus Damage Index Questionnaire (LDIQ) (18). The questionnaire has been tested for criterion, content, and construct validity. An international examination of criterion validity in French, Spanish and Portuguese has also been completed (19). The LDIQ has 56 questions to assess all of the original SDI items and was designed for administration as a written survey. In a concurrent effort, we developed and tested a shorter patient-reported proxy for the SDI meant for interviewer administration in-person or on the telephone. The aims of the current study were threefold: 1) To develop and test the criterion validity of a new patient-reported damage index, the Brief Index of Lupus Damage (BILD); 2) To examine the BILD’s construct validity in a large, observational cohort of individuals with SLE; and 3) To compare the criterion validity of the BILD to the LDIQ, an instrument that was undergoing validation testing at the time the BILD development was initiated.
Methods
Study populations
There were two sources of data for the study. One sample included 81 patients from two university-affiliated SLE clinics and was used to test the criterion validity of the BILD against the SDI. In addition, the clinic-based sample was used to test criterion validity of the newly developed LDIQ. All continuing patients seen between February and September 2009 at one clinic or between December 2010 and February 2011 at the other with a diagnosis of SLE for at least one year were eligible for this study.
Recruitment for the clinic-based study took place in the University of California, San Francisco (UCSF) lupus center and at the San Francisco General Hospital lupus clinic, which is staffed by UCSF physicians. Purposive sampling was used to recruit study participants. Patients were initially approached before their regular SLE clinic appointments about participating in a short study of SLE manifestations. Those responding affirmatively were given the BILD questionnaire over the telephone at home prior to their next clinic appointment. Basic demographic questions were included in the interviews, which averaged 10 minutes. Upon returning to the clinic, patients completed the LDIQ in the waiting room prior to their appointment. During that appointment, their rheumatologist completed the SDI. This protocol was designed to ensure that physician queries regarding SDI items occurred last, and therefore did not influence patient responses on either the BILD or the LDIQ. The mean time between the telephone interview and the clinic appointment was four months. Of 109 patients approached, eight declined to participate, eleven could not be reached by telephone, and nine did not return to the clinic before the end of the data collection period. Thus, 81 (74%) patients completed the study. Seven patients were unable to complete the LDIQ, primarily due to time constraints in the clinic.
A sample size calculation was performed to determine the minimum necessary sample size to assess BILD’s criterion validity. A 0.05 one-sided Fisher’s z test of the null hypothesis that the Spearman correlation coefficient rho=0 was estimated to have 80% power to detect an alternative rho of 0.28 when the sample size was 80; moreover with n=80 power was estimated at 82% to detect an alternative rho of 0.50 versus the null of rho=0.25.
The second data source was the Lupus Outcomes Study (LOS), an ongoing cohort of individuals with SLE, interviewed annually by telephone. Recruitment for the LOS took place in several settings, including university-based rheumatology clinics (25%), community rheumatology offices (11%), and non-clinical sources, including patient support groups and conferences (26%), and other forms of media (38%). All patients have a diagnosis of SLE from a physician, confirmed by a formal review of the medical record to document American College of Rheumatology criteria for SLE. Details of the LOS methodology have been published previously (14). The LOS (N=728) was used to test the construct validity of the BILD. The BILD items were included as part of the fifth annual wave of the LOS, which also included validated self-report measures of disease activity, general health status, employment status, work disability, health care utilization, and demographics. Eleven LOS participants who were also part of the clinic sample were excluded, leaving 717 for the present independent analysis.
Development of the BILD
The BILD was designed for administration by telephone interview as part of a longer survey. Investigators modified the existing SDI items to be comprehensible to a lay respondent. The goal was not to replicate the SDI item for item, but to develop a reasonable proxy measure that would distinguish between greater and lesser degrees of SLE damage. Therefore, not all items in the SDI were included, because the investigators deemed the manifestation either too rare to be likely to contribute meaningfully to the score (e.g., shrinking lung), or not likely to be interpreted with enough specificity to capture the concept of damage (e.g. alopecia). The initial set of questions was reviewed by three patients with SLE for acceptability, feasibility, and understanding; comments were solicited on the clarity and rationale of the directions, the meaning of the items, and the appropriateness of the response choices. Based on this feedback, several questions were revised for clarity. Next, the instrument was included in the LOS interview. During the first month of the survey wave, interviewers recorded all questions from 77 respondents regarding the BILD items. These questions were evaluated by study rheumatologists, which resulted in several revisions in item prompts and wording. The resulting BILD instrument contained 28 questions that captured information on 26 of the original SDI items (Appendix A). The final BILD survey was then administered to the entire LOS sample.
When the BILD was administered to the larger LOS sample, interviewers were instructed to provide clarifications on items if respondents were unclear on their meaning using scripted comments (parenthetical explanations that appear on the instrument). Occasionally, respondents had additional questions that were recorded as free text notes. A rheumatologist adjudicated these notes when necessary.
Study protocols were approved by the UC San Francisco Committee on Human Research. All participants gave their informed consent to be part of the study.
Statistical Analysis
Clinical sample analysis.
In the clinic sample, we compared the BILD and LDIQ item responses to the corresponding SDI responses. We calculated the item-by-item percent observed agreement (po) with the SDI for each of the two proxy measures rather than a kappa statistic, given the low prevalence of individual items. The kappa coefficient is significantly influenced by the prevalence of attributes and its magnitude is difficult to interpret meaningfully if attributes are either very common or very rare, resulting in the so-called kappa paradox with high observed agreement but low kappa (20). The prevalence-adjusted bias-adjusted kappa (PABAK = 2po − 1) has been proposed as a better measure of agreement than kappa when prevalence varies or when the prevalence of each method or instrument differs (21). Like kappa, a PABAK value of −1 indicates perfect disagreement, 0 indicates no agreement, while 1 indicates perfect agreement.
We also compared the distributions of the overall SDI, LDIQ, and BILD scores, calculating the Spearman rank correlation coefficients (rs) for both proxy measures with the SDI.
LOS analysis.
The LOS was used to assess the construct validity of the BILD and to quantify its acceptability. Because the BILD does not have a normal distribution, we divided the scores from the LOS sample into quartiles to examine its correspondence with demographic, SLE status, general health, and health care utilization measures found in literature to relate to disease damage measured by the SDI. Sociodemographic measures included age, gender, race/ethnicity (nonwhite vs. white), education (high school or less vs. some college education or more), household income (at or below 125% of the Federal Poverty Threshold), and employment status. SLE measures included disease duration and the Systemic Lupus Activity Questionnaire (SLAQ) (16), averaged over four prior interviews. General health status measures included global health (categorized as excellent/very good/good vs. fair/poor), the SF-36 physical and mental component scores, and work disability status. Health care utilization measures included the annual mean number of outpatient medical visits for SLE over the first five years of the study, as well as the total number of hospitalizations during that time. Distributions of categorical measures were compared across quartiles of BILD using 1 degree of freedom chi-square trend tests. Continuous measures were compared using ANOVA F-tests. Finally, to explore the independent association of the sociodemographic measures with BILD, we modeled the BILD score (either top quartile versus not, bottom quartile versus not, and raw score) as a function of age, disease duration, gender, race/ethnicity, and education, using logistic regression.
Results
Both samples were comprised mainly of women and were fairly well-educated (Table 1). The clinic sample, however, was younger than the LOS survey sample, had been diagnosed more recently, and had a larger proportion of racial/ethnic minorities, likely due to recruitment from urban health care settings. Survey sample participants had moderate disease activity levels over the past four years and an average SF-36 score of 38, typical of a chronically ill population (general population mean = 50; higher scores represent better health). Fewer than half the sample was employed and 30% reported work disability.
Table 1.
Sample descriptions for clinic and Lupus Outcomes Study (LOS) survey samples.
| Variable | Clinic Sample (n=81) |
LOS Survey Sample (n=717) |
|---|---|---|
| n (%) or mean (SD), range | ||
| Female | 75 (93%) | 661 (92%) |
| Age at interview | 35 (11), 18-61 | 51 (12), 22-86 |
| High school education or less | 24 (30%) | 103 (14%) |
| Nonwhite | 61 (75%) | 239 (33%) |
| Currently employed | 324 (45%) | |
| Poverty-level income | 78 (11%) | |
| Disease duration | 10 (8), 1-44 | 17 (8), 3-50 |
| Disease activity (SLAQ, over 4 yrs) | -- | 13 (7), 0-37 |
| Self-rated health fair/poor | -- | 305 (43%) |
| SF-36 physical component score | -- | 38 (12), 8-69 |
| Unable to work | -- | 218 (30%) |
| Number of hospitalizations (over 5 yrs) | -- | 2 (3), 0-35 |
| Average annual MD visits for SLE (over 5 years) |
-- | 12 (10), 0-83 |
SLAQ = Systemic Lupus Activity Questionnaire
The percent agreement between each SDI item and corresponding BILD and LDIQ items is displayed in Table 2. Of the 26 SDI items assessed in the BILD, two items were not reported by any patients: pulmonary fibrosis and osteomyelitis. Of the 42 SDI items assessed in the LDIQ, only chronic peritonitis was not reported. Observed agreement between BILD items and the SDI ranged from 75-100%, while prevalence-adjusted bias-adjusted kappas (PABAKs) ranged from 0.70-1.00, except for deforming or erosive arthritis (PABAK=0.68) and extensive scarring/panniculum (PABAK=0.51). Observed agreement between SDI and LDIQ for only the items retained in the BILD ranged from 77-100% with PABAKs from 0.68-1.00, except for deforming or erosive arthritis (PABAK=0.61), extensive scarring/panniculum (PABAK=0.54), and cognitive impairment (PABAK=0.54). Observed agreement between SDI and LDIQ for all LDIQ items ranged from 53-100% with PABAKs from 0.05 to 1.00, including 7 items with PABAK from 0.05 to 0.61; four of the 14 items in LDIQ but not in BILD had PABAKs in that range. After analysis of the correspondence between the BILD and SDI, two items (erosive arthritis and extensive scarring of the skin) were so commonly reported as to render them uninformative to the BILD score as a whole. Their PABAK scores were <0.70 so they were therefore dropped from the subsequent calculation of the BILD score.
Table 2.
Item by item comparison of SLICC Damage Index (SDI), Lupus Damage Index Questionnaire (LDIQ), and Brief Index of Lupus Damage (BILD) in clinic sample of 81 patients.
| BILD vs. SDI | LDIQ vs. SDI** | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Organ System and Item | SDI # (%) |
BILD # (%) |
% agree |
PABAK | LDIQ # (%) |
% agree |
PABAK | |||
| Ocular | ||||||||||
| Any cataract ever | 14 | 17% | 14 | 17% | 93% | 0.85 | 14 | 19% | 97% | 0.95 |
| Retinal change or optic atrophy | 2 | 2% | 6 | 7% | 93% | 0.85 | 10 | 14% | 89% | 0.78 |
| Neuropsychiatric | ||||||||||
| Cognitive impairment or major psychosis | 6 | 7% | 4 | 5% | 88% | 0.75 | 17 | 23% | 77% | 0.54 |
| Seizures requiring therapy 6+ mos. | 7 | 9% | 4 | 5% | 94% | 0.88 | 3 | 4% | 93% | 0.86 |
| Cerebrovasc accident/resection | 4 | 5% | 10 | 12% | 90% | 0.80 | 5 | 7% | 93% | 0.86 |
| Transverse myelitis | 0 | 1 | 1% | 99% | 0.98 | 12 | 16% | 84% | 0.68 | |
| Cranial/peripheral neuropathy | 6 | 7% | (omitted) | 18 | 24% | 73% | 0.46 | |||
| Renal | ||||||||||
| ESRD (dialysis/transplant) | 6 | 7% | 8 | 10% | 95% | 0.90 | 4 | 5% | 96% | 0.92 |
| GFR <50% | 4 | 5% | (omitted) | 32 | 43% | 62% | 0.24 | |||
| Proteinuria >3.5g/24h | 5 | 6% | (omitted) | 40 | 54% | 53% | 0.05 | |||
| Pulmonary | ||||||||||
| Pulm. hypertension | 2 | 2% | 5 | 6% | 96% | 0.93 | 7 | 9% | 93% | 0.86 |
| Pulm. fibrosis | 0 | - | 0 | - | 100% | 1.00 | 5 | 7% | 93% | 0.86 |
| Shrinking lung | 0 | - | (omitted) | 1 | 1% | 99% | 0.97 | |||
| Pleural fibrosis | 1 | 1% | (omitted) | 4 | 5% | 96% | 0.92 | |||
| Pulm. infarction or resection | 1 | 2% | (omitted) | 5 | 7% | 95% | 0.89 | |||
| Cardiovascular | ||||||||||
| Angina or coronary artery bypass | 0 | - | 2 | 2% | 98% | 0.95 | 3 | 4% | 96% | 0.92 |
| Myocardial infarction | 2 | 2% | 1 | 1% | 98% | 0.95 | 1 | 1% | 99% | 0.97 |
| Pericarditis 6+mos, or pericardiectomy | 5 | 9% | 3 | 5% | 93% | 0.86 | 5 | 9% | 96% | 0.93 |
| Cardiomyopathy | 4 | 5% | (omitted) | 3 | 4% | 99% | 0.97 | |||
| Valvular disease | 1 | 1% | (omitted) | 3 | 4% | 95% | 0.89 | |||
| Peripheral vascular | ||||||||||
| Minor tissue loss (pulp space) | 4 | 5% | 14 | 17% | 88% | 0.75 | 4 | 5% | 95% | 0.89 |
| Sig. tissue loss, ever | 0 | - | 2 | 2% | 98% | 0.95 | 3 | 4% | 96% | 0.92 |
| Venous thrombosis with swelling, | 8 | 10% | 10 | 12% | 88% | 0.75 | 5 | 7% | 88% | 0.76 |
| ulceration or venous stasis | ||||||||||
| Claudication, 6+mos | 0 | - | (omitted) | 9 | 12% | 88% | 0.76 | |||
| Gastrointestinal | ||||||||||
| Chronic peritonitis | 0 | - | 2 | 2% | 98% | 0.95 | 0 | - | 100% | 1.00 |
| Stricture or upper GI tract surgery | 0 | - | 1 | 1% | 99% | 0.98 | 6 | 8% | 92% | 0.84 |
| Infarction/resection bowel, spleen liver, gall bladder |
3 | 4% | (omitted) | 5 | 7% | 97% | 0.95 | |||
| Mesenteric insufficiency | 0 | - | (omitted) | (omitted) | ||||||
| Pancreatic insufficiency | 1 | 1% | (omitted) | 1 | 1% | 100% | 1.00 | |||
| Musculoskeletal | ||||||||||
| Deforming or erosive arthritis* | 2 | 2% | 12 | 15% | 84% | 0.68 | 12 | 16% | 81% | 0.61 |
| Osteoporosis w. fracture or vertebral. collapse |
4 | 5% | 6 | 74% | 95% | 0.90 | 4 | 5% | 92% | 0.84 |
| Avascular necrosis | 7 | 9% | 6 | 7% | 96% | 0.93 | 5 | 7% | 95% | 0.89 |
| Osteomyelitis | 0 | - | 0 | - | 100% | 1.00 | 1 | 1% | 99% | 0.97 |
| Muscle atrophy or weakness | 4 | 5% | (omitted) | 1 | 1% | 93% | 0.86 | |||
| Ruptured Tendon | 0 | - | (omitted) | 1 | 1% | 98% | 0.96 | |||
| Skin | ||||||||||
| Extensive scarring/panniculum* | 8 | 10% | 28 | 35% | 75% | 0.51 | 21 | 28% | 77% | 0.54 |
| Skin ulceration (not thrombosis), >6 mos. |
4 | 5% | 8 | 10% | 93% | 0.85 | 8 | 11% | 92% | 0.84 |
| Scarring chronic alopecia | 5 | 6% | (omitted) | 13 | 18% | 80% | 0.59 | |||
| Premature gonadal failure | 1 | 2% | 13 | 16% | 85% | 0.70 | 6 | 8% | 91% | 0.81 |
| Diabetes | 4 | 5% | 7 | 9% | 90% | 0.80 | 6 | 8% | 95% | 0.89 |
| Malignancy | 1 | 2% | 2 | 2% | 99% | 0.98 | 2 | 3% | 99% | 0.97 |
These two items were dropped from the final BILD score.
Based on 74 patients with LDIQ score.
Despite differences in the item-by-item comparisons of the SDI and the BILD, the distributions of these two scores were similar, while the LDIQ scores were predictably considerably higher. The BILD and SDI had a moderately high Spearman rank correlation (rs) of 0.64 (p<0.001). In our sample, the rs of the SDI and the LDIQ was 0.54 (p<0.001, Table 3), comparable to that reported in the original LDIQ validation paper (rs=0.48) (18).
Table 3.
Distributions and comparisons of three SLE damage scores in a clinic sample of 81 patients.
| Score | scores >0 | mean | median | I.Qtl. range |
max. score |
Correlation* with SDI |
|---|---|---|---|---|---|---|
| SLICC Damage Index (SDI) (n=81) | 54 (67%) | 2 | 1 | 0-3 | 7 | -- |
| Brief Lupus Damage Index (BILD) (n=81) |
57 (70%) | 2 | 1 | 0-3 | 6 | 0.64 (p<0.001) |
| Lupus Damage Index Questionnaire (LDIQ) (n=74†) |
64 (86%) | 4 | 3 | 1-5 | 14 | 0.54 (p<0.001) |
Spearman rank correlation (rs)
Seven patients did not complete LDIQ.
The next phase of the study involved administering the BILD to participants in the LOS. The acceptability of the BILD for LOS respondents was very high. Only four items had more than 1% of individuals who did not respond (resulting in missing values); these included history of angina or bypass (n=12), retinal disease (n=11), peritonitis (n=10), and interstitial lung disease (n=9). In the LOS, the median BILD score was 1 with an interquartile range of 0 to 3 and a maximum score of 6 (data not shown), identical to the clinic sample.
In Table 4, we evaluated the construct validity of the BILD by comparing demographic, health status, and health utilization characteristics of LOS participants to their quartile of BILD (quartile scores were 0, 1, 2-3, and 4+ points). As BILD quartiles increased (reflecting greater damage), respondent age increased, as did the percent with incomes below poverty and the percent reporting being unemployed (all p<0.001). No clear relationship was seen by race/ethnicity or gender. Individuals in higher BILD quartiles also had longer disease duration, higher 4-year mean disease activity scores, lower SF-36 PCS and MCS scores, and higher percentages with poor self-rated health and work disability (all p<0.001), with each of these demonstrating monotonic relationships. Finally, individuals in the higher BILD quartiles also had more hospitalizations and a greater number of physician visits for SLE over the previous five years (p<0.001).
Table 4.
Comparisons of demographic, health status, and health utilization measures, by quartiles of BILD, in the Lupus Outcomes Study survey sample (n=717).
| Characteristic | BILD score 0 (n=141) |
BILD score 1 (n=189) |
BILD score 2-3 (n=222) |
BILD score 4+ (n=165) |
p- value* |
|---|---|---|---|---|---|
| Sociodemographics | |||||
| Age at interview: mean (SD) | 46 (12) | 50 (12) | 52 (12) | 56 (12) | <0.001 |
| % Female | 20 | 27 | 30 | 23 | 0.048 |
| % Nonwhite | 21 | 26 | 35 | 18 | 0.199 |
| % High school education or less | 20 | 28 | 26 | 25 | 0.916 |
| % Household incom below poverty | 12 | 22 | 26 | 41 | <0.001 |
| % Employed | 27 | 32 | 32 | 9 | <0.001 |
| SLE status | |||||
| Disease duration: mean(SD) | 13 (6) | 16 (8) | 18 (9) | 20 (9) | <0.001 |
| SLAQ, averaged over 4 years: mean(SD) | 9 (6) | 11 (7) | 12 (7) | 17 (8) | <0.001 |
| General health status | |||||
| % Fair/poor self-rated health | 11 | 19 | 33 | 38 | <0.001 |
| SF-36 physical component score: mean(SD) | 45 (11) | 41 (11) | 37 (11) | 30 (11) | <0.001 |
| SF-36 mental component score: mean(SD) | 48 (10) | 48 (12) | 48 (12) | 44 (14) | <0.001 |
| % Unable to work | 9 | 20 | 30 | 41 | <0.001 |
| Health care utilization | |||||
| Hospitalizations over 5 years: mean (SD) | 0.5 (0.9) | 0.8 (1.4) | 1.7 (3.1) | 3.4 (4.2) | <0.001 |
| Average annual MD visits for SLE over 5 years: mean(SD) |
6.9 (5.7) | 8.6 (5.8) | 11.2 (8.3) | 19.2 (13.7) | <0.001 |
p-values from ANOVA F-test for continuous variables, and for chi-square test for trend for binary variables.
To further examine the relationship between sociodemographic characteristics and damage, we constructed several multivariable models, where the outcome was the BILD score (top quartile versus not, bottom quartile versus not, and raw score), and predictors included age, gender, race/ethnicity, disease duration, and poverty status. All analyses yielded similar conclusions, with disease duration and poverty emerging as significant predictors of damage; no statistically significant effect was seen by race/ethnicity.
Discussion
Extending scientific research outside the clinical setting in SLE remains a challenging task. The relative rarity and complexity of the disease remain barriers for population studies, as do the lack of suitable case finding and disease assessment tools. Although measures of general health, such as the SF-36, provide some insight into disease status in epidemiologic studies, more specific tools have the potential to better detect health–related outcome changes. The development and validation of patient–reported instruments hold promise in addressing this gap. In this study, we report our methods for developing and performing an initial validation study of a patient-reported instrument, the BILD, designed to assess disease damage. Our findings suggest that the BILD is acceptable to respondents, is efficient to administer, and has content, criterion and construct validity.
We designed the BILD to capture the overall concept of damage in SLE for epidemiological research. It is important to note that it is not a direct substitute for the SDI, since the BILD omits many items from the SDI that were either not suitable for patient self-report or were not informative because of their frequent reporting by patients. Instead, among a group of patients (rather than in any individual patient), the BILD is able to differentiate between those with high or low degrees of SLE damage. We found good agreement between items in the BILD and the corresponding items in the SDI, and an overall moderately high correlation between the two measures (0.64), suggesting criterion validity. In addition, through both pilot testing and administration of the instrument to over 700 individuals with SLE, we found that the instrument was acceptable to patients, as evidenced by the very high individual item response rate.
To ascertain construct validity, we compared patients in the four quartiles of BILD scores on measures found in previous literature to relate to disease damage as measured by the SDI. Consistent with studies of the SDI, we found that BILD scores in the LOS were higher among older individuals, those with longer disease duration, and those living below the federal poverty level (9, 22-26). Some previous studies have suggested greater damage among certain racial/ethnic minority groups (22), but many have not, once poverty was accounted for (27-29); we also did not find a statistically significant association between damage and race/ethnicity in our multivariable analyses. As expected, those with a higher mean disease activity score over a four year period had higher damage scores (24). Higher BILD scores were also associated with worse self-rated health, a lower SF-36 physical component score, work disability and employment (6, 9, 10). Finally, individuals with higher BILD scores had significantly greater health care utilization, including a greater number of hospitalizations and physician visits over the last four years. This is consistent with health care utilization studies involving the SDI (8, 30).
A written survey to assess patient-reported damage, the LDIQ, was developed concurrently with our effort to develop and test the BILD for telephone or interviewer administration. LDIQ investigators allowed us to simultaneously test the criterion validity of that instrument in our clinic sample, providing a second U.S. validation for that instrument. We found that both the LDIQ and BILD correlated acceptably with the SDI (rs for LDIQ=0.54, rs for BILD 0.64). The correlation of the LDIQ and SDI in our study sample was similar to the published LDIQ criterion validity assessments performed in the United States (rs=0.48). Important differences between the BILD and LDIQ include the mode of administration (LDIQ is a written survey, BILD is designed for administration by an interviewer in-person or on the telephone) and length (LDIQ has 56 items, the final BILD instrument has 26 items). Given four large international patient samples, criterion validity testing for the LDIQ has been significantly more extensive; the BILD will require further testing to confirm criterion validity in larger, independent samples. Construct validity testing for the two instruments has been comparable in two large community-based samples (the National Databank of Rheumatic Diseases for the LDIQ and the Lupus Outcomes Study for the BILD) (18).
Although the analyses presented here support the content, criterion validity and construct validity of the BILD, it is important to note that characterization of the other psychometric properties of the instrument will require further research. For example, we did not assess the reliability of the BILD (either test-retest, or inter-interviewer reliability). Assessment of external validity in an independent sample with different sociodemographic or clinical characteristics should also be performed. The clinic-based sample used to assess criterion validity and the LOS sample differed significantly based on race/ethnicity, disease duration and age. Theoretically, the BILD could correlate differently among these subgroups with the physician-assessed SDI, and future studies of criterion validity with a larger, more heterogeneous sample should investigate this possibility. Finally, an important strength of the SDI is its association with significant long-term clinical outcomes, such as mortality. It remains to be seen whether either of the two newly developed patient-reported measures of disease damage will have similar predictive validity.
In summary, we have developed and performed a preliminary validation study of a new patient reported instrument of disease damage in SLE. The BILD, which is designed for telephone or interviewer administration, had content, criterion and construct validity in this study. Although further studies are needed to examine its reliability and to document its psychometric properties in populations with different sociodemographic or clinical characteristics, the BILD appears to represent a promising tool for studies of SLE outside the clinical setting.
Supplementary Material
Acknowledgments
Funding: Supported by NIAMS P60-AR-053308. Additional support from the Arthritis Foundation, AHRQ/NIAMS 2 RO1 HS013893, NIAMS 5R01AR56476, State of California Lupus Fund and the Rosalind Russell Medical Research Center for Arthritis. The study was also carried out in part in the General Clinical Research Center, Moffit Hospital, University of California, San Francisco, with funds provided by the National Center for Research Resources, 5 M01 RR-00079, U.S. Public Health Service.
References
- 1.Nived O, Jonsen A, Bengtsson AA, Bengtsson C, Sturfelt G. High predictive value of the Systemic Lupus International Collaborating Clinics/American College of Rheumatology damage index for survival in systemic lupus erythematosus. J Rheumatol. 2002;29(7):1398–400. [PubMed] [Google Scholar]
- 2.Alarcon GS, McGwin G, Jr., Bastian HM, Roseman J, Lisse J, Fessler BJ, et al. LUMINA Study Group Systemic lupus erythematosus in three ethnic groups. VII [correction of VIII]. Predictors of early mortality in the LUMINA cohort. Arthritis Rheum. 2001;45(2):191–202. doi: 10.1002/1529-0131(200104)45:2<191::AID-ANR173>3.0.CO;2-2. [DOI] [PubMed] [Google Scholar]
- 3.Cardoso CR, Signorelli FV, Papi JA, Salles GF. Initial and accrued damage as predictors of mortality in Brazilian patients with systemic lupus erythematosus: a cohort study. Lupus. 2008;17(11):1042–8. doi: 10.1177/0961203308093829. [DOI] [PubMed] [Google Scholar]
- 4.Chambers SA, Allen E, Rahman A, Isenberg D. Damage and mortality in a group of British patients with systemic lupus erythematosus followed up for over 10 years. Rheumatology (Oxford) 2009;48(6):673–5. doi: 10.1093/rheumatology/kep062. [DOI] [PubMed] [Google Scholar]
- 5.Rahman P, Gladman DD, Urowitz MB, Hallett D, Tam LS. Early damage as measured by the SLICC/ACR damage index is a predictor of mortality in systemic lupus erythematosus. Lupus. 2001;10(2):93–6. doi: 10.1191/096120301670679959. [DOI] [PubMed] [Google Scholar]
- 6.Mok CC, Ho LY, Cheung MY, Yu KL, To CH. Effect of disease activity and damage on quality of life in patients with systemic lupus erythematosus: a 2-year prospective study. Scand J Rheumatol. 2009;38(2):121–7. doi: 10.1080/03009740802415527. [DOI] [PubMed] [Google Scholar]
- 7.Wang C, Mayo NE, Fortin PR. The relationship between health related quality of life and disease activity and damage in systemic lupus erythematosus. J Rheumatol. 2001;28(3):525–32. [PubMed] [Google Scholar]
- 8.Clarke AE, Petri M, Manzi S, Isenberg DA, Gordon C, Senecal JL, et al. The systemic lupus erythematosus Tri-nation Study: absence of a link between health resource use and health outcome. Rheumatology (Oxford) 2004;43(8):1016–24. doi: 10.1093/rheumatology/keh229. [DOI] [PubMed] [Google Scholar]
- 9.Baker K, Pope J, Fortin P, Silverman E, Peschken C. Work disability in systemic lupus erythematosus is prevalent and associated with socio-demographic and disease related factors. Lupus. 2009;18(14):1281–8. doi: 10.1177/0961203309345784. [DOI] [PubMed] [Google Scholar]
- 10.Bertoli AM, Fernandez M, Alarcon GS, Vila LM, Reveille JD. Systemic lupus erythematosus in a multiethnic US cohort LUMINA (XLI): factors predictive of self-reported work disability. Ann Rheum Dis. 2007;66(1):12–7. doi: 10.1136/ard.2006.055343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gladman D, Ginzler E, Goldsmith C, Fortin P, Liang M, Urowitz M, et al. The development and initial validation of the Systemic Lupus International Collaborating Clinics/American College of Rheumatology damage index for systemic lupus erythematosus. Arthritis Rheum. 1996;39(3):363–9. doi: 10.1002/art.1780390303. [DOI] [PubMed] [Google Scholar]
- 12.Gladman DD, Urowitz MB. The SLICC/ACR damage index: progress report and experience in the field. Lupus. 1999;8(8):632–7. doi: 10.1191/096120399680411335. [DOI] [PubMed] [Google Scholar]
- 13.Gladman DD, Urowitz MB, Goldsmith CH, Fortin P, Ginzler E, Gordon C, et al. The reliability of the Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index in patients with systemic lupus erythematosus. Arthritis Rheum. 1997;40(5):809–13. doi: 10.1002/art.1780400506. [DOI] [PubMed] [Google Scholar]
- 14.Yelin E, Trupin L, Katz P, Criswell L, Yazdany J, Gillis J, et al. Work dynamics among persons with systemic lupus erythematosus. Arthritis Rheum. 2007;57(1):56–63. doi: 10.1002/art.22481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wolfe F, Petri M, Alarcon GS, Goldman J, Chakravarty EF, Katz RS, et al. Fibromyalgia, systemic lupus erythematosus (SLE), and evaluation of SLE activity. J Rheumatol. 2009;36(1):82–8. doi: 10.3899/jrheum.080212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Karlson EW, Daltroy LH, Rivest C, Ramsey-Goldman R, Wright EA, Partridge AJ, et al. Validation of a Systemic Lupus Activity Questionnaire (SLAQ) for population studies. Lupus. 2003;12(4):280–6. doi: 10.1191/0961203303lu332oa. [DOI] [PubMed] [Google Scholar]
- 17.Yazdany J, Yelin EH, Panopalis P, Trupin L, Julian L, Katz PP. Validation of the systemic lupus erythematosus activity questionnaire in a large observational cohort. Arthritis Rheum. 2008;59(1):136–43. doi: 10.1002/art.23238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Costenbader KH, Khamashta M, Ruiz-Garcia S, Perez-Rodriguez MT, Petri M, Elliott J, et al. Development and initial validation of a self-assessed lupus organ damage instrument. Arthritis Care Res (Hoboken) 62(4):559–68. doi: 10.1002/acr.20193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Pons-Estel BA, Sanchez-Guerrero J, Romero-Diaz J, Iglesias-Gamarra A, Bonfa E, Borba EF, et al. Validation of the Spanish, Portuguese and French versions of the Lupus Damage Index questionnaire: data from North and South America, Spain and Portugal. Lupus. 2009;18(12):1033–52. doi: 10.1177/0961203309105590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The Problems of Two Paradoxes. J Clin Epidemiol. 1990;43:543–548. doi: 10.1016/0895-4356(90)90158-l. [DOI] [PubMed] [Google Scholar]
- 21.Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423–429. doi: 10.1016/0895-4356(93)90018-v. [DOI] [PubMed] [Google Scholar]
- 22.Alarcon GS, McGwin G, Jr., Bartolucci AA, Roseman J, Lisse J, Fessler BJ, et al. Systemic lupus erythematosus in three ethnic groups. IX. Differences in damage accrual. Arthritis Rheum. 2001;44(12):2797–806. doi: 10.1002/1529-0131(200112)44:12<2797::aid-art467>3.0.co;2-9. [DOI] [PubMed] [Google Scholar]
- 23.Alarcon GS, Roseman JM, McGwin G, Jr., Uribe A, Bastian HM, Fessler BJ, et al. Systemic lupus erythematosus in three ethnic groups. XX. Damage as a predictor of further damage. Rheumatology (Oxford) 2004;43(2):202–5. doi: 10.1093/rheumatology/keg481. [DOI] [PubMed] [Google Scholar]
- 24.Becker-Merok A, Nossent HC. Damage accumulation in systemic lupus erythematosus and its relation to disease activity and mortality. J Rheumatol. 2006;33(8):1570–7. [PubMed] [Google Scholar]
- 25.Gladman DD, Urowitz MB, Rahman P, Ibanez D, Tam LS. Accrual of organ damage over time in patients with systemic lupus erythematosus. J Rheumatol. 2003;30(9):1955–9. [PubMed] [Google Scholar]
- 26.Lotstein DS, Ward MM, Bush TM, Lambert RE, van Vollenhoven R, Neuwelt CM. Socioeconomic status and health in women with systemic lupus erythematosus. J Rheumatol. 1998;25(9):1720–9. [PubMed] [Google Scholar]
- 27.Karlson EW, Daltroy LH, Lew RA, Wright EA, Partridge AJ, Fossel AH, et al. The relationship of socioeconomic status, race, and modifiable risk factors to outcomes in patients with systemic lupus erythematosus. Arthritis Rheum. 1997;40(1):47–56. doi: 10.1002/art.1780400108. [DOI] [PubMed] [Google Scholar]
- 28.Rivest C, Lew RA, Welsing PM, Sangha O, Wright EA, Roberts WN, et al. Association between clinical factors, socioeconomic status, and organ damage in recent onset systemic lupus erythematosus. J Rheumatol. 2000;27(3):680–4. [PubMed] [Google Scholar]
- 29.Peschken CA, Katz SJ, Silverman E, Pope JE, Fortin PR, Pineau C, et al. The 1000 Canadian faces of lupus: determinants of disease outcome in a large multiethnic cohort. J Rheumatol. 2009;36(6):1200–8. doi: 10.3899/jrheum.080912. [DOI] [PubMed] [Google Scholar]
- 30.Sutcliffe N, Clarke AE, Taylor R, Frost C, Isenberg DA. Total costs and predictors of costs in patients with systemic lupus erythematosus. Rheumatology (Oxford) 2001;40(1):37–47. doi: 10.1093/rheumatology/40.1.37. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.