Assessment of damage in vasculitis: expert ratings of damage

Abstract

Objectives. Current measures of damage in vasculitis do not account for the possibility that some forms of damage may exert greater impact than others. As part of an international effort to revise how damage is quantified in vasculitis clinical research, an exercise was performed to measure expert ratings of damage items.

Methods. Members of the Vasculitis Clinical Research Consortium and European Vasculitis Study Group were given a list of 129 items of damage related to WG and microscopic polyangiitis (MPA). Participants were asked to rate each item of damage on an integer scale from 0 to 10, where 10 represented the most severe form of damage and 0 indicated ‘no impact’.

Results. A multidisciplinary panel of 50 investigators from North America, Europe and Australia–New Zealand participated. The highest median ratings (8–10) were assigned to items of damage associated with malignancy, tissue ischaemia, the central nervous system and cardiopulmonary manifestations. The mean scores ranged from 1.3 to 9.5. The highest s.d.s (⩾2.5) were associated with forms of damage that may benefit from surgical intervention or may not be causally associated with WG or MPA. Lower scores were assigned by nephrologists in comparison with rheumatologists and by Americans in comparison to Europeans, although the difference in median ranks used by these groups was not statistically significant (P > 0.05 for the comparisons).

Conclusions. This exercise represents an important step in the development of a weighting system that may increase the utility of damage index scores for the assessment of patients with vasculitis.

Introduction

Any description of chronic disease requires multiple components, including an assessment of damage, disease activity, mortality, quality of life and health care costs [1]. The concept of damage denotes the consequences of disease that do not reverse with therapy [2]. Although clinical trials frequently focus on measures of disease activity, damage represents the long-term outcome experienced by the patient and may be a better measure of the chronic burden of disease. The prevention of cumulative damage is an important long-term goal of any vasculitis therapy.

Although advances in the treatment of the primary systemic vasculitides have resulted in a dramatic reduction in disease-related mortality, patients still experience cycles of relapse and remission that lead to the accrual of damage due to both the underlying disease and the toxic effects of treatment [3].

WG and microscopic polyangiitis (MPA) have been the focus of several randomized clinical trials in the USA and Europe. In clinical trials of vasculitis, damage is assessed using the Vasculitis Damage Index (VDI), which catalogues 64 forms of chronic morbidity associated with the primary systemic vasculitides [4]. More recently, the ANCA-associated Vasculitis Index of Damage (AVID) was created to record forms of damage unique to WG and MPA [5].

These damage assessment instruments are used to obtain a ‘score’ that represents the total burden of damage experienced by a patient. Both AVID and the VDI weigh all forms of damage equally, so that the damage score is simply a summation of each form of damage experienced by the individual patient. This does not capture our intuitive sense that some forms of damage exert greater impact than others [6]. To address this possibility, we performed an exercise in which physicians with expertise in the assessment of vasculitis were asked to rate items of damage in terms of severity.

Methods

Data forms and data collection

Items of damage from AVID and the VDI were extracted into a single data collection form, yielding a list of 129 items of damage, divided into 17 organ-based categories. The complete list, known as the Combined Damage Assessment (CDA) Index, is shown in Appendix 1 (available as supplementary data at Rheumatology Online). Participants were instructed to rate each item of damage using an integer scale from 0 to 10, where 10 represented the most severe forms of damage and 0 indicated that the item of damage had no impact. The instructions noted that each item of damage represents a spectrum of severity, but the rating assigned should reflect the impact that an item of damage has for the majority of patients with WG or MPA.

Forms were distributed to investigators with expertise in the assessment of systemic vasculitis from the Vasculitis Clinical Research Consortium (VCRC) and the European Vasculitis Study Group (EUVAS).

Analysis

Analyses were conducted using STATA 9.2 statistical software (College Park, TX, USA). Median scores, mean scores and s.d.s were calculated for each item of damage. Subgroup analysis was conducted by geographic region (European vs American) and by primary specialty (rheumatology vs nephrology). These subgroups were compared using the Wilcoxon–Mann–Whitney test.

Kurtosis and skewness were calculated for each subgroup. Kurtosis measures whether the distribution is peaked or flat relative to a normal distribution. Skewness measures the symmetry of the distribution. Together, these statistics allow one to determine the degree to which a population departs from a normal distribution, which should have a kurtosis of 3 and a skewness of 0.

Results

Participants

Fifty investigators participated in this exercise, including 20 rheumatologists, 17 nephrologists, 8 internists, 2 clinical immunologists and 1 specialist in pulmonary and critical care medicine; 2 did not list a specialty. These investigators represented three continents: 35 investigators from Europe, 8 from North America and 2 from Australia–New Zealand.

Distribution of damage rating scores

Median item ratings are listed in Table 1. Investigators used the entire range of scores, from 0 to 10. No item of damage received a median rating of 0 or 1. Twenty-eight items were in the lowest range (scores 2–4), 52.8% of the items fell into the middle range (scores 5–7), and 19% of the items were assigned to the highest range (scores 8–10). The highest scores were given to items of damage associated with malignancy, tissue ischaemia and organ failure.

Table 1.

Items of damage in the CDA index

Item of damage	Median	Mean	s.d.	Item of damage	Median	Mean	s.d.
Muscle atrophy, normal strength	2	1.3	1.0	Diplopia	6	5.9	1.8
Alopecia	2	2.3	1.2	Sensorineural hearing loss	6	5.9	0.9
Mouth ulcers	2	2.1	1.1	Vena caval filter placement	6	4.7	1.9
Striae	2	2.1	1.1	Percutaneous coronary intervention	6	5.8	2.4
Easy bruising	2	1.8	0.9	Valvular disease	6	6.2	2.1
Muscle atrophy with weakness	3	2.9	1.3	Major vessel stenosis	6	5.5	2.8
Cataracts	3	3.8	1.6	Claudication	6	5.3	2.2
Eustachian tube dysfunction	3	3.2	1.1	Complicated DVT	6	6.5	1.4
Auricular deformity	3	3.1	1.4	Carotid artery disease	6	5.5	2.3
Cholesteatoma	3	4.0	1.2	Renal artery stenosis	6	5.6	2.1
Chronic rhinitis/crusting	3	3.5	1.3	Proteinuria >3 g/24 h	6	6.0	1.3
Nasolacrimal duct obstruction	3	4.1	1.6	Premature ovarian failure	6	5.5	1.6
Nasal septal perforation	3	3.3	1.6	Azospermia	6	4.8	1.5
Sensory neuropathy, mild	3	3.	1.2	Cystitis requiring transfusion	6	6.8	1.3
Weight gain >10 lbs/4.4 kg	3	3.5	1.9	Pulmonary embolism	7	5.2	1.6
Fibromyalgia	3	3.4	1.9	Neuropathic pain	7	5.6	2.2
Cystitis with micro-haematuria	3	3.5	1.8	Scleral perforation	7	7.5	1.8
Scleral thinning	4	3.7	1.8	Optic nerve atrophy	7	6.6	1.9
Minor tissue loss	4	3.8	2.3	Retinal artery occlusion	7	6.9	1.8
Medicines to manage side effects	4	3.0	2.0	Low vision	7	6.4	1.2
Hypogammaglobulinaemia	4	3.4	1.9	Orbital wall destruction	7	6.4	2.0
Cutaneous ulcers	4	3.3	0.9	Large airway obstruction	7	6.2	1.7
Glaucoma	4	4.1	1.8	Pulmonary fibrosis	7	6.8	1.4
Tympanic membrane perforation	4	2.7	1.3	Pulmonary infarction	7	5.9	1.7
Tinnitus	4	4.1	1.7	Chronic breathlessness	7	6.5	2.0
Anosmia	4	4.0	1.5	Myocardial infarction	7	7.6	2.0
Ageusia	4	4.0	1.6	Coronary artery bypass graft	7	6.8	2.2
Chronic sinusitis	4	3.9	1.1	Major tissue loss	7	7.3	1.7
Neo-ossification of sinuses	4	3.7	1.8	Arterial thrombosis/occlusion	7	5.5	2.3
Pleural fibrosis	4	3.5	2.0	Hepatic fibrosis	7	6.0	2.5
Hypertension	4	4.1	1.8	Oesophageal stricture	7	5.8	2.2
Loss of pulses	4	4.3	2.7	Chronic peritonitis	7	6.5	2.8
Deep venous thrombosis	4	4.4	1.7	Chronic kidney disease	7	6.1	1.8
Proteinuria <3 g/24 h	4	3.6	1.9	Seizures	7	7.4	1.7
Impaired fasting glucose	4	3.2	1.8	Sensory neuropathy, severe	7	7.5	1.8
Osteoporosis	5	5.2	1.7	Motor neuropathy (mononeuritis)	7	7.3	1.1
Fracture with renal dystrophy	5	5.0	2.1	Cognitive impairment	7	7.2	1.6
Fracture with osteoporosis	5	5.2	1.7	Damage requiring surgery	7	6.1	2.8
Muscle atrophy, impaired ADLs	5	5.3	1.5	Subglottic stenosis with surgery	8	6.6	1.7
Deforming/erosive arthritis	5	4.7	2.1	Irreversible loss of lung function	8	6.3	1.4
Proptosis	5	4.8	2.4	Pulmonary hypertension	8	7.2	1.1
Pseudotumour	5	4.9	1.8	LV dysfunction, NYHA III/IV	8	7.5	1.9
Retinal changes	5	3.9	1.6	Major tissue loss, second event	8	7.7	1.4
Conductive hearing loss	5	4.6	1.3	Mesenteric insufficiency	8	7.3	1.6
Nasal bridge collapse	5	5.7	2.1	Transverse myelitis	8	8.8	1.3
Subglottic stenosis, no surgery	5	4.5	2.2	Stroke	8	8.3	1.7
Asthma	5	5.3	1.9	Major psychosis	8	7.2	2.8
Angina	5	5.7	2.0	Cervical carcinoma	8	7.4	1.9
LV dysfunction NYHA I/II	5	4.5	2.3	Refractory cytopenia	8	8.0	1.6
Third-degree AV block	5	5.9	2.1	Myelodysplastic syndrome	8	8.1	1.7
Pericarditis	5	5.6	1.8	Cystitis with cystectomy	8	8.7	1.5
Loss of pulses, second event	5	5.2	2.8	Blindness, one eye	9	8.2	1.5
Sensory neuropathy, moderate	5	5.3	1.5	Continuous oxygen dependency	9	8.8	1.1
Anxiety due to vasculitis	5	4.9	1.4	Gut infarction/resection	9	8.2	2.1
Mood disorder due to vasculitis	5	4.7	2.0	End-stage renal disease	9	8.7	1.4
Diabetes insipidus	5	4.9	1.8	Dialysis	9	8.7	1.4
Cystitis with gross haematuria	5	5.2	1.6	Renal transplant	9	7.7	2.5
Avascular necrosis	6	5.7	2.1	Second stroke	9	9.0	1.5
Osteomyelitis	6	6.2	2.6	Bladder cancer	9	8.3	1.5
Gangrene with tissue loss	6	5.7	2.1	Solid tumour malignancy	9	9.0	1.4
Optic nerve oedema	6	4.5	1.6	Blindness, second episode	10	8.2	1.5
Retinal vein occlusion	6	6.1	2.2	Haematopoietic malignancy	10	9.2	1.2

Mean scores for individual items of damage ranged from 1.3 to 9.5 and s.d.s for the scores ranged from 0.8 to 2.8 (Table 1). The highest s.d.s (i.e. ⩾2.5) were predominantly associated with forms of damage that may benefit from surgical (e.g. renal transplantation, damage requiring surgical intervention) or may not be directly associated with WG or MPA (e.g. osteomyelitis, hepatic fibrosis and chronic peritonitis).

Damage rating by specialty

An analysis was conducted based on the primary specialty of the participants (i.e. nephrologists vs rheumatologists). Other specialties (including internal medicine, clinical immunology and pulmonary medicine) were represented, but in insufficient numbers to allow subgroup analysis. There was considerable overlap in the ratings assigned by these groups (Fig. 1A). Nephrologists rated more items of damage in the lower range (skewness 0.23), whereas rheumatologists rated more items of damage in the higher range (skewness −0.23), but the difference was not statistically significant (P > 0.05). The median scores assigned by nephrologists to renal items of damage (including chronic kidney disease, end-stage renal disease, dialysis and proteinuria) were similar to the scores assigned by rheumatologists (Fig. 2).

Fig. 1.

(A) Histogram showing the frequency of median scores by specialty. Distribution of nephrologists’ scores (black bars): variance = 3.90, skewness = 0.24, kurtosis = 2.31. Distribution of rheumatologists’ scores: variance = 4.29, skewness = −0.23, kurtosis = 2.17. (B) Histogram showing the frequency of median scores by region. Distribution of European investigators' scores (black bars): variance = 3.38, skewness = −0.03, kurtosis = 2.35. Distribution of American investigators’ scores (grey bars): variance = 5.06, skewness = 0.012, kurtosis = 2.12.

Fig. 2.

Median scores for renal items of damage by specialty. Nephrologists’ scores in black; rheumatologists’ scores in grey. (A) End-stage renal disease, (B) haemodialysis, (C) proteinuria <3 g/24 h, (D) proteinuria >3 g/24 h, (E) chronic kidney disease (any severity) and (F) glomerular filtration rate <50% predicted.

Damage rating by region

When participants were grouped by geographic region, European investigators favoured lower ratings compared with American investigators (skewness −0.03 vs 0.01) although the overall difference in median scores was small (P > 0.05) (Fig. 1B). European and American investigators selected an almost identical set of items of damage—including muscle atrophy with normal strength, alopecia, oral ulcerations, striae and bruising—to receive the lowest scores. Only two items of damage—blindness in second eye and haematopoietic malignancy—received a median score of 10 from the European investigators, whereas American investigators assigned a median score of 10 to 10 additional items of damage, including gut infarction, cystitis requiring cystectomy, stroke, transverse myelitis, solid tumour malignancy, myelodysplastic syndrome, dialysis and renal transplant.

Discussion

Damage assessment instruments in vasculitis do not use a differential weighting schema. Therefore, a patient with chronic sinusitis, cataracts and renal insufficiency would be assigned the same total damage index score as a patient with blindness, malignancy and end-stage renal failure. This does not capture our intuitive sense that some forms of damage exert a greater impact on patient health, outcomes and quality of life. This study clearly demonstrates that vasculitis experts, regardless of background and training, assign remarkably similar ratings to different types of disease- and treatment-related damage.

There was a substantial amount of agreement among investigators regarding the impact of specific forms of damage in vasculitis. Investigators tended to agree on which items of damage exert the greatest and least impact. When disagreement existed, it largely centred on items of damage that may benefit from surgical intervention (such as angioplasty or renal transplant) or items of damage that are not clearly linked to vasculitis (such as osteomyelitis or chronic peritonitis).

Although the rank order of the items of damage was similar among the subgroups, the ratings assigned by nephrologists to individual items of damage were lower overall than the ratings assigned by rheumatologists. Interestingly, this difference cannot be attributed to disagreement over the impact of renal-associated items of damage. The difference in absolute scores may reflect differing spectrums of patients with WG and MPA treated by each specialty. Similarly, European investigators tend to assign somewhat lower ranks than American investigators. The impact that these differences may have on treatment decisions and assessment bears further investigation.

In clinical trials of vasculitis, mortality alone is no longer a sufficient measure of success. New therapies must also demonstrate the ability to prevent the accumulation of damage and long-term morbidity. Ideally, a damage assessment instrument should serve three purposes: (i) differentiate between activity and damage, (ii) record the natural history of vasculitis and (iii) serve as an end point for clinical trials [5]. To accomplish the last goal, a damage index must be comparable across studies, sensitive to change and relevant to the individual patient. The development of a damage assessment index that takes into account the varying impact of different forms of damage is an important step towards accomplishing this goal.

The high level of agreement among investigators of diverse backgrounds regarding the severity of various forms of damage is a novel finding. The fact that consensus is so readily achieved provides justification for a new approach to damage assessment that incorporates a differential scoring system. Differential weighting by an expert panel has not previously been used by damage assessment instruments in vasculitis. The scores assigned to each item of damage by this panel of experts create a spectrum that could serve as the basis of a weighted damage index score. This approach differs markedly from the approach taken in the past, and has the potential of leading to a more meaningful damage assessment score. Since a single gold standard to assess damage does not exist, however, a weighted index would have to be validated against multiple surrogate end points (such as mortality, renal survival and quality of life) to demonstrate that the weighted index conveys more information that the unweighted version.

These data also add to the growing literature on increasing the scalability and utility of outcome measures in vasculitis [5, 6]. It is particularly interesting to note that patient-assigned ranks differ substantially from physician-assigned ranks of severity. For example, patients with vasculitis assign relatively low ranks to haemodialysis and oxygen dependency and high ranks to fatigue and weight gain [7]. Future comparisons of physician- and patient-reported outcomes may lead to new insights into the long-term impact of therapies and the definition of disease remission. Additional studies of patient-reported outcomes and the predictive value of damage items on mortality and organ failure are ongoing.

This exercise is one component of a larger effort by European and American investigators to standardize the assessment of vasculitis for clinical care and the conduct of clinical trials. This is an iterative process that benefits from an open approach; sharing these initial steps will improve the end product. As multinational trials in vasculitis become commonplace, such standardization across nationalities and specialties will become increasingly important.

Supplementary data

Supplementary data are available at Rheumatology Online.