The Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI): grading disease severity and assessing responsiveness to clinical change in epidermolysis bullosa

Abstract

Background

The lack of validated outcome measures for epidermolysis bullosa (EB) presents major barriers to evaluating disease severity and comparing the efficacy of therapies. The Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) was recently introduced as a valid and reliable instrument for EB; however, its interpretation for use in clinical practice and clinical trials is yet to be defined.

Objective

To assess the interpretability of the EBDASI in classifying patients according to disease severity and clinical response.

Methods

A total of 53 outpatients with EB at two interstate institutions were prospectively evaluated. At each visit, the principal dermatologist completed the EBDASI and global assessments of disease severity and change. Classifications for mild, moderate and severe disease using the EBDASI were determined using receiver operating characteristic curves. Minimal clinically important differences for the EBDASI activity subscale were calculated and compared with the standard error of measurement.

Results

Total EBDASI score ranges of 0–42, 43–106 and 107–506 corresponded to mild, moderate and severe disease respectively. Reduction in EBDASI activity scores of greater than 9 indicated clinically significant improvement. An increase of 3 in the activity score indicated deterioration.

Conclusion

The EBDASI is a responsive tool and may be useful in characterizing disease severity and response. The cut‐offs proposed in this study provide the first practical guide for interpreting the EBDASI, further supporting its use for longitudinal patient assessment and in clinical trials.

Introduction

Epidermolysis bullosa (EB) encompasses a heterogeneous group of rare inherited genodermatoses characterized by blistering and erosions of the skin and mucosa following minimal trauma. Currently, there are limited therapeutic options for EB, although recent advances in basic research have contributed to the expansion of new therapeutic possibilities. Multicentre randomized controlled trials of therapies for EB, however, are hampered by the lack of validated tools that can consistently quantify disease severity and capture therapeutic response. Developing such tools is important given that in the future, regulatory bodies will likely require objective thresholds of severity to justify the allocation of experimental and expensive treatments in EB. A validated assessment tool can also be useful for clinicians and patients to chart disease progression and optimize management in clinical practice.

Four scoring systems have been proposed as potential outcome measures for EB, including the Japanese indices,1 the Birmingham EB Severity (BEBS) score,2 the Instrument for Scoring Clinical Outcome of Research for Epidermolysis Bullosa (iscorEB)3 and the Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI).4 The former three have not been fully validated and are likely limited in their ability to detect changes in disease severity over time, due to their combination of features of potentially modifiable disease activity amenable to treatment and more permanent disease‐induced damage. For example, although the iscorEB states that only disease activity is scored, chronic damage components such as microstomia and chronically reduced renal and cardiac function are included in the score. The BEBS and Japanese indices similarly include damage components, such as scarring of hands, along with measures of activity, such as blistering and erosions. Separating activity and damage is important when measuring treatment response, as damage scores may increase despite the resolution of activity, thus negating the measured benefit.5, 6, 7

The EBDASI has separate scores for activity and damage and was found to have excellent validity and reliability in a previous study.4 Our aim was to evaluate how EBDASI scores could be interpreted in clinical practice, including how to classify disease severity and clinically significant change in disease activity, which is an important further step in validating an outcome measure.

Materials and methods

Study design

Patients attending 3‐monthly EB outpatient clinics were recruited at two separate sites: St George Hospital, Sydney (site 1), and The Royal Children's Hospital, Melbourne (site 2). Patients were eligible if they had a clinical and histopathological diagnosis of EB. All subtypes of EB and patients of all ages were included. Patients with EB acquisita were also eligible since EB acquisita has similar features of disease activity and damage. All patients were undergoing standard clinical care. This study was approved by Human Research and Ethics Committees at both sites (HREC/11/STG/234 in May 2013 and HREC/34022/A in May 2014) and was conducted in accordance with the Declaration of Helsinki principles. All patients, or parents if applicable, gave written informed consent prior to enrolment in the study.

At each patient visit, the principal investigator at each site (D.F.M. and J.C.S) assessed each patient using the EBDASI (Supplementary Fig. S1) and the Physician's Subjective Assessment of Severity (PSAS). Each investigator had significant expertise in EB and was familiar with scoring the EBDASI. Patients at site 1 who presented for multiple visits were additionally scored using the Physician's Subjective Assessment of Change (PSAC) at all subsequent visits. Participants older than 13 years of age completed the Quality of Life in EB (QOLEB) questionnaire at the baseline visit.

Outcome measures

Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI)

The EBDASI (Supplementary Fig. S1) is a partially validated EB‐specific instrument that assesses disease activity and damage at 12 cutaneous sites in addition to the scalp, mucous membranes, nails and other epithelialized surfaces.4 Total activity (out of 276) and damage (out of 230) are combined to give an overall score out of 506.

Physician's Subjective Assessment of Severity (PSAS)

Physician global assessments are often used in validation studies as external standards of disease severity.8, 9 The PSAS is a 3‐point Likert static scale that classifies patients as having mild, moderate or severe disease based on the physician's subjective global assessment of overall disease severity.10

Physician's Subjective Assessment of Change (PSAC)

The PSAC is a 3‐point Likert dynamic physician global impression of change that categorizes each patient as improved, stable or deteriorated when compared to the patient's previous visit.10 Prior to scoring, the principal investigators were encouraged to review clinical photographs from the patient's previous visit to assist recall of patients’ prior clinical condition; however, they were blinded to the patient's previous EBDASI and physician global assessment scores.

Quality of Life in EB questionnaire

The quality of life in EB (QOLEB) questionnaire is an EB‐specific quality of life instrument validated in adults with EB.11 The QOLEB contains 17 questions divided into an emotional scale (0–15 points) measuring the psychological impact of the disease, and the functional subscale (0–36 points) measuring the impact on activities of daily living.12 A higher score represents a more significantly affected quality of life.

Statistical analysis

All data were analyzed using spss Version 20 software (SPSS, Chicago, IL, USA). Data are presented as mean or median where appropriate.

Severity analysis

Epidermolysis Bullosa Disease Activity and Scarring Index total score cut‐offs for each physician‐derived severity group as determined by the PSAS were calculated using receiver operating characteristic (ROC) curves. Two ROC analyses were used to derive cut‐offs with the optimal balance of sensitivity and specificity for classifying mild disease (mild vs. moderate and severe) and severe disease (severe vs. moderate and mild). Scores falling between the mild and severe disease cut‐offs were designated as representing moderate disease. Separate site‐specific analyses were undertaken due to the possible influence of the study site and differing severity prevalence on EBDASI scores.

To investigate whether there was a relationship between quality of life and disease severity, Spearman rho correlation coefficients (ρ) were used to examine the convergent validity between QOLEB emotional and functional subscale scores and baseline total EBDASI scores which served as a proxy for disease severity.

Responsiveness analysis

Responsiveness is defined as the ability of a measurement tool to detect important change over time when it has occurred. Responsiveness can be assessed by multiple methods, including those that measure the magnitude of change (distribution‐based methods) or clinically meaningful change based on an external anchor (anchor‐based methods).13

Responsiveness was assessed by comparing the change in EBDASI activity scores between consecutive visits within each classification of change as determined by the PSAC. Standardized response mean (SRM) and standardized effect size (SES) are distribution‐based methods and were used to describe the magnitude of the change in activity and damage scores. SRM was calculated by dividing the mean change in scores by the standard deviation of the change scores. SES was calculated by dividing the mean change in scores by the standard deviation of the baseline score. Effect sizes were interpreted according to Cohen's criteria of large (>0.8), moderate (>0.5) or small (>0.2).13

Minimal clinically important difference

Minimal clinically important differences (MCIDs) in EBDASI activity scores were calculated using the anchor‐based method of ROC analyses. The sensitivity, specificity and percentage of correctly classified patients in discriminating improved patients from non‐improved patients were evaluated for all EBDASI activity scores. The final MCID chosen was based on the cut‐off with the highest accuracy of classification.10 The MCID for deterioration was calculated by examining the classification of deteriorated and non‐deteriorated patients.

To examine whether the MCIDs were greater than measurement error and thus indicative of true change, the standard error of measurement (SEM) was calculated. The SEM is calculated from the baseline standard deviation (SD) by the formula SEM = SD (1 – test‐retest reliability of the instrument)^½.14 The test‐retest reliability for the EBDASI activity score (ICC = 0.903) was derived from the previous validation study of the EBDASI.4

Results

Baseline demographics

A total of 53 patients with a range of EB subtypes were enrolled in the study; 40 patients with a mean age of 31.6 (±18.6 years) from site 1 and 13 patients with a mean age of 9.7 (±6.2 years) from site 2 (Table 1). At baseline, EBDASI scores in sites 1′s population ranged from 2 to 183 (median 35, interquartile range 91), compared to 4–104 (median 20, interquartile range 26) in site 2′s population (Supplementary Table 1). Twenty‐nine patients from site 1 who were scored on two or more visits were included in the responsiveness analysis.

Table 1.

Baseline demographics of patients with epidermolysis bullosa included in the study

	St George Hospital (site 1)		The Royal Children's Hospital (site 2)
	Severity analysis (N = 40)	Responsiveness analysis (N = 29)	Severity analysis (N = 13)
Mean age ± SD, years	31.6 ± 18.6	32.7 ± 18.9	9.7 ± 6.2
Age range, years	1–66	2–66	2–18
Female (%)	20 (50%)	16 (55%)	5 (38%)
EB subtype (%)
EBS, localized	19 (47.5%)	11 (38%)	4 (31%)
EBS, generalized severe	0 (0)	0 (0)	4 (31%)
EBS, autosomal recessive	0 (0)	0 (0)	1 (8%)
EBS, Ogna	1 (2.5%)	1 (3%)	0 (0)
JEB, generalized intermediate	5 (12.5%)	4 (14%)	0 (0)
DDEB, generalized	5 (12.5%)	5 (17%)	4 (31%)
RDEB, generalized severe	8 (20%)	7 (24%)	0 (0)
EBA	2 (5%)	1 (3%)	0 (0)

EB, epidermolysis bullosa; EBS, epidermolysis bullosa simplex; JEB, junctional epidermolysis bullosa; DDEB, dominant dystrophic epidermolysis bullosa; RDEB, recessive dystrophic epidermolysis bullosa; EBA, epidermolysis bullosa acquisita; SD, standard deviation.

Severity analysis

Each patient included in this analysis had 1–5 visits, resulting in a total of 100 assessments from 53 patients (87 assessments from 40 patients at site 1; 13 assessments from 13 patients at site 2). At site 1, the median EBDASI scores for each severity classification was 14 (range 1–68) for the mild group, 94 (range 40–141) for the moderate group and 139 (range 82–196) for the severe group (see Supplementary Figs S2 and S3 for patient examples of severe and moderate disease). At site 2, the median EBDASI scores were 16 (range 4–34) for the mild group and 79 (range 54–104) for the moderate group, with no patients classified as severe (Table 2).

Table 2.

Distribution of Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) total scores according to the physician's subjective assessment of severity at two study sites

Physician's subjective assessment of severity (PSAS)	St George Hospital (site 1)		The Royal Children's Hospital (site 2)
	N (%)	EBDASI range (median)	N (%)	EBDASI range (median)
Mild	39 (45%)	1–68 (14)	11 (85%)	4–34 (16)
Moderate	20 (23%)	40–141 (94)	2 (15%)	54–104 (79)
Severe	28 (32%)	82–196 (139)	0 (0)	–

EBDASI, Epidermolysis Bullosa Disease Activity and Scarring Index; IQR, interquartile range; PSAS, Physician's Subjective Assessment of Severity.

Receiver operating characteristic analyses to determine severity score cut‐offs were undertaken using data from site 1, given its wider distribution of EBDASI scores in each severity category. ROC analyses indicated optimal EBDASI score cut‐offs of 42 to differentiate mild disease from moderate or severe disease (sensitivity, 98%; specificity, 97.4%; 98% of patients correctly classified) and 107 to differentiate severe disease from mild or moderate disease (sensitivity, 96.3%; specificity, 88.3%; 92% of patients correctly classified). Therefore, EBDASI scores were graded into mild (0–42), moderate (43–106) and severe (≥107), which corresponded with the score ranges at both sites (Fig. 1).

Figure 1.

Distribution of Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) total scores (out of 506) according to physician's subjective assessment of severity at the two study sites. Dashed lines correspond to EBDASI disease severity cut‐offs determined from Receiver Operating Characteristic curve analysis using data from site 1 (mild disease severity cut‐off of 42, severe disease severity cut‐off of 107).

Twenty‐nine patients completed the QOLEB questionnaire at the baseline visit. Correlations between QOLEB functional and emotional subscales with baseline total EBDASI scores was moderate, indicating good convergent validity between quality of life and disease severity (QOLEB functional score with total EBDASI score, p = 0.678, P < 0.01; QOLEB emotional score with total EBDASI score, p = 0.48, P < 0.01).

Responsiveness analysis

A total of 54 pairs of visits from 29 patients at site 1 were included in the responsiveness analysis. Of these, 22 (41%) patient visits were classified as stable, 19 (35%) as improved and 13 (24%) as deteriorated. The median time between patient visits was 4 months (range 1–27 months). Patients who were classified as improved or deteriorated had a greater mean change in the EBDASI activity score and moderate effect sizes compared with patients in the stable group (Table 3). The responsiveness of the damage score was less in all change groups compared with that of the activity score.

Table 3.

Magnitude of change in Epidermolysis Bullosa Disease Activity and Scarring Index activity and damage scores between patient visits by physician classification of change groups at site 1

	Stable (N = 22)		Improved (N = 19)		Deteriorated (N = 13)
	Activity	Damage	Activity	Damage	Activity	Damage
Mean change ± SD	−0.7 ± 5.4	1.4 ± 7.9	−7.9 ± 8.7	−4.9 ± 9.6	7.0 ± 11.4	9.4 ± 16.9
Standardized response meana	−0.13	0.18	−0.91	0.51	0.61	0.56
Standardized effect sizeb	−0.05	0.03	−0.57	0.12	0.49	0.32

Changes in Epidermolysis Bullosa Disease Activity and Scarring Index activity and damage scores were compared between the change groups of stable, improved and deteriorated, as determined by the principal investigator at site 1 based on the physician's subjective assessment of change compared to the previous visit.

^aStandardized response mean calculated as mean of score change divided by the standard deviation of the score change.

^bStandardized effect size calculated as mean of score change divided by the standard deviation of the baseline score.

Minimal clinically important difference

MCIDs for improvement and deterioration were calculated for EBDASI activity scores using ROC analyses. The ROC (AUC 0.83) revealed that a 3‐point decrease in the EBDASI activity score was 75% sensitive and 73% specific for improvement and corresponded with approximately 76% of patients being correctly classified as improved or non‐improved. However, a 9‐point decrease in the EBDASI activity score had higher specificity (97%) and correctly classified 82% of patients (Table 4). For deterioration, a 3‐point increase in the EBDASI activity score was determined from the ROC (AUC 0.80) to be 45% sensitive and 85% specific for deterioration. This threshold constituted the highest percentage of correct classification, with 75% of patients correctly classified. One standard error of measurement (SEM) was calculated to be 3.

Table 4.

Responsiveness of the change in Epidermolysis Bullosa Disease Activity and Scarring Index activity scores in discriminating improved from non‐improved patients according to physician classification of change

Change in EBDASI activity score	Sensitivity (%)	Specificity (%)	% Correctly classified
−3	87.5	72.2	77.8
−4	75.0	75.0	75.9
−5	68.8	80.6	77.8
−6	62.5	80.6	74.1
−7	56.3	86.1	75.9
−8	50.0	91.7	77.8
−9	43.8	94.4	79.6
−10	37.5	97.2	77.8

For each change in EBDASI activity score between consecutive visits, the sensitivity, specificity and percentage of patients correctly classified into improved or non‐improved groups were calculated. Patients were classified as improved or non‐improved (stable or deteriorated) based on the physician's subjective assessment of change by the principal investigator at site 1.

EBDASI, Epidermolysis Bullosa Disease Activity and Scarring Index.

Discussion

To date, outcome measures for EB, such as the Japanese indices, BEBS and iscorEB have only been partially evaluated, with the EBDASI demonstrating better inter‐rater and intra‐rater reliability compared to the BEBS in our group's previous comparative study.4 This study further assesses the validity of the EBDASI and provides a guide to translating EBDASI scores into clinically meaningful measures of disease severity and change.

Using multiple patient visits and by comparing total EBDASI scores with physician‐graded severity, we defined specific cut‐off points to categorize mild, moderate and severe disease. The recommended severity categories according to the total EBDASI score are as follows: mild (0–42), moderate (43–106) and severe (≥107). These cut‐offs were derived using data from one site, as physician assessments of severity are subjective and thus must be considered in site‐specific analyses. Although our data are limited in that we were unable to determine severity cut‐offs from the second site for comparison due to small patient numbers and a smaller range of severity, our proposed cut‐offs correlated with the data from the second site. Defining objective thresholds of severity is useful for multicentre clinical trials, particularly as new and expensive treatments may only be justified for severe disease. Standardized EBDASI severity cut‐offs will also facilitate physicians who are not experts in the disease to measure and communicate severity consistently in EB.

Total EBDASI scores were lower on average at site 2 than at site 1, where the mean age was significantly higher. The difference in average total EBDASI scores between the two sites may be due to the greater proportion of milder EB subtypes and lack of more severe subtypes, such as recessive dystrophic epidermolysis bullosa, recruited at site 2. Additionally, certain damage features of EB such as skin cancer and pseudosyndactyly take time to develop and likely contributed to lower damage scores. Consequently, a low EBDASI score in a child must be interpreted with caution, as it may not necessarily be consistent with the categorization of mild disease according to the severity cut‐offs derived in this study. Ideally, different cut‐offs based on age could be developed in future studies with a larger cohort of patients.

Using a quality of life functional score, we were able to correlate severity with quality of life. It was found that the association was weaker for the emotional component of the score as opposed to the functional component. The non‐linear relationship between quality of life and disease severity has been reported for numerous dermatological diseases, including EB.11, 15, 16 Severity from the patient's view may be influenced by psychological factors, such as coping skills and social circumstances, in addition to the specific physical lesions measured by the EBDASI. Holistic assessment of disease severity in clinical trials should therefore include both patient‐reported and physician‐reported assessments.

A clear difference was found in the mean change in EBDASI activity scores between patients who were classified as improved or deteriorated compared with patients who were classified as stable, suggesting the EBDASI activity score is sensitive to change. Damage scores generally had poorer sensitivity to change than activity scores, most likely because features of damage tend to be more static. Our study did find changes in the damage scores in patients seen 3 months apart (mean 4.9 point decrease in improved patients and mean 9.4 point increase in deteriorated patients), which could possibly relate to secondary scarring after changes in activity, changes in more dynamic features of damage such as erythema and hyperkeratosis or intra‐rater variability. A study with longer follow‐up could further characterize the relationship between damage and activity. This would provide useful information to guide patients about how activity and damage may change with treatment.

Although the sensitivity of a score can be measured by the magnitude of change within the cohort, in clinical practice, it is more useful to determine the clinical relevance of change in a single patient. It has therefore been recommended that MCIDs should be calculated using anchor‐based methods which utilize external criterions of change, as opposed to distribution‐based methods examining the magnitude of change, which produce varying results depending on the statistical sample.13, 17, 18 Given that no formal consensus exists on the optimal method to determine responsiveness, we used both anchor‐ and distribution‐based methods. In this study as in the literature, physician global assessments were used as anchors due to the absence of ‘gold standard’ markers of disease severity in EB, based on the rationale that expert physicians are well‐positioned to rate change having qualitatively assessed change in a large number of patients.8, 9 Using ROC analyses to determine MCID estimates, we found that MCIDs with higher specificity relative to sensitivity generally had higher rates of correct classification. For the purposes of a clinical trial, higher specificity may be more desirable than sensitivity so that patients who are stable are not misclassified as responders.10, 19 This is particularly relevant in EB as high placebo response rates due to spontaneous reductions in wound size in stable patients has been reported.20, 21 MCIDs in this study were therefore chosen based on the highest percentage of correctly classified patients associated with the highest specificity.

It should be noted that MCIDs can vary depending on population and context. We acknowledge that the single MCIDs in this study are unlikely to accurately detect true change in all patients with EB, particularly given the heterogeneity of the disease. Ideally, multiple MCID values should be determined using a range of physician and patient anchors to confirm appropriate thresholds in different groups, e.g. according to baseline severity. Although only one physician anchor was used in our study, scoring was done by a consistent expert physician, which increases confidence in the validity of our derived thresholds.

Further limitations of our study include the small sample size due to the rarity of EB and practical difficulties in fully undressing patients, particularly in children for whom this process is often distressing, for scoring. The external validity of our proposed MCIDs may also be limited by the current lack of effective therapies for EB, which contributed to the small changes in severity and EBDASI scores seen in our study population. New treatments are expected to cause greater change in severity and will therefore require more stringent MCID values. Nevertheless, the MCIDs derived in this study may help to define the lower bound for response against which to assess the efficacy of treatments in the future.

In conclusion, this pilot study provides the first guide to interpreting EBDASI scores, which may be useful to optimize patient management and interpret the efficacy of treatments. For example, a significant change in EBDASI scores between patients’ visits may help to trigger a clinical review, prompting improved wound care or supportive measures to be implemented, or conversely, highlight treatments that are effective. While studies in different populations are necessary to continually refine the presented thresholds, our study provides preliminary indications of the thresholds of clinically important change and supports the use of the EBDASI as a valid, reliable and sensitive outcome measure.

Supporting information

Figure S1. The 4‐page Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) quantifies disease severity by scoring disease activity and damage in five sections.

Figure S2. (a–e) Photographic examples of patient (a: abdomen; b: torso; c: lower limbs; d: nail loss in right foot; e: partial syndactyly in left hand) classified as ‘severe’ (EBDASI total score >107) according to the Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) and physician classification of severity.

Figure S3. (a–b) Photographic examples of patient (a: lower limbs; b: back) classified as ‘moderate’ (EBDASI total score between 43 and 106) according to the Epidermolysis Bullosa Disease Activity and Scarring Index (EBDASI) and physician classification of severity.

Table S1. Distribution of Epidermolysis Bullosa Disease Activity and Scarring Index scores at baseline according to study site and subgroup analysis.