A Core Domain Set For Hidradenitis Suppurativa Trial Outcomes: An International Delphi Process

Abstract

Background

There is no consensus on core outcome domains for hidradenitis suppurativa (HS). Heterogeneous outcome measure instruments in clinical trials likely leads to outcome reporting bias and limits the ability to synthesise evidence.

Objectives

To achieve global multi-stakeholder consensus on a Core Outcome Set (COS) of domains regarding what to measure in clinical trials for HS.

Methods

Six stakeholder groups participated in a Delphi process which included five anonymous e-Delphi rounds and four face-to-face consensus meetings to reach consensus on the final COS. The aim was for a 1:1 ratio of patients: Health Care Professionals (HCPs).

Results

A total of 41 patients and 52 HCPs from 19 countries in four continents participated in the consensus process which yielded a final COS that included five domains: pain, physical signs, HS specific quality of life, global assessment and progression of course. A sixth domain, symptoms, was highly supported by patients and not by healthcare professionals but is recommended for the core domain set.

Conclusions

Routine adoption of the COS in future HS trials should ensure that core outcomes of importance to both patients and HCPs are collected.

INTRODUCTION

Hidradenitis Suppurativa (HS) is a chronic, inflammatory skin disease with an estimated prevalence of 0.1–4 % worldwide.^1–4 The primary lesions are inflammatory nodules that may develop into abscesses and sinus tracts with subsequent scarring, affecting flexural sites such as the axillae and groins on a recurrent basis^5,6 Lesions of HS are typically described by patients as painful boils which, along with associated pus and odour, may produce a large impact on quality of life.^7–11

Interventions for HS are diverse and include topical treatment, systemic antibiotics, retinoids, immunomodulatory oral therapy, biologics, laser therapy and surgery.^12,13 The level of evidence for existing treatments is low, suggesting a particular need for more clinical trials in HS.¹³

Validated outcome measure instruments are necessary to ensure that study results are comparable and that, as a consequence, patients receive the most effective interventions. For HS, numerous outcome measure instruments exist, with a total of 30 instruments used in the 12 randomised controlled trials included in the recent Cochrane review.^{13, 14} Heterogeneity of outcome measure instruments in HS limits evidence synthesis, including meta-analysis,¹³ and likely leads to outcome reporting bias because of selective reporting of more favourable outcomes.¹⁵ Because no consensus on core outcomes for HS exists, researchers use various instruments, which may or may not be valid. Furthermore, current instruments emphasize clinical features with limited incorporation of patient-reported outcomes, despite recommendations emphasizing the importance of the patient perspective in outcomes research.¹⁶

To tackle these issues, the HIdradenitis SuppuraTiva cORe outcomes set International Collaboration (HISTORIC) was formed as a collaboration between the International Dermatology Outcome Measures (IDEOM) initiative, the Cochrane Skin Group - Core Outcome Set Initiative (CSG-COUSIN) and Zealand University Hospital, Roskilde. The first HISTORIC goal was to develop a Core Outcome set (COS) of domains that is relevant to all major stakeholders, including patients, to be recommended for use in all subsequent HS clinical trials.^17–19

We performed and moderated an international multi-perspective Delphi consensus project with a scope to develop a COS suitable for all HS clinical trials. The COS is intended to suit all types of interventions for all HS patients, regardless of setting or mode of administration.

METHODS

The study is reported in accordance with the newly developed Core Outcome Set STAndards for Reporting (COS-STAR).²⁰ A detailed description of the methods can be found in our protocol article.²¹ Methodological guidance was followed from Core Outcome Measures in Effectiveness Trials (COMET)¹⁹ and Outcome Measures in Rheumatology (OMERACT).²² We were also guided by the Harmonizing Outcome Measures for Eczema (HOME) roadmap.²³ The methodology involves a stepwise approach for the development of a COS. The first step is to identify which domains one should measure and report in all clinical controlled trials of a specific condition (what to measure: the core domain set).¹⁵ The second step is to identify the instruments that should be used to assess these domains (how to measure: the core outcome measurement set).^17,19 The present study achieved the first step, determining what to measure.

Participants

The involvement of multiple stakeholders for the development of a COS is strongly recommended by methodologists.^17,19,23,24 Six groups of stakeholders were invited to participate in our development process: patients, dermatology HS experts, surgical HS experts, HS nurses, industry representatives and drug regulatory authorities. Patients were analysed as one stakeholder group and the remaining stakeholder groups were combined into a second group referred to as Health Care Professionals (HCPs). The HCP group contained one representative of a drug regulatory authority (the European Medicines Agency) and one industry representative (table 3). Other drug regulators and pharmaceutical companies with an interest in HS were contacted but chose not to contribute. The aim was for a 1:1 ratio of patients: HCPs.

Table 3.

Demographics of Hidradenitis Suppurativa patient and HCP participating in the e-Delphi

Patient participants	e-Delphi Round 1	Attrition Round 2	Attrition Round 3	Attrition Round 4	Attrition Round 5
Total invited (n)	58
Total accepted invitation (n)	45
Total complete answers (n)	41	−3	−3	−2	−1
Age, mean (SD)	41.2 (10.7)
Disease duration, years, mean (SD)	19.4 (11.0)
Participants per country (n)
Australia	2
Belgium	1				−1
Canada	6		−1
China	1
Denmark	2
France	2
Germany	1
The Netherlands	1		−1
Slovakia	1
United Kingdom	11	−1	−1	−1
USA	13	−2		−2
HCP participants	e-Delphi Round 1	Attrition Round 2	Attrition Round 3	Attrition Round 4	Attrition Round 5
Total invited (n)	80
Total accepted invitation (n)	59
Total complete answers (n)	52	−4	0	−2	0
Age, mean (SD)	51.4(10.6)
Participants per stakeholder (n)
Dermatologists	26	−2		−1
Dermatologists/Dermatologic surgeons	18	−2		−1
Surgeons	2
Nurses	4
Industry representatives	1
Drug regulatory authorities (EMA)	1
Participants per country (n)
Australia	3	−1
Belgium	1
Bulgaria	1
Canada	3
China	1
Denmark	5
Germany	1
Japan	2
Malaysia	2
The Netherlands	3
Norway	2
Poland	1
Spain	2
Sweden	2
Taiwan	1			−1
United Kingdom	3
USA	19	−3		−1

EMA, European Medicines Agency; HCP, Health Care Professional; SD, standard deviation.

Patients were identified through patient associations and via dermatologists with a special interest in HS in countries without a formal patient association. HCPs were identified from the community of HCPs working with HS patients. A clinical background of at least five years of experience managing HS was required for all HCPs and publications on HS or participation in scientific meetings on HS was required for dermatologists.

Information sources

Identification of initial list of candidate items and potential core domains

The initial list of candidate items was obtained in a three-step manner:

1: Systematic review of literature

A recent systematic Cochrane review on interventions for HS and another systematic review on outcome measure instruments reviewed the validation evidence for existing instruments and mapped them according to potential domains.^13,14

2: Qualitative studies

Semi-structured interviews and focus groups were conducted at the Department of Dermatology, Zealand university hospital, Roskilde, Denmark and Department of Dermatology, Penn State College of Medicine, Hershey, Pennsylvania, USA. Purposive sampling of a wide diversity of age groups, sex, treatments received and disease severities was employed. Inclusion of patients ceased when saturation was achieved, defined as when no new knowledge was obtained from the subsequent interviews. Patients were identified primarily among those undergoing treatment at the two Departments of Dermatology. Eligibility was based on a confirmed diagnosis of HS and willingness to participate.

All interviews were tape recorded and transcribed verbatim and initially examined for units of meaning, coded as items and grouped into categories. Qualitative interviews do not require ethical approval in Denmark. In USA, the project was approved by the institutional review board of the Penn State College of Medicine.

The two lists of candidate items generated from the Danish and US qualitative studies were combined into one patient-generated item list.

3: Identification of items of importance to HCPs

To identify outcomes of importance to HCPs an item generation e-Delphi round zero was conducted among the HCP stakeholders. Participants were first provided with background information on the rationale for development of a HS COS. They were then asked to list all items that they considered important regarding HS, with items being related to any aspect of the disease, or treatment of the disease.

The steering group reviewed all items suggested by the HCPs and produced a preliminary list of candidate items by combining the results from the systematic reviews, the qualitative studies and the HCPs item generation survey.

Consensus process

A summary of the consensus process is in Figure 1. An international steering group (the first 12 authors of this manuscript) consisting of researchers, clinicians and a patient research partner guided development of the COS.

Fig. 1.

Study summary. See text for details

Methods to reach consensus on the core domain set

An anonymous Delphi approach was applied to make sure that the views of all participants were obtained. The e-Delphi survey was delivered using DelphiManager^® (round one and two) and SurveyMonkey^® (round three to five) software. A unique identifier code allowed identification of participants completing all rounds of the Delphi survey. Only participants who had completed the previous round of the survey were invited to participate in subsequent rounds. All surveys were pilot-tested by at least two members of the steering group, including the patient representative, and at least two additional panel members.

Items/domain scoring

Participants were asked to score each item/domain using a modified scale from the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) from one to nine. Explanation was provided that scores of one to three are ‘not important’, scores of four to six are ‘ important but not critical’ and scores of seven to nine are ‘critical’ to include.²⁵ From round two of the Delphi onwards, participants were provided feedback in the form of their own scores in the previous round and the aggregate scores from the previous round, sub-divided into the patient and HCP groups.

Definition of consensus

Pre-specified consensus endpoints are outlined in Table I.

Table 1.

Definition of consensus

Consensus classification	Description	Definition
Consensus in	Consensus that the item/domain should be included in the core domain set	70% or more participants scoring 7 to 9 AND <15% participants scoring 1 to 3
Consensus out	Consensus that the item/domain should not be included in the core domain set	70% or more participants scoring 1 to 3 AND <15% of participants scoring 7 to 9
No consensus	Uncertainty about importance of the item/domain	Anything else

E-Delphi round one and two

Participants were provided with background information explaining how the candidate items were identified and were then asked to rate each of the items listed, based on their importance in being measured as an outcome in all clinical trials for HS. Participants were also asked to suggest items not represented in the list. Items suggested were reviewed by the steering group to ensure they represented new items and all items were carried forward to round two. In round two, the number of participants who ranked each item and the distribution of scores (as percentages of the total) by stakeholder group from round one were shown graphically in the survey and participants were asked to consider responses from other panel members and to re-score all items. All items were carried forward to consensus meeting I and II and e-Delphi round three.

HISTORIC consensus meeting I and II

After the first two e-Delphi rounds, participants were invited to take part in two in-person consensus meetings (Vienna, September, 2016 and New York, October, 2016). In these meetings, patients and HCPs collaborated on nominating items for exclusion based on round 2 e-Delphi results and grouping remaining candidate items into domains. The process of defining domains was achieved in small groups using nominal group theory. Prior to the nominal group exercise, participants were made aware that items could form their own stand-alone domain or be collected into an umbrella domain, if the items were sufficiently congruent and capable of being measured by a single instrument.

As only a sub-set of the e-Delphi group was able to attend the in-person meetings all decisions taken at the meetings required confirmation by the larger HISTORIC project group in a subsequent online confirmation survey before implementation. Prior to completion of the survey, e-Delphi panel members were provided with a summary of the in-person meeting discussions. A detailed description of consensus meeting I and II and the online confirmation survey has been published.²⁶

E-Delphi round three

In round three, the items were shown under their newly designated domain, following the work from the in-person meetings. Items that were marked for exclusion at the consensus meetings were shown at the end, under a heading of ‘marked for exclusion’. Items that were marked for exclusion and did not reach ‘consensus in’ were not carried forward to round four.

HISTORIC consensus meeting III

After the third e-Delphi round, it was noted that some items and domains were considered ‘critical’ to include in the COS only by patients or HCPs but not both. These discordant items were discussed by patients and HCPs at a third in-person consensus meeting (Copenhagen, February, 2017). This discussion was followed by non-binding voting.

E-Delphi round four and five

In round four, participants voted at the domain level for the first time and voted again on items within each domain that had still not reached clear ‘consensus in’. A summary of the discussion and voting results from consensus meeting III were provided, as well as the results from e-Delphi round three. In round five, participants voted on two domains for which consensus was nearly achieved (more than 67 % combined critical votes) to determine whether these should be included in the COS.

HISTORIC consensus meeting IV

The results of e-Delphi rounds one to five and HISTORIC in-person consensus meetings I-III were presented at the annual IDEOM meeting in Washington, DC in May 2017. All meeting participants were asked if they considered the HISTORIC HS COS process to be methodologically robust and inclusive and if the project had developed an appropriate COS. Consensus for the final core domain set was defined as >70% of all participants voting yes to both these questions.

RESULTS

Participants

Patient characteristics from the qualitative studies are found in Table II and demographics of all Delphi participants are found in Table III. A total of 42 patients participated in the qualitative studies and a total of 93 (41 patients and 52 HCPs) from 19 countries in four continents participated in the first round of the e-Delphi. In the last round of the e-Delphi, 78 participants continued to take part, a 16% attrition rate. Of the 15 drop outs, nine were patients and six were HCPs, while nine were from North America and six were from other continents..

Table 2.

Hidradenitis Suppurativa patient characteristics, item generation interviews

Variables	North American patients (n=21)	Danish patients (n=21)
Age, years	46.8 (13.7)	37.9 (10.8)
Females, n (%)	16 (76%)	13 (62%)
Ethnicity
Non-Hispanic white, n (%)	13 (61%)	21 (100%)
Hispanic, n (%)	3 (14%)	0
Black, n (%)	2 (9%)	0
Asian, n (%)	1 (4%)	0
Mixed ethnicity, n (%)	2 (9%)	0
Hurley stage
1, n (%)	0	3 (14%)
2, n (%)	12 (57%)	12 (57%)
3, n (%)	9 (43%)	6 (29%)
Disease duration, years	20.5 (12.7)	19.8 (10.0)

Data is presented as mean (SD) unless other is stated.

^*Missing values for three of the Danish patients

SD, standard deviation.

Candidate outcome items and domains

A list of all 56 items initially included in the Delphi exercise is found in Table IV. The systematic review identified 16 potential candidate items, 33 additional items were identified by patients in the qualitative studies and the HCP item generation survey identified 7 further items (Table IV). One item suggested by a patient participant in the first e-Delphi round (number of chronic areas) was judged to represent a new outcome and added to the list of candidate items in round two and subsequent rounds. Item numbers 1, 4, 7, 8, 10, 13, 16, 30, 39, 41, 45, 47, 53 (Table IV) were nominated for exclusion following consensus meeting I and II,²⁶ did not reach ‘consensus in’ in e-Delphi round 3 and were therefore excluded. One item, ‘Pain’ was ranked so highly by both patients and HCPs that it was nominated to form a domain in its own right. Other domains were formed by collecting together similar items during the nominal group exercises, as detailed in the report of consensus meetings I and II.²⁶

Table 4.

List of all items initially included in the Delphi exercise (item 57 included after the first round). The Help text was shown with each item in the e-Delphi. The item numbers were generated at random before round one of the e-Delphi.

Item number	Item name	Help-text
1*	Biomarkers	Measures of disease presence or activity in blood samples
2	Drainage	Secretion, blood, stains, suppuration
3	Edema	Swelling of the skin
4	Economic burden	Economic burden to the patient related to the disease (e.g., doctor appointments, surgery, medication), management (e.g., bandages, pads, or diet), time lost
5	Coping	Being able to handle (cope with) having the disease
6	Odour	Unpleasant odour
7†	Satisfaction with treatment	Satisfaction with effectiveness; time spent on treatment
8	Adverse effects of surgical treatments	All types of side effects from surgical treatments (e.g. bleeding, infection, contractures)
9†	Number of cysts	Number of sac-like pockets under the skin which contain fluid or debris from the skin
10	Comorbidities	Associated diseases e.g. metabolic syndrome, PCOS or other inflammatory diseases
11	Intimacy	Impact on sexual having desire or feeling desired, pain during sexual activity, abstaining from sex, fear of being rejected
12	Ability to work or study	Ability to work or study, ability to gain or keep employment, influence on type of job or study, time off from work or study, impact on career
13	Adverse effects of medical treatments	All types of side effects from medical treatments
14†	Number of non-inflamed nodules	Number of skin colored nodules which may not be painful or tender
15	Itch	Itch
16	Self-treatment, not prescribed	Self-treatment which is not prescribed (e.g. self-incision to obtain pain relief, placing ice cubes or warm compresses on boils
17†	Number of abscesses	Number of collections of pus (sterile or infected)
18†	Total lesion count	Total number of all types of lesions
19	Psychosocial functioning	Feelings of being accepted by others, nervous to be in public, withdrawn from relationships
20	Scarring from HS	Scar formation in involved areas
21	Need for treatment and bandages	Requirements for prescribed treatment, e.g. acute treatment, pain killers, topic treatment, in-hospital treatment and bandages
22	Surface area	Area of the skin surface involved
23	Impact on close relationships	Impact on relationship to partner or family member, neglect of family, poor understanding of disease by family
24†	Time to recurrence	Time to reappearance of activity, such as after surgery or after ending medical therapy
25	Emotional well-being	Feelings of powerlessness, embarrassment, low self-esteem
26†	Decreased mobility	Decreased mobility, skin tightness, may be associated with restrictions in exercising, walking, reaching out, standing, sitting, activities of daily living (e.g. housework)
27	Satisfaction with social roles	Satisfaction with oneself as a partner, parent, family member, friend, or colleague
28*	Progression of course	Worsening of disease, prevention of worsening
29	Recreation and leisure activity	Interference with leisure/recreational activities (e.g., sports, do-it-yourself, playing instruments, scouting, hiking or outdoor life). Interference with planning of such activities
30*	Dyspigmentation*	Changes (lighter or darker) to the normal colour of your skin
31	Anatomic location	Body areas and number of body areas involved
32†	Number of inflamed nodules	Number of red, painful or tender nodules
33	Psychological functioning	Feelings of depression, apathy, loneliness, suicidal thoughts. Feelings of irritation, anxiety, stress
34†	Health related Quality of life	Perceived physical, mental and social health over time
35†	Number of fistulae	Number of connections to skin surface
36†	Pain	Pain
37	Cognition	Impact on concentration (e.g. at work or at school, or in leisure activities)
38	Fatigue	Physical weariness sometimes combined with mental weariness
39	Cosmesis	Visual appearance of a person’s skin from his/her own perspective related to the disease and surgery for the disease
40†	Patient global evaluation	Overall assessment of the disease from the perspective of the patient himself or herself, alone and without the influence of anyone else
41	Washing or Bathing	Ability to wash or bathe oneself; having to frequently wash or bathe oneself
42*	Ulceration	Absence of upper layers of the skin forming an ulcer
43†	Physician global evaluation	Overall assessment of the disease from the perspective of the physician alone
44†	Number of sinus tracts	Number of tunnel-like connections between lesions
45	Scarring from surgery	Scars resulting from surgery
46*	Compliance*	A patient’s adherence to a recommended course of treatment
47	Satisfaction with care	Access to care, satisfaction with the doctor’s knowledge of disease, quality of care, feeling supported by medical personnel
48	Independence	Need to be independent, not to dependent on others
49†	Time to post-op recovery	Time to healing after surgery
50	Clothing restrictions	Impact on choice of clothing (e.g. choosing clothes that do not irritate lesions, that cover lesions, that cover stains
51	Flare frequency	Frequency of flares
52†	Inflammatory lesion count	Total number of all red, painful or tender lesions (abscesses or inflamed nodules)
53*	Comedones	Appearance of small “blackheads” on the surface of the skin formed by the blockage of pores
54	Constitutional symptoms	The experience of one or more symptom(s) associated with the development of new lesions (e.g. fatigue, fever-like sensation, headache)
55*	Erythema	Redness of the skin
56	Sleep-disturbance	Difficulty sleeping, inability to sleep, poor quality of sleep
57	Number of chronic areas	Number of chronic areas open for more than 6 weeks

^*items generated in health care professional item generation survey.

^†items generated in review of literature and re-found in patient interview qualitative studies.

Remaining items were generated in the patient interview qualitative studies.

The final core domain set

The final core domain set is illustrated in Figure 2. After the last e-Delphi and the final consensus meeting the participants agreed to include five domains in the HS COS for clinical trials: pain, physical signs, HS specific quality of life, global assessment and progression of course. The domains are further defined by the items that were fused in the process of creating domains (Fig. 2). A ‘symptoms’ domain, containing the items ‘drainage’ and ‘fatigue’, was strongly supported by patients but did not quite reach our a priori definition of ‘consensus in’ from the perspective of HCPs.

Fig. 2.

The final core domain set in an adapted OMERACT onion model. (a) Inner ring: the core set, domains (in black) and items (in white) that reached ‘consensus in’ for patients and Health Care Professionals (HCPs). (b) Middle ring: domains (in black) and Items (in white) that reached ‘consensus in’ for patients or HCPs. (c) Outer Ring: Items that did not reach ‘consensus in’, but were marked at consensus meetings for the research agenda or important in specific trials.

78.6 % of consensus meeting IV participants considered the HISTORIC COS process to be inclusive and methodologically robust and 82.3% felt that an appropriate COS had been achieved and voted to ratify the HISTORIC COS.

Protocol deviations

The HISTORIC consensus meeting III was not planned a priori but was added after round three to allow further discussion of some items and domains where disagreement between patients and HCPs was identified. E-Delphi round five was added after round four to discuss domains where consensus had nearly, but not quite, been achieved. This resulted in inclusion of the ‘progression of course’ domain in the final COS, but did not affect lack of consensus between HCPs and patients regarding the symptoms domain. As no items or domains ever reached the predefined ‘consensus out’ rule in any rounds, the process focused only on the predefined ‘consensus in’ rule. By comparing the proportions voting critical among HCPs and patients, when both proportions were above 70% threshold, these items/domains were considered part of the COS.

DISCUSSION

We used a rigorous, iterative and inclusive approach to identify consensus among an international group of HS patients and HCPs, producing five core domains relevant for all types of clinical trials for HS, namely pain, physical signs, HS specific quality of life, global assessment and progression of course. There was close agreement among all stakeholders to include the final five domains in the COS. Based on our protocol, the symptoms domain is not included as a core domain because it only reached ‘consensus in’ from the patient perspective and support was insufficient from HCPs. However, the HISTORIC Steering group reflected that, because symptoms is a patient reported domain and was considered critical by our patient participants, the patient view supersedes that of HCPs in this instance. As a result, the HISTORIC Steering group agreed that the symptoms domain should be included in step two of the COS process to search for a suitable instrument for the domain.

Limitations to the present study include that our aim to involve a 1:1 ratio of patients: HCPs was not completely reached and that we did not succeed in involving participants from the continents of Africa and South America in the project.

The HISTORIC initiative has begun the process to develop a COS for HS trials. The implementation of a COS for HS clinical trials should improve the interpretation and comparison of future studies testing interventions for HS and reduce the risk of outcome reporting bias and heterogeneity across studies. After achieving consensus on what to measure in HS clinical trials, the next step for the HISTORIC initiative will be to reach consensus on the outcome measurement instruments best suited to measure each of the core domains in the COS.

In conclusion, our present study reports on the robust development of a comprehensive COS for use in all trials assessing interventions for HS. The final COS includes five core outcome domains agreed by both patients and HCPs and a sixth domain, symptoms, is recommended by the HISTORIC Steering Group because it is a patient reported domain that received strong support from our patient stakeholder group. The routine adoption of this COS in future HS trials should ensure that outcome domains of importance to both patients and HCPs are included and reported.

What’s already known about this topic?

• Outcome measure instruments used for hidradenitis suppurativa are markedly heterogeneous with 30 instruments recently found in 12 randomized trials.

• Lack of consensus regarding outcome measure instruments limits evidence synthesis and increases the risk of outcome reporting bias.

• A core domain set is an agreed minimum set of what to measure that should be reported in all clinical trials of a specific condition.

What does this study add?

• Our study provides global multi-stakeholder consensus on core outcome domains for hidradenitis suppurativa.

• The final core domain set includes five domains: pain, physical signs, hidradenitis suppurativa-specific quality of life, global assessment and progression of course.

• A sixth domain, symptoms, was highly supported by patients and not by healthcare professionals; it is recommended by the HISTORIC Steering Committee as an additional core domain, in the context of being a patient-reported domain.