Summary
Background
A core outcomes set (COS) is an agreed minimum set of outcomes that should be measured and reported in all clinical trials for a specific condition. Hidradenitis suppurativa (HS) has no agreed-upon COS. A central aspect in the COS development process is to identify a set of candidate outcome domains from a long list of items. Our long list had been developed from patient interviews, a systematic review of the literature and a healthcare professional survey, and initial votes had been cast in two e-Delphi surveys. In this manuscript, we describe two in-person consensus meetings of Delphi participants designed to ensure an inclusive approach to generation of domains from related items.
Objectives
To consider which items from a long list of candidate items to exclude and which to cluster into outcome domains.
Methods
The study used an international and multistakeholder approach, involving patients, dermatologists, surgeons, the pharmaceutical industry and medical regulators. The study format was a combination of formal presentations, small group work based on nominal group theory and a subsequent online confirmation survey.
Results
Forty-one individuals from 13 countries and four continents participated. Nine items were excluded and there was consensus to propose seven domains: disease course, physical signs, HS-specific quality of life, satisfaction, symptoms, pain and global assessments.
Conclusions
The HISTORIC consensus meetings I and II will be followed by further e-Delphi rounds to finalize the core domain set, building on the work of the in-person consensus meetings.
Introduction
Development of evidence-based and consensus-driven outcome measures is necessary to ensure that study results are comparable, to permit meta-analyses and hence better inform healthcare decisions. As a consequence, consensus on outcomes is a prerequisite for patients to receive the benefits of top-level evidence-based medicine. Hidradenitis suppurativa (HS) is a chronic, inflammatory skin disease, characterized by repeated outbreaks of painful inflamed nodules or boils in the apocrine-gland-bearing regions (axillae, genital area, groin, breasts and perianal region).1,2 The estimated prevalence is 1–4% worldwide.3–5 HS is associated with significant disability due to pain and subsequent loss of mobility.6 Interventions for HS are diverse and include topical treatment, systemic antibiotics, anti-inflammatory therapy, biologics and surgical therapy including laser surgery.7 There is a need for continuing research on therapies, as the level of evidence for existing treatments is low, suggesting a particular need for trials.8
Clinical trials should have well-defined primary and secondary outcomes to answer questions generated by the main hypotheses. A core outcomes set (COS) is an agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population, a recommendation of what should be measured and reported in all clinical trials.9 Once a COS is defined, the next step is to achieve consensus on the instruments most suitable to measure each core domain.10 This selection process includes evaluation of the quality of the instruments, assessing their validity, reliability, responsiveness to change and feasibility.11
Like most diseases, HS has no agreed-upon COS, and the reported outcome measurement instruments are numerous. In a recent systematic review, the authors identified a total of 30 outcome measure instruments in 12 randomized controlled trials, and the quality of studies looking at the validity of the instruments was generally low.12 Consequently, trialists and researchers use various instruments, which may or may not be representative of the most important aspects of the disease. In addition, the heterogeneity and lack of consensus regarding use of outcome measure instruments limits the possibility to perform evidence synthesis, including meta-analysis,8 and likely leads to outcome reporting bias because of selective reporting of more favourable outcomes. Empirical evidence of this phenomenon has been highlighted in the literature.13
Based on these existing problems within HS outcome measures, the HIdradenitis SuppuraTiva cORe outcomes set International Collaboration (HISTORIC) was formed as a collaboration between the International Dermatology Outcome Measures (IDEOM) initiative, the Cochrane Skin Group – Core Outcome Set Initiative, and Zealand University Hospital, Roskilde.
The first HISTORIC goal was to develop a COS for HS clinical trials, reducing the risk of heterogeneity in instruments and outcome reporting bias and ensuring that researchers report on outcomes that are relevant to all major stakeholders.14,15 The intention is that the COS for efficacy measures should help guide all HS clinical trials on a global basis, covering both medical and surgical trials.
A central aspect in the COS development process is to identify a list of candidate items from which domains can be developed. It is strongly recommended that the views of all relevant stakeholders are heard in this process.14,15 An effective way of ensuring that the initial list of candidate items is important to the whole community of stakeholders, including patients, is to incorporate qualitative research into the development process.16 For our COS process the generation of items by patients was based on qualitative studies conducted in both Denmark and North America as a mixture of individual and focus-group interviews until the point of saturation.17
Results from the qualitative studies were combined with the results of a systematic review of relevant literature and an online healthcare professional (HCP) item-generation survey to generate one combined list of candidate items. In the process of combining and analysing the results of the three data-sets, the steering group realized that some important decisions needed to be taken, in particular how to combine related items into broader domains. It was decided that a wider set of HS stakeholders should have the chance to vote by e-Delphi on 56 nominal items to help guide formation of candidate domains.
After the first two e-Delphi rounds the Delphi participants were invited to take part in two consensus meetings, at which patients and HCPs worked side by side, together with members of the steering committee, to combine the nominated items into domains, informed by the results of the first two e-Delphi rounds. The meetings were followed by an online confirmation survey. Our method permitted inclusion of opinions from a wide set of patients and other Delphi participants in the important phase of domain formation from candidate items. The consensus meetings took place in September and October 2016 in Vienna and New York, respectively, and the results of these and the confirmation survey are reported here.
The aims of the first consensus meeting were to (i) review the results of the first two e-Delphi rounds, (ii) discuss whether any items could be removed from the list of potential items, (iii) discuss grouping of items into domains and (iv) discuss appropriate names for the created domains.
The aim of the second consensus meeting was to obtain a North American perspective on the same four points including the results of the first meeting. Specific questions addressed were: (i) Should any items excluded at the first meeting be retained? (ii) Do all items fit in their domains? (iii) Should any combined items form their own domain? (iv) Is the name for each domain appropriate?
The aim of the subsequent online confirmation survey was to gain confirmation from the larger HISTORIC project group for decisions taken at the meetings before implementation.
Materials and methods
Initial steps
An overview of our COS development methodology highlighting the contribution of the in-person consensus meetings can be found in Figure 1. Initiatives including Core Outcome Measures in Effectiveness Trials (COMET), Outcome Measures in Rheumatology (OMERACT)18 and Harmonizing Outcome Measures for Eczema (HOME)10 provided methodological guidance that was used and adapted by HISTORIC. Prior to the current study, a list of 56 candidate items was identified as described in the introduction. More details for these initial phases can be found in our COS development protocol.17
Fig 1.
Summary of the hidradenitis suppurativa core domain development process, highlighting (circle) the part described in this study. HCP, healthcare professional.
In brief, the e-Delphi exercise involved 94 participants (42 patients with HS and 52 HCPs) from 19 countries across four continents. In the first two Delphi rounds, participants voted on an unsorted list of candidate items in terms of their importance in being measured as outcomes in all future HS trials. The results of the first two rounds were then used to inform the structure of two consensus meetings, which are reported here.
Study design
The study was international and multiprofessional involving patients, dermatologists, dermatological surgeons, industry representatives and drug regulatory authorities. The study took place at two face-to-face consensus meetings in Vienna and New York with a following online confirmation survey. The meeting locations were planned for both Europe and North America to ensure that European and North American patient and HCP opinions from both continents were incorporated. The meetings were planned by the HISTORIC steering group, consisting of researchers, HS clinicians and a patient research partner.17
Meeting participants
All Delphi participants from the e-Delphi surveys were invited to attend either the first or the second meeting. If attendance in person was not possible, they were invited to join the Vienna meeting via a Skype® connection. Identification and purposive sampling of the e-Delphi participants is described in the study protocol.17 A few additional individuals who had shown an interest in joining the initiative were invited to take part in the second meeting. Our aim was to maintain a 1 : 1 ratio of patients to HCPs if possible.
Study procedures
An overview of the study procedures can be found in Figure 2. Both consensus meetings had the same overall structure, but differed slightly in the required tasks. Tasks for the first meeting were ranking of the items in order of priority, identifying items that could be excluded, grouping of the remaining items into domains, and ranking of domains in order of priority. The participants of the second meeting were asked to mirror the first by considering whether any excluded items should be retained, checking whether the participants agreed with the item combinations that were put forward by the first meeting to form domains, and considering whether the domain names were appropriate.
Fig 2.
Summary of the study procedures. COS, core outcomes set.
The structure consisted of initial formal presentations, followed by small group work and subsequent plenary sessions, based on nominal group theory.19 The spoken language was English. Introductory presentations included a description of the HISTORIC collaboration, a summary of the need for a COS for HS clinical trials, and results from the first two rounds of the e-Delphi.17 Background information about how the candidate items were identified was also provided, together with an introduction to the small group work designed to generate consensus using nominal group theory. It was stressed that the views of all participants at the meeting, both patients and HCPs, were of equal importance.
The introductory presentations were followed by a series of small group sessions (six in the first meeting and three in the second meeting). During each session, two small groups worked independently and in parallel, supervised by neutral facilitators. Both facilitators were medical doctors and PhD students studying HS, who were not voting in the e-Delphi surveys. The neutral facilitators encouraged contributions from quieter group members. Group members were switched between each session to ensure that different combinations of patients and HCPs were formed; however, each small group contained at least two patients so that HCPs did not dominate the discussion.
Physical cards, one for each item, were placed on the table for each small group to provide a visual aid for the discussion. On the front of each card was the name and a description of an item and on the reverse side were summary statistics of the votes cast for the item in the preceding e-Delphi exercise, subdivided by patients and HCPs. Each small group session lasted 20–40 min.
Results from each of the two small groups were presented to all participants in subsequent plenary sessions, stimulating discussion if there were differences between the groups. Consensus was sought by discussion, and if consensus was not possible then no decision was imposed. In particular, when discussion involved an item for possible exclusion, if no consensus was reached then the item was retained.
As only a subset of the e-Delphi group was able to attend the in-person meetings because of the global locations of HISTORIC participants, discussion was not followed by binding voting. Instead all decisions taken at the meetings required confirmation by the larger HISTORIC project group in a subsequent online confirmation survey sent to all participants. A summary of the discussion was provided in the survey together with the results, and participants were asked if they agreed with exclusion of the nominated items, if they thought all items fitted in the created domains, if any of the combined items should form their own domain and if each domain name was appropriate. Consensus for implementation was defined as > 70% of each stakeholder group voting in agreement with the decision. Further information on the methodology applied to the confirmation survey and the following e-Delphi rounds is described in the study protocol.17
Results
Participants
A list of study participants subdivided by stakeholder group, country and sex can be found in Table S1 (see Supporting Information).
The HISTORIC consensus meeting I had 19 participants (five patients, 14 HCPs) from 11 countries across four continents, the majority being European. The HISTORIC consensus meeting II had 25 participants (six patients, 19 HCPs), the majority being North American.
The 11 participating patients with HS represented six different patient organizations. One additional Canadian patient participated in the first meeting via a Skype® connection. The participating HCPs were dermatologists (n = 14), dermatological surgeons (n = 5), U.S. Food and Drug Administration (FDA) representatives (n = 2), pharmaceutical industry representatives (n = 2), epidemiologists (n = 3) and nonvoting (in e-Delphi) steering group members or facilitators (n = 4). For comparison, the 52 HCPs included in the e-Delphi round one were dermatologists (n = 41), dermatological surgeons (n = 5), medical regulators (n = 1), nurses (n = 4) and pharmaceutical industry representatives (n = 1).
Excluded items
The comprehensive list of unsorted items (n = 57) that the participants evaluated is shown in Table S2 (see Supporting Information). Nine items were marked for exclusion during HISTORIC consensus meeting I, due to either lack of relevance, being unrelated to measurement of disease severity, or not being directly linked to the disease (Table S3; see Supporting Information). Some participating HCPs spoke in favour of excluding coping, itch and fatigue, but the participating patients did not approve and the items were retained.
At the HISTORIC meeting II, there was consensus that all of the items identified at the first meeting were appropriately designated for exclusion. However, it was agreed that the biomarker item should be marked as an area of specific future research interest. It was noted that if, in the future, a biomarker is proven to be strongly related to disease activity or treatment response then the biomarker item or domain should be reconsidered for inclusion in the core domain set.
Grouping of items into domains and naming of domains
Creation of potential domains was achieved by small group and plenary sessions at the HISTORIC consensus meeting I, producing consensus to group the items into nine domains (Table S4; see Supporting Information). These domains and their contributing items were reviewed at the HISTORIC consensus meeting II. HISTORIC consensus meeting II participants recommended switching the ‘number of chronic areas’ item from the ‘physical signs’ domain to the ‘disease course’ domain, as the item would be reported by the patient rather than being measured by the physician. It was highlighted that the term ‘chronic’ in this context needs to be defined further and this issue was marked as a future task for the HISTORIC project. The group provisionally agreed that ‘chronic’ relates to a duration of ≥ 6 weeks.
Both working groups at consensus meeting II independently agreed to rename the ‘decreased mobility’ item as ‘physical functioning’ and to combine this domain with the ‘psychological–social’ domain to form an ‘HS-specific quality of life’ domain. Participants emphasized that it is crucial that this domain should capture the specific aspects of the patient's quality of life that are affected by HS, so a generic health-related quality-of-life domain would not be sufficient.
Another recommendation from meeting II was to group together the ‘patient global assessment’ and ‘physician global assessment’ domains to produce a single ‘global assessment’ domain encompassing both the patient and HCP perspectives. This fusion and the global assessment items and domains themselves were heavily debated. Some participants felt that the global assessments should be excluded altogether because, by definition, global assessment provides a relatively nonspecific overview of disease severity. Others spoke in favour of global assessments because they considered a global anchor to be very useful. Another argument in favour of retaining global assessments is that these domains are considered important by the FDA. Creation of a single ‘global assessment’ domain was suggested by a group member and supported by the rest of the group based on the concept that both the patient and HCP global perspectives are important and should be assessed in a similar manner.
After HISTORIC consensus meeting II, there was consensus to suggest seven core domains: disease course, physical signs, HS-specific quality of life, satisfaction, symptoms, pain and global assessments (Table S5; see Supporting Information).
Consensus meeting online confirmation survey
Nearly all decisions taken at the meetings were confirmed by > 70% of each stakeholder group. The only exception was confirmation of the name for the created domain ‘satisfaction’ which was confirmed by only 61% of HCPs. This domain therefore needed adjustment.
Discussion
In total, 41 stakeholders including patients, dermatologists, dermatological surgeons, epidemiologists, statisticians, pharmaceutical industry representatives and drug regulatory representatives participated in the HISTORIC consensus meetings I and II. Important progress was made towards reaching global consensus on core outcomes for HS clinical trials. Seven potential core domains were put forward for consideration by the larger e-Delphi consensus group in subsequent e-Delphi surveys.
Our study differs from other COS processes in that our domains were developed through in-person discussion, combining items from a comprehensive list of candidate items. This discussion was guided by votes cast in preceding e-Delphi surveys. In most previous studies, domains are created by the steering committee alone without broader dialogue with Delphi participants before the first round of the Delphi survey is launched. The concept of involving more patients and other Delphi participants in the creation of the domains is based on the principle of inclusivity, in keeping with the philosophy of our HISTORIC initiative. Feedback from stakeholders was very positive and the general view was that an inclusive approach is important to ensure relevance to patients and subsequent global acceptance and use of the HS COS by clinical trial designers.
One methodological limitation is that it was not possible to have all e-Delphi participants present at the meetings, and therefore it was not possible to incorporate everyone's opinion in the formation of domains. To address this issue, the next step will be to ask the larger e-Delphi group if they agree with the decisions made at the meetings in an evaluation and confirmation survey. After this, the next planned steps are to perform two additional e-Delphi rounds. The results from these rounds will finalize the core domain set, having built on the work from our in-person consensus meetings.
Another limitation to the study is that we did not achieve our aim of a 1 : 1 ratio of participating patients and HCPs. However, the meeting facilitators did ensure that the patients provided equal input to the HCPs, even though they were outnumbered by the HCPs, by encouraging patient involvement in every aspect of the discussion.
With the present study, we have come a lot closer to global consensus on a COS for HS research. The number of randomized controlled trials of HS therapy is still limited. However, interest in the disease is growing and the number of trials planned is considerable. The development of a COS is thus particularly timely for HS, and an HS COS should substantially improve future HS trial design.
Supplementary Material
Table S1. Study participants subdivided by stakeholder group, country and sex.
Table S2. List of all items included in the Delphi exercise (item 57 included after the first round).
Table S3. Items marked for exclusion at HISTORIC consensus meetings I and II with the arguments for their exclusion.
Table S4. Results of HISTORIC consensus meeting I in Vienna: list of created domains and their included items.
Table S5. Results of HISTORIC consensus meetings I and II: list of created domains and their included items.
Video S1. Author video.
Powerpoint S1. Journal Club Slide Set.
What's already known about this topic?
Reported outcome measure instruments for hidradenitis suppurativa (HS) are numerous and diverse, with 30 instruments recently found in 12 randomized trials.
This diverse use of instruments limits the possibility to perform evidence synthesis and may produce outcome reporting bias.
A core outcomes set (COS) is an agreed minimum set of outcomes that should be measured and reported in all clinical trials.
What does this study add?
The study used an international and multistakeholder approach, involving patients, dermatologists, surgeons, the pharmaceutical industry and medical regulators.
Two consensus meetings, in Europe and North America, considered potential HS core domains, within a nominal group theory structure.
Seven potential core domains were put forward to the subsequent e-Delphi: disease course, physical signs, HS-specific quality of life, satisfaction, symptoms, pain and global assessments.
What are the clinical implications of this work?
The development of a COS is particularly timely for HS and should substantially improve future HS trial design.
Acknowledgments
We thank Sabrina Mai Nielsen, Musculoskeletal Statistics Unit, The Parker Institute, Bispebjerg and Frederiksberg Hospital, Copenhagen, Denmark, for statistical support. We thank Selma Bajric, Medical University of Vienna, Department of Dermatology, and Amanda Pacia, IDEOM, for logistic help.
Funding sources: This work is supported by grants from the International Dermatology Outcome Measures (IDEOM). L.T. is supported by the Region Zealand Research Foundation. The Musculoskeletal Statistics Unit, The Parker Institute, is supported by grants from The Oak Foundation. J.R.I. is supported by a Health Research Fellowship from Health and Care Research Wales.
Footnotes
Plain language summary available online
Supporting Information: Additional Supporting Information may be found in the online version of this article at the publisher's website:
Conflicts of interest: L.T., J.R.I., F.G.B., L.E., J.S.K., L.M., E.P.P., H.H.vdZ. and G.B.E.J. are or have been involved in the development of instruments that could potentially be used to measure core outcomes for hidradenitis. However, specific instruments were not discussed.
L.T. and A.G. share joint first authorship
References
- 1.Jemec GB. Clinical practice Hidradenitis suppurativa. N Engl J Med. 2012;366:158–64. doi: 10.1056/NEJMcp1014163. [DOI] [PubMed] [Google Scholar]
- 2.Zouboulis CC, Del Marmol V, Mrowietz U, et al. Hidradenitis suppurativa/acne inversa: criteria for diagnosis, severity assessment, classification and disease evaluation. Dermatology (Basel) 2015;231:184–90. doi: 10.1159/000431175. [DOI] [PubMed] [Google Scholar]
- 3.Jemec GB, Heidenheim M, Nielsen NH. The prevalence of hidradenitis suppurativa and its potential precursor lesions. J Am Acad Dermatol. 1996;35:191–4. doi: 10.1016/s0190-9622(96)90321-7. [DOI] [PubMed] [Google Scholar]
- 4.Revuz JE, Canoui-Poitrine F, Wolkenstein P, et al. Prevalence and factors associated with hidradenitis suppurativa: results from two case–control studies. J Am Acad Dermatol. 2008;59:596–601. doi: 10.1016/j.jaad.2008.06.020. [DOI] [PubMed] [Google Scholar]
- 5.Vinding GR, Miller IM, Zarchi K, et al. The prevalence of inverse recurrent suppuration: a population-based study of possible hidradenitis suppurativa. Br J Dermatol. 2014;170:884–9. doi: 10.1111/bjd.12787. [DOI] [PubMed] [Google Scholar]
- 6.Dufour DN, Emtestam L, Jemec GB. Hidradenitis suppurativa: a common and burdensome, yet under-recognised, inflammatory skin disease. Postgrad Med J. 2014;90:216–21. doi: 10.1136/postgradmedj-2013-131994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Zouboulis CC, Desai N, Emtestam L, et al. European S1 guideline for the treatment of hidradenitis suppurativa/acne inversa. J Eur Acad Dermatol Venereol. 2015;29:619–44. doi: 10.1111/jdv.12966. [DOI] [PubMed] [Google Scholar]
- 8.Ingram JR, Woo PN, Chua SL, et al. Interventions for hidradenitis suppurativa. Cochrane Database Syst Rev. 2015;10:CD010081. doi: 10.1002/14651858.CD010081.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39. doi: 10.1186/1745-6215-8-39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Schmitt J, Apfelbacher C, Spuls PI, et al. The Harmonizing Outcome Measures for Eczema (HOME) roadmap: a methodological framework to develop core sets of outcome measurements in dermatology. J Invest Dermatol. 2015;135:24–30. doi: 10.1038/jid.2014.320. [DOI] [PubMed] [Google Scholar]
- 11.Prinsen CA, Vohra S, Rose MR, et al. How to select outcome measurement instruments for outcomes included in a ‘Core Outcome Set’ – a practical guideline. Trials. 2016;17:449. doi: 10.1186/s13063-016-1555-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ingram JR, Hadjieconomou S, Piguet V. Development of core outcome sets in hidradenitis suppurativa: a systematic review of outcome measure instruments to inform the process. Br J Dermatol. 2016;175:263–72. doi: 10.1111/bjd.14475. [DOI] [PubMed] [Google Scholar]
- 13.Cooney RM, Warren BF, Altman DG, et al. Outcome measurement in clinical trials for ulcerative colitis: towards standardisation. Trials. 2007;8:17. doi: 10.1186/1745-6215-8-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132. doi: 10.1186/1745-6215-13-132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Boers M, Kirwan JR, Wells G, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67:745–53. doi: 10.1016/j.jclinepi.2013.11.013. [DOI] [PubMed] [Google Scholar]
- 16.Keeley T, Williamson P, Callery P, et al. The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. 2016;17:230. doi: 10.1186/s13063-016-1356-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Thorlacius L, Ingram JR, Garg A, et al. Protocol for the development of a core domain set for hidradenitis suppurativa trial outcomes. BMJ Open. 2017;7:e014733. doi: 10.1136/bmjopen-2016-014733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Tugwell P, Boers M, Brooks P, et al. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38. doi: 10.1186/1745-6215-8-38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gallagher M, Hares T, Spencer J, et al. The nominal group technique: a research tool for general practice? Fam Pract. 1993;10:76–81. doi: 10.1093/fampra/10.1.76. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Table S1. Study participants subdivided by stakeholder group, country and sex.
Table S2. List of all items included in the Delphi exercise (item 57 included after the first round).
Table S3. Items marked for exclusion at HISTORIC consensus meetings I and II with the arguments for their exclusion.
Table S4. Results of HISTORIC consensus meeting I in Vienna: list of created domains and their included items.
Table S5. Results of HISTORIC consensus meetings I and II: list of created domains and their included items.
Video S1. Author video.
Powerpoint S1. Journal Club Slide Set.