Abstract
Background
Due to a large variety in treatment outcomes reported in therapeutic trials and lacking patient‐relevant outcomes, it is hard to adequately compare and improve current therapies for patients with capillary malformations (CMs). The Core Outcome Set for Capillary Malformations (COSCAM) project aims to develop a core outcome set (COS) for use in future CM trials, in which we will first develop a core outcome (sub)domain set (CDS). Here, we describe the methods for the development of a CDS and present the results of the first development stage.
Methods
The COSCAM project is carried out according to the recommendations of the Cochrane Skin Core OUtcomes Set INitiative (CS‐COUSIN) and the Core Outcome Measures in Effectiveness Trials (COMET) initiative. During the first stage, we identified all potentially relevant outcome subdomains based on a systematic review, two focus group sessions and input from patient representatives of Dutch patient organizations and the COSCAM‐founding group. In stage two, we will present the subdomains in a three‐round e‐Delphi study and online consensus meeting, in which CM patients, parents/caregivers and CM experts worldwide rate the importance of the proposed subdomains, hereby finalizing the core outcome (sub)domains of the CDS.
Results
A total of 67 potential outcome subdomains were included; sixteen were previously used in the literature, 20 were proposed by Dutch patients and their parents/caregivers (n = 13) in focus group sessions and 38 were suggested by the experts of the COSCAM‐founding group. Seven were excluded because of overlap.
Conclusion
The final CDS may serve as a minimum standard in future CM trials, thereby facilitating adequate comparison of treatment outcomes. After this CDS development, we will select appropriate outcome measurement instruments to measure the core outcome subdomains.
Introduction
Capillary malformations (CMs), also known as port‐wine stains or birthmarks, are red‐to‐purple lesions affecting the dermis and are present at birth.1 These congenital lesions have been associated with somatic mosaic mutations in the GNA (Q or 11) and Pik3ca gene.2, 3, 4 They occur in 0.04–2.1% of newborns and frequently appear as flat, pink macules that may gradually evolve into more hypertrophic, red‐to‐purple lesions.5, 6 About two‐thirds of the patients develop nodular or papular elements as a result of soft tissue overgrowth, possibly leading to asymmetry, dysmorphosis and sporadic spontaneous bleeding.7, 8, 9 Furthermore, most CMs are located on the head and neck and have shown to seriously affect health‐related quality of life (HRQoL), ranging from impaired emotional HRQoL to functional problems.10 Over the last decades, the pulsed dye laser has been the treatment of choice. However, its effectiveness in terms of clearance rate has barely improved over the last decades, with post‐treatment recurrences still possible leaving patients with a desire for improved treatment regimens.11
The existence of broad diverse treatment outcomes reported in CM research coupled with a lack of outcome measures relevant to patients poses challenges to improvement in the current treatment approach.12 The development of a core outcome set (COS) may help reduce heterogeneity in outcome reporting and focus on more patient‐relevant outcomes. A COS, containing a core outcome domain set (CDS) and a core outcome measurement set (COMS), comprises a minimum set of outcomes that should be measured and reported in clinical research when studying a specific health condition.13, 14 Thus, a COS comprises what (outcome domains and subdomains) should be measured and how to do so (outcome measurement instruments). A recent taxonomy, which classifies outcomes according to a hierarchy in core areas, domain and subdomain levels, was published to use as a starting point for dermatological COS developers.15 Developing a COS has become a prerequisite to conduct meaningful research in the field of dermatology.16, 17, 18, 19, 20 Delphi studies in particular have proven essential in the development of COSs, e.g. for atopic dermatitis and congenital melanocytic nevi.17, 20 Concerning vascular malformations, such a set of outcomes was recently developed for peripheral lymphatic, venous and arteriovenous malformations in the OVAMA‐project.16 Yet, no COS exists for CMs.
This Core Outcome Set for CApillary Malformations (COSCAM) project first aims to reach international consensus on a CDS as part of a COS, in order to unify outcome use and reporting for CMs worldwide. Here, we describe the methods to develop a CDS and present the results of the first development stage.
Methods/design
The research team
The research team consists of the COSCAM founding group and the international COSCAM steering group. The main responsibility of the founding group is the daily management of the study. It includes two methodological experts, one researcher, one plastic surgeon, one plastic surgeon in training and one dermatologist from the Amsterdam UMC. International CM experts form the steering group and contribute to crucial points in the study process (i.e. protocol development, stakeholder recruitment and the international consensus meeting).
Study design
The COSCAM project was registered at the Core Outcome Measures in Effectiveness Trials (COMET) website (http://www.comet‐initiative.org/Studies/Details/1599 ; registration number: 1599) and the Cochrane Skin Core Outcome Set INitiative (CS‐COUSIN) website (http://cs‐cousin.org/coscam/). The steps of the HOME (Harmonizing Outcome Measures for Eczema) initiative roadmap and the COS‐STAD (Core Outcome Set‐STAndards for Development) guidelines will be followed to serve as a directory for the development of the CDS.20, 21 The recently proposed dermatological taxonomy by Lange et al.15 will be used to categorize the outcomes into outcome domains and subdomains (see Table 1 for definitions). Additionally, the systematic review by Boulkedid et al.22 provided guidance for the design of the Delphi study. The results of the Delphi study will be reported according to the Core Outcome Set‐ STAndards for Reporting (COS‐STAR) checklist.23
Table 1.
Outcome definitions according to the proposed hierarchy by Lange et al.15
| Core Area | ‘An aspect of health or a health condition that needs to be measured to appropriately assess the effects of a health intervention. Core Areas are broad concepts consisting of a number of more specific concepts called outcome domains. Example: Life Impact’. |
| Outcome Domain | ‘Component of a Core Area: a concept to be measured, a further specification of an aspect of health, categorized within a core area. Example: Quality of Life’. |
| Outcome Subdomain level 1 | ‘Component of an Outcome Domain: a concept to be measured, a further specification of an aspect of health, categorized within an Outcome Domain. Example: functioning’. |
| Outcome Subdomain level 2 | ‘Component of an Outcome Domain: a concept to be measured, a further specification of an aspect of health, categorized within an Outcome Subdomain. Example: physical functioning’. |
Definitions retrieved from Lange et al.15
Developmental stages of the CDS
The overall development stages of the CDS can be found in Fig. 1.
Figure 1.
General developmental stages of the CDS.
Stage I – Identification of potential outcome subdomains
As the first step of stage I was already completed before the start of this project, predominantly clinician‐reported outcome subdomains could be retrieved from a systematic review (literature search from 2005 to May 2020).12 Next, two small online focus group sessions, including nine Dutch CM patients and/or their parents/caregivers, were organized to collect patient‐reported outcome subdomains. The focus group sessions were conducted in May 2020. Patients visiting the outpatient clinic of the Amsterdam UMC between March and May 2020 were invited by telephone and joined following verbal consent. Two experienced researchers designed prespecified open questions and led the online meetings. The questions were discussed, ranging from the patients’ experience with their CM to treatment outcomes. The sessions were audio‐recorded, transcribed and analysed for content. In the analysis, topics were extracted, grouped and translated into English. Patient representatives of two Dutch patient organizations (n = 4) were asked to add potentially missing outcome subdomains. The list with retrieved clinician‐reported and patient‐reported subdomains was discussed with the founding group and complemented (Table 2) according to the taxonomy by Lange et al. Subsequently, a final list with potentially relevant outcome (sub)domains (1st and 2nd level; see Table 1 for outcome definitions) was generated. The study protocol and list with potential outcome subdomains was checked and approved by the CS‐COUSIN methods working group and the COSCAM steering group prior to its application in the e‐Delphi study.
Table 2.
Proposed outcome domains and subdomains for capillary malformations in clinical research
| Core area | Outcome domain | Subdomain 1st level | Subdomain 2nd level |
|---|---|---|---|
| (Skin) Pathophysiological manifestations | Clinical assessment | Appearance† |
General appearance§ Redness§ Texture† Thickness§ Size§ Skin stiffness† Noticeability‡ |
| Signs and symptoms¶ |
Bleeding§ Pain† Itching† Pyogenic granuloma¶ |
||
| Life impact | Health‐related Quality of Life | Overall HRQoL¶ | Overall HRQoL¶ |
| Functioning¶ |
Emotional functioning‡ Cognitive functioning¶ Social functioning‡ Occupational (role) functioning‡ Physical functioning¶ |
||
| Family impact¶ |
Burden for parents/caregivers¶ Parental HRQoL¶ |
||
| Perception of health¶ |
Perception of cosmetic results¶ Perception of functional results¶ |
||
| Global perception of disease¶ |
Perception of symptoms related to CMs (e.g. sleeping problems)¶ Perception of CM severity¶ |
||
| Coping¶ | Specific skin (care) behavior‡ | ||
| Treatment |
Adherence to treatment¶ No of required procedures‡ Tolerability of intervention¶ Patient satisfaction with treatment result§ |
Satisfaction with cosmetic and/or functional outcome‡ |
|
| Treatment adverse events |
Pain‡ Bruising‡ Wound‡ Hypopigmentation† Hyperpigmentation† Hypertrophic scarring† Atrophic scarring† Blistering‡ Crusting§ Swelling‡ Textural changes¶ Bleeding¶ Pyogenic granuloma¶ Adverse events of anesthetics‡ |
Presence and severity of pain¶ Presence and severity of bruising¶ Presence and severity of wound¶ Presence and severity of hypopigmentation¶ Presence and severity of hyperpigmentation¶ Presence and severity of hypertrophic scarring¶ Presence and severity of atrophic scarring¶ Presence and severity of blistering¶ Presence and severity of crusting¶ Presence and severity of swelling¶ Presence and severity of textural changes¶ Presence and severity of bleeding¶ Presence and severity of pyogenic granuloma¶ Presence and severity of adverse events of anesthetics¶ |
|
| Practical issues |
Economic Hospital |
Treatment costs¶ Number of hospital visits¶ |
CM; capillary malformation, HRQoL; Health‐related Quality of life.
Outcome subdomains retrieved from a systematic review.
Patient‐reported outcome subdomains collected from focus group sessions
Overlapping outcome subdomains, as collected from a systematic review and focus group sessions.
Subdomains complemented by the COSCAM‐founding group based on the taxonomy by Lange et al (2020).
Stage II – Selection of core outcome subdomains
This stage entails an international modified e‐Delphi study (i.e. an e‐Delphi method with a set number of study rounds) followed by a consensus meeting. The design and number of study rounds of this three‐round e‐Delphi study are based on common methods used in previously published COS studies.16, 17, 24, 25, 26
Stakeholders
Participants of the e‐Delphi study comprise CM patients from various countries and an international group of CM experts. Selection criteria per stakeholder group are listed in Table 3. No official consensus exists on how many participants should be involved with an e‐Delphi study.27, 28 We aim to include as many international CM experts and CM patients as possible, while striving for a representative group of people from various cultural and demographic backgrounds. Each member of the international COSCAM steering group will be responsible for recruiting 5‐10 CM patients and CM experts from their country. We aim to include at least 100 participants in total (CM patients, parents/caregivers and CM experts).17 Since a 30% response rate among the invited participants is commonly expected, we aim to invite a total of approximately 300 participants.
Table 3.
Selection criteria per stakeholder group for the e‐Delphi study
| CM experts | CM patients and parents/caregivers |
|---|---|
| Inclusion criteria: | Inclusion criteria: |
|
|
| Exclusion criteria: | Exclusion criteria: |
|
|
CM, capillary malformation.
CM experts
CM expert selection criteria are presented in Table 3. Experts will be sought among authors of published CM literature, through personal networks of the COSCAM steering group, contact lists of the International Society of the Study for Vascular Anomalies (ISSVA), and through the OVAMA (Outcome measures for VAscular MAlformations) project participant list. Most of the COSCAM‐founding group and steering group members are considered CM experts and therefore eligible for participation in the e‐Delphi study.
CM Patients and parents/caregivers
CM patient selection criteria are shown in Table 3. Patients aged 16 and over are invited to complete the survey themselves, if possible. For patients who are unable to complete the survey alone, and for patients below 16 years of age, the parent or caretaker will be asked to complete the survey. All participating patients will be approached and informed in person or via email by the COSCAM steering group, participating CM experts and through patient organizations from the Netherlands, Belgium, UK, Spain, Italy and USA. Patients may also register themselves via social media advertising on the social media channels (Facebook or Instagram) of the national and foreign patient organizations. At the start of the e‐Delphi study, patients will be asked for their consent to use their data anonymously.
e‐Delphi survey procedure
At the first round, general information will be given about the aim and content of the survey. The participants will be presented with the list of potentially relevant outcome subdomains via online Dutch and English surveys (by means of Google forms and Paperform) in lay language. In the first round, participants may also propose other relevant (missing) outcome subdomains concerning CMs. Data on baseline characteristics of the participants will be collected anonymously to allow for subgroup analyses. For the CM expert group, we will collect data on their specialty, years of experience, country of employment, type of hospital, member of multidisciplinary working group, and number of new patients seen and treated annually. For the CM patient and parent/caregiver group, we will collect data on educational level, age of the patient, country of residence, skin type, location of the CM, presence of hypertrophy, previous and current therapies and whether the CM is part of the Surge‐Weber syndrome or other type of vascular anomaly.
Participants will have 4–6 weeks per round to fill out the survey. Reminders will be sent frequently. If the response rate is below 70%, an extra week is given to fill out the survey. Only participants who have completed all previous rounds will be invited for the next.
In the two subsequent rounds, participants will anonymously receive feedback on the scores of the previous round of both stakeholder groups. The outcome subdomains on which no consensus is reached are to be re‐evaluated. Also, the proposed outcome subdomains suggested in the first round will be rated. Eventually, after three study rounds, the outcome subdomains will be labelled as ‘excluded from the CDS’ (consensus on non‐importance in both stakeholder groups), ‘included in the CDS’ (consensus on the importance in both stakeholder groups) or ‘undecided’ (no consensus on the importance reached yet, or consensus reached in only one stakeholder group).
Definition of consensus
In each study round, participants can rate the importance of the proposed outcome subdomains on a seven‐point Likert scale1, 2, 3, 4, 5, 6, 7 (Table 4). Currently, there are no strict standards on the appropriate level of consensus for Delphi studies. This, however, depends on the importance of the issue for which consensus is required. As we aim to determine the fundamental outcome subdomains needed in clinical research for CMs, we therefore stipulate that an 80% level of consensus is fitting, which is in agreement with other e‐Delphi studies.16, 29, 30 If ≥80% of the CM expert group and ≥80% of the CM patient/parent/caregiver group both score an outcome subdomain a six or seven, it is deemed ‘important’ or ’crucial’, respectively, and is included in the CDS. Outcome subdomains will be excluded from the CDS when there is ≥80% agreement on score one or two on the Likert scale in both stakeholder groups.
Table 4.
Scoring system for the importance of individual capillary malformation subdomains
| Very unimportant | Unimportant | Slightly unimportant | Neutral | Slightly important | Important | Crucial |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 |
| O | O | O | O | O | O | O |
Scores of 6 (‘important’) or 7 (‘crucial’) and 1 (‘very unimportant’) or 2 (‘unimportant’) are counted towards in‐ or exclusion of a subdomain, respectively.
Red color means an outcome subdomain is ‘unimportant'.
Orange color means an outcome subdomain is ‘not important enough'.
Green color means an outcome subdomain is ‘important/crucial'.
Consensus meeting
Following the third study round, the ‘undecided’ outcome subdomains will be discussed in an international online consensus meeting. This meeting will be held with the COSCAM steering group and stakeholders who completed at least two e‐Delphi rounds to reach a final IN or OUT decision for the remaining outcome subdomains. This vote will be held separately for the CM patient group and the CM expert group. When 80% or more of the participants in both stakeholder groups vote IN, the outcome subdomain will be included in the CDS. Otherwise, the outcome subdomain will be excluded. The final core outcome subdomains will be classified according to the prespecified outcome domains as proposed by Lange et al.15
Data analyses
Data will be analysed and presented in absolute numbers and percentages. The percentage agreement in each e‐Delphi round will be calculated for all outcome subdomains. The results will be presented separately for the CM experts and the CM patient/parent/caregiver group. Furthermore, we will separately analyse results for patients with Sturge‐Weber syndrome or a CM combined with another type of vascular malformation, to find any differences in relevant outcome subdomains between these subgroups.
Ethics approval and consent
The Medical Ethics Review Board of the Amsterdam University Medical Center location AMC confirmed that the Dutch Medical Research Involving Human Subjects Act (WMO) does not apply to this study (Committee reference number: W20_351 # 20.389). Hence, a full review of this study was not required. The Board waived the need for a written informed consent. Participants will, however, need to give online consent at the beginning of the first survey, for their data to be used anonymously.
Results
During the first stage of the CDS development, a total of 67 potential outcome subdomains (1st and 2nd level) were recognized (see Table 2). Sixteen, predominantly clinician‐reported outcome subdomains, were retrieved from a previously published systematic review. These mostly focussed on CM appearance and treatment side‐effects. In addition, 20 patient‐reported outcome subdomains were collected from 13 Dutch CM patients and/or their parents/caregivers, of which seven overlapped with the outcome subdomains retrieved from the systematic review. The founding group complemented the list with 38 outcome subdomains. The results of the second stage will be published separately.
Discussion
This COSCAM study protocol describes the development process of a CDS for CMs as part of a COS for clinical research. The first development stage has been completed, which focussed on the identification of potential core outcome subdomains. Relatively few outcome subdomains were retrieved from the literature. Specifically, patient‐reported clinical outcome subdomains were insufficiently reported, with no outcome subdomains on HRQoL or functioning.12 This is remarkable since CM patients often experience a decreased HRQoL.10 Furthermore, patient‐reported outcome subdomains generally measured by Patient Reported Outcome Measures (PROMs) seem inevitable to ensure that treatment outcomes meet the patient’s needs. Also, by integrating patient‐reported outcomes, the principle of value‐based health care, i.e. health care based on patient’s individual needs and goals, is incorporated.31 Even though a heightened importance has been put on the inclusion of patient‐reported outcomes in dermatology, these relevant outcomes still seem to be lacking in CM trials.12, 18, 19, 32, 33, 34 Patient‐reported outcome subdomains are therefore an important constituent in our CDS.
Once the CDS has been developed, we will focus on selecting appropriate outcome measurement instruments (OMI) measuring the core outcome subdomains according to the guidelines of the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) initiative.35 In a recent systematic review, the measurement properties of previously proposed and/or used OMIs for CMs were assessed using the COSMIN methodology.36 However, the authors may have presented an incomplete overview of potential CM OMIs, as they did not follow the Harmonizing Outcome Measures for Eczema (HOME) roadmap recommending the establishment of a CDS prior to identifying adequate OMIs.20 Only after the development of the CDS, we will thus be able to determine whether the recommended OMIs cover all core outcome subdomains in the CDS, whether further OMI identification is needed or whether new OMIs need to be developed. The latter has been previously done for vitiligo and hidradenitis suppurativa outcome measurement.37, 38
The development of the CDS will be the first step towards uniform CM outcome use and reporting worldwide, hereby facilitating adequate comparison of CM treatment outcomes. In addition, with the inclusion of patient‐reported outcome subdomains in future CM trials, we aim to improve value‐based health care. Lastly, the CDS may form the basis of internationally approved CM treatment guidelines, which are, until this date, still lacking.
Acknowledgements
We thank the Dutch CM patients who participated in the focus group sessions, the patient representatives of the Dutch Wijnvlek‐Sturge Weber Association and the HEVAS association for providing feedback on the list with potentially important outcome subdomains, and the international steering committee for providing feedback on the study protocol and for recruiting stakeholders in stage two of the CDS development.
Funding sources
None.
Conflicts of interest
Dr. Haedersdal reports grants from Leo Pharma, grants from Procter and Gamble, non‐financial support from Cherry Imaging, non‐financial support from Cynosure‐Hologic, grants and non‐financial support from Lutronic, grants and non‐financial support from Mirai Medical, grants and non‐financial support from Novoxel, non‐financial support from Perfaction Technologies, outside the submitted work. Dr. Kelly reports grants and other from Michaelson Diagnostics, personal fees and other from Sciton, personal fees from IQVIA, other from Candela, outside the submitted work; and Board member of the American Society for Lasers in Surgery and Medicine. Dr. Robertson reports personal fees from Candela, outside the submitted work. Prof. dr. Spuls has done consultancies in the past for Sanofi 111017 and AbbVie 041217 (unpaid), received a departmental independent research grant for TREAT NL registry Pharma since December 2019 for the TREAT NL registry, is involved in performing clinical trials with many pharmaceutical industries that manufacture drugs used for the treatment of, e.g. psoriasis and atopic dermatitis, for which financial compensation is paid to the department/hospital and, is Chief Investigator (CI) of the systemic and phototherapy atopic eczema registry (TREAT NL) for adults and children and one of the main investigator of the SECURE‐AD registry. The other authors have nothing to declare.
References
- 1.Mulliken JBBP, Fishman SJ. Mulliken and Young's Vascular Anomalies: Hemangiomas and Malformations, Oxford: Oxford University Press, 2013. [Google Scholar]
- 2.Shirley MD, Tang H, Gallione CJet al. Sturge‐Weber syndrome and port‐wine stains caused by somatic mutation in GNAQ. N Engl J Med 2013; 368: 1971–1979. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jordan M, Carmignac V, Sorlin Aet al. Reverse phenotyping in patients with skin capillary malformations and mosaic GNAQ or GNA11 mutations defines a clinical spectrum with genotype‐phenotype correlation. J Invest Dermatol 2020; 140: 1106–10 e2. [DOI] [PubMed] [Google Scholar]
- 4.Vahidnezhad H, Youssefian L, Uitto J. Klippel‐Trenaunay syndrome belongs to the PIK3CA‐related overgrowth spectrum (PROS). Exp Dermatol 2016; 25: 17–19. [DOI] [PubMed] [Google Scholar]
- 5.Schneider BV, Mitsuhashi Y, Schnyder UW. Ultrastructural observations in port wine stains. Arch Dermatol Res 1988; 280: 338–345. [DOI] [PubMed] [Google Scholar]
- 6.Kahana M, Feldman M, Abudi Z, Yurman S. The incidence of birthmarks in Israeli neonates. Int J Dermatol 1995; 34: 704–706. [DOI] [PubMed] [Google Scholar]
- 7.Enjolras O, Mulliken JB. The current management of vascular birthmarks. Pediatr Dermatol 1993; 10: 311–333. [DOI] [PubMed] [Google Scholar]
- 8.Geronemus RG, Ashinoff R. The medical necessity of evaluation and treatment of port‐wine stains. J Dermatol Surg Oncol 1991; 17: 76–79. [DOI] [PubMed] [Google Scholar]
- 9.van Drooge AM, Beek JF, van der Veen JP, van der Horst CM, Wolkerstorfer A. Hypertrophy in port‐wine stains: prevalence and patient characteristics in a large patient cohort. J Am Acad Dermatol 2012; 67: 1214–1219. [DOI] [PubMed] [Google Scholar]
- 10.Hagen SL, Grey KR, Korta DZ, Kelly KM. Quality of life in adults with facial port‐wine stains. J Am Acad Dermatol 2017; 76: 695–702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Van Raath I, Wolkerstorfer A, Heger M. Port wine stain treatment outcomes have not improved over the past three decades. J Eur Acad Dermatol Venereol 2019; 33: 1369–1377. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.van Raath MI, Chohan S, Wolkerstorfer Aet al. Clinical outcome measures and scoring systems used in prospective studies of port wine stains: A systematic review. PLoS One 2020; 15: e0235657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Williamson PR, Altman DG, Blazeby JMet al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012; 13: 132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Chiarotto A, Ostelo RW, Turk DC, Buchbinder R, Boers M. Core outcome sets for research and clinical practice. Braz J Phys Ther 2017; 21: 77–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lange T, Kottner J, Weberschock Tet al. Outcome assessment in dermatology clinical trials and cochrane reviews: call for a dermatology‐specific outcome taxonomy. J Eur Acad Dermatol Venereol 2020; 35: 523–535. [DOI] [PubMed] [Google Scholar]
- 16.Horbach SER, van der Horst C, Blei Fet al. Development of an international core outcome set for peripheral vascular malformations: the OVAMA project. Br J Dermatol 2018; 178: 473–481. [DOI] [PubMed] [Google Scholar]
- 17.Oei W, Fledderus AC, Korfage Iet al. Protocol for the development of core set of domains of the core outcome set for patients with congenital melanocytic naevi (OCOMEN project). J Eur Acad Dermatol Venereol 2020; 34: 267–273. [DOI] [PubMed] [Google Scholar]
- 18.Gottlieb AB, Levin AA, Armstrong AWet al. The International Dermatology Outcome Measures Group: formation of patient‐centered outcome measures in dermatology. J Am Acad Dermatol 2015; 72: 345–348. [DOI] [PubMed] [Google Scholar]
- 19.Calvert M, Kyte D, Mercieca‐Bebber Ret al. Guidelines for inclusion of patient‐reported outcomes in clinical trial protocols: The SPIRIT‐PRO extension. JAMA 2018; 319: 483–494. [DOI] [PubMed] [Google Scholar]
- 20.Schmitt J, Apfelbacher C, Spuls PIet al. The Harmonizing Outcome Measures for Eczema (HOME) roadmap: a methodological framework to develop core sets of outcome measurements in dermatology. J Invest Dermatol 2015; 135: 24–30. [DOI] [PubMed] [Google Scholar]
- 21.Kirkham JJ, Davis K, Altman DGet al. Core outcome set‐STAndards for development: The COS‐STAD recommendations. PLoS Medicine 2017; 14: e1002447. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011; 6: e20476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kirkham JJ, Gorst S, Altman DGet al. Core outcome set‐STAndards for reporting: The COS‐STAR statement. PLoS Medicine 2016; 13 : e1002148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Al Wattar BH, Tamilselvan K, Khan Ret al. Development of a core outcome set for epilepsy in pregnancy (E‐CORE): a national multi‐stakeholder modified Delphi consensus study. BJOG 2017; 124: 661–667. [DOI] [PubMed] [Google Scholar]
- 25.Hirsch M, Duffy JM, Barker Cet al. Protocol for developing, disseminating and implementing a core outcome set for endometriosis. BMJ Open 2016; 6: e013998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Sahnan K, Tozer PJ, Adegbola SOet al. Developing a core outcome set for fistulising perianal Crohn's disease. Gut 2019; 68: 226–238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Williams PL, Webb C. The Delphi technique: a methodological discussion. J Adv Nurs 1994; 19: 180–186. [DOI] [PubMed] [Google Scholar]
- 28.Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol 2005; 5: 37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Langlands RL, Jorm AF, Kelly CM, Kitchener BA. First aid for depression: a Delphi consensus study with consumers, carers and clinicians. J Affect Disord 2008; 105: 157–165. [DOI] [PubMed] [Google Scholar]
- 30.Carey B, Farag AM, Nasri‐Heir Cet al. IMMPACT‐recommended outcome measures and tools of assessment in burning mouth syndrome RCTs: an international Delphi survey protocol. Trials 2020; 21: 711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Elf M, Flink M, Nilsson M, Tistad M, von Koch L, Ytterberg C. The case of value‐based healthcare for people living with complex long‐term conditions. BMC Health Serv Res 2017; 17: 24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Armstrong AW, Reich K, Foley Pet al. Improvement in Patient‐Reported Outcomes (Dermatology Life Quality Index and the Psoriasis Symptoms and Signs Diary) with Guselkumab in Moderate‐to‐Severe Plaque Psoriasis: Results from the Phase III VOYAGE 1 and VOYAGE 2 Studies. Am J Clin Dermatol 2019; 20: 155–164. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rijsbergen M, Niemeyer‐van der Kolk T, Rijneveld Ret al. Mobile e‐diary application facilitates the monitoring of patient‐reported outcomes and a high treatment adherence for clinical trials in dermatology. J Eur Acad Dermatol Venereol 2020; 34: 633–639. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Perez‐Chada L, Taliercio VL, Gottlieb Aet al. Achieving consensus on patient‐reported outcome measures in clinical practice for inflammatory skin disorders: on behalf of international dermatology outcome measures and the American Academy of dermatology. J Am Acad Dermatol 2019; 10.1016/j.jaad.2019.09.008. [DOI] [PubMed] [Google Scholar]
- 35.Prinsen CAC, Vohra S, Rose MRet al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials 2016; 17: 449. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.van Raath MI, Chohan S, Wolkerstorfer Aet al. Outcome measures for treatment of port wine stains: a systematic review of their measurement properties. J Dermatol 2020; 1–17. 10.1159/000511438. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Batchelor JM, Tan W, Tour S, Yong A, Montgomery AA, Thomas KS. Validation of the Vitiligo Noticeability Scale: a patient‐reported outcome measure of vitiligo treatment success. Br J Dermatol 2016; 174: 386–394. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Kirby JS, Thorlacius L, Villumsen Bet al. The Hidradenitis Suppurativa Quality of Life (HiSQOL) score: development and validation of a measure for clinical trials. Br J Dermatol 2020; 183: 340–348. [DOI] [PMC free article] [PubMed] [Google Scholar]