Abstract
Introduction
Velopharyngeal dysfunction (VPD) is present in up to 40% of patients following cleft palate repair. Children with VPD display hypernasal speech, nasal air emission and are at a high risk for developing articulation disorders. The overall result is decreased intelligibility and acceptability of speech, as well as significant functional and social impairments. Although there are several surgical approaches for the management of children with VPD, standard treatment protocols have not been well defined. There is a need for a core outcome set (COS) to reduce outcome reporting bias and heterogeneity across studies of VPD. The COS-VPD Initiative is an international effort to establish a COS for the reporting of studies of the management of VPD.
Methods and analysis
The study has been developed according to the Core Outcome Set-STAandards for Development standards for the design of a COS study and will be carried out according to the guidance of the Core Outcome Measures in Effectiveness Trials (COMET) initiative. A long list of clinical and patient-reported outcomes will be identified from a systematic review of the literature. A two-stage Delphi consensus process will be used to refine this list into a COS. An international panel of key stakeholders including patients, parents and multidisciplinary clinical and academic experts will be invited to participate in this process. Consensus criteria will be specified a priori and the steering group will ratify the final COS.
Ethics and dissemination
The study has ethical approval through Children’s Health Ireland at Crumlin Research and Ethics Committee, Ref: GEN/683/18. The study is registered with the COMET Initiative (http://www.cometinitiative.org/studies/details/1146?result=true). The COS will be disseminated by publication in the peer-reviewed literature, presentation at international research meetings and distribution to patient-representative organisations. This will facilitate the application of the COS in future studies of the management of VPD.
Keywords: paediatric plastic & reconstructive surgery, speech pathology, plastic & reconstructive surgery
Strengths and limitations of this study.
A core outcome set (COS) will reduce both outcome reporting bias and heterogeneity between studies, thus allowing meaningful collation of results across multiple institutions.
The study achieves stakeholder engagement from multidisciplinary clinicians, patients and parents/ guardians.
There is international expertise contributed by the study steering group.
The COS identified will be broadly applicable to case series, cohort studies, as well as randomised controlled trials.
Further study will be required to identify outcome measurement instruments to assess the outcomes selected.
Introduction
Background
The velopharyngeal valve is made up of the soft palate, the palatopharyngeus muscle and the superior pharyngeal constrictor muscle. It is critically important during speech because it controls the degree and balance of sound energy and airflow into the oral versus nasal part of the vocal tract. Velopharyngeal dysfunction (VPD) refers to inadequate closure of the velopharyngeal sphincter during the speech, resulting in hypernasal resonance, nasal emission of air and decreased intraoral pressure for oral pressure consonants. A patient with VPD, unable to easily produce oral consonants, may develop an articulation disorder, including maladaptive compensatory articulations such as glottal stops and pharyngeal fricatives.1 The overall result is decreased intelligibility of speech and functional and social impairment.2
The velopharyngeal valve may fail for structural reasons such as overt or submucous cleft palate or a repaired cleft palate that remains short or insufficiently mobile due to deficiency of palatal tissue, surgical scarring and/or abnormally positioned palatal muscles. VPD is present in up to 40% of patients following cleft palate repair.3–5 Neuromuscular disorders such as stroke, cerebral palsy, myopathy or neuropathy may also result in failure of closure of a structurally normal velopharyngeal sphincter.6–8 Although speech therapy may help to correct articulation errors secondary to VPD, correction of a structurally or neuromuscularly incompetent velopharyngeal port requires a physical intervention, most commonly surgery.
The aim of surgical intervention in VPD is to create a functional seal between the nasopharynx and the oropharynx during speech production while avoiding nasal obstruction but maintaining a nasal airway. Surgical interventions can be divided into three broad categories: palatal procedures, pharyngeal procedures and palatopharyngeal procedures. Palatal procedures involve reorientating malpositioned palate muscles by carrying out a secondary intravelar veloplasty9 10 or a Furlow double opposing z-plasty.11–14 Alternatively, extra tissue can be introduced into the palate in the form of buccal myomucosal flaps raised from the inner aspect of the cheek.15–17 Pharyngeal procedures comprise circular pharyngoplasties and posterior pharyngeal wall augmentation. Hynes described the first circular pharyngoplasty, which uses musculomucosal flaps based on the salpingopharyngeus muscle placed high in the nasopharynx to create a static constriction.18 Several variations of circular pharyngoplasty have subsequently been described.19–21 The posterior pharyngeal wall can be augmented using autologous fat or material implants.22 23 Finally, a palatopharyngeal flap procedure consists of raising a flap of mucosa and superior pharyngeal constrictor muscle from the posterior pharyngeal wall and suturing it into the nasal layer of the soft palate.24 25
The recording of outcomes of surgery has become standardised in many centres with the advent of programmes such as the American College of Surgeons National Surgical Quality Improvement Programme in the USA26 or the Dutch nationwide routine reporting programme.27 Outcomes such as bleeding or infection, as well as unplanned intensive care unit admission and patient length of stay are routinely included in these large-scale datasets. More specific outcomes relevant to subspecialty procedures such and cleft and speech surgery will not be captured however. In addition to the speech outcome, a particular concern in surgery for VPD is the impact of the procedure on the nasal airway. Surgical procedures carried out on the velopharyngeal sphincter with the aim of correcting nasal escape of air during speech may result in obstructed airflow during sleep. Sleep disordered breathing (SDB) is an umbrella term for several chronic conditions in which partial or complete cessation of breathing occurs many times throughout the night. Symptoms may include snoring, pauses in breathing and disturbed sleep. The result is daytime fatigue that interferes with a person’s ability to function and reduces the quality of life. Therefore, in addition to standard surgical outcomes, it is crucial to screen for SDB and record it as an outcome following surgery for VPD.
As outlined, the surgical options in the management of VPD are numerous, and the literature lacks prospective comparative series. We recently carried out a systematic review of the literature up to 2015 (including randomised controlled trials, cohort studies and case series).28 Eighty-three studies satisfied the inclusion criteria, comprising data on 4011 patients. Overall, 70.7% of patients attained normal resonance and 65.3% attained a resolution of abnormal nasal emission following surgical intervention. There was no notable difference in speech outcomes, need for further surgery or occurrence of SDB across the categories of surgery examined. However, it was noted that outcomes were recorded using diverse parameters, particularly with regard to perceptual speech assessment, often with weak speech methodologies. This made the comparison, even of well-defined cohort studies, problematic and meant that the clinical application potential of the review was limited.
In this context, one might assume that randomised controlled trials comparing different types of VPD surgery would emerge as the preferred study design. However, there are practical difficulties in achieving this because of the relatively small numbers of patients with VPD, their heterogeneity and the existence of well-established protocols in individual units. Therefore, comparative, cross-centre cohort studies are likely to continue to be important in research into VPD surgery. In order to usefully inform clinical decision making, it is essential that the results of such studies can be compared in a standardised way.
One method to achieve this is to develop a COS. A COS is a minimum set of outcomes that should be measured and reported in all studies in a specific field. COSs have been demonstrated to improve outcome reporting in healthcare trials.29 It is important to note that a COS represents a minimum set of relevant outcomes that should be measured in a clinical study of a particular condition. The intent is not to limit researchers but rather to provide them with a minimum list of outcomes to include in their studies along with others of their choosing. There is a precedent for COS development in cleft care. The MOMENT study, published in 2015, developed a COS for the reporting of effectiveness trials for the management of otitis media with effusion (OME) in children with cleft palate.30 A COS reflecting the opinions of clinicians and parents was developed, which included nine core outcomes that can be used in future trials of the management of OME in patients with clefts.
Objectives
The aim of this study is to develop a COS for consistent reporting of outcomes in studies of the management of VPD. There is currently no available COS for studies of patients with VPD (http://www.comet-initiative.org/studies/search). Developing a COS would reduce outcome reporting bias and heterogeneity across studies of VPD. This would allow meaningful collation and comparison of results between different aetiologies, surgical protocols and institutions. Such an instrument would strengthen evidence for clinical decision making regarding intervention selection and would ultimately improve care for patients with VPD.
The objectives of the study are:
To compile a comprehensive list of clinical and patient-reported outcomes based on review of the published literature.
To group the listed outcomes into predefined themes.
To achieve consensus on a minimum set of relevant outcomes for reporting studies of interventions for VPD.
Scope
The scope of the study will include patients with both cleft and non-cleft VPD. The population will include both child and adult patients with VPD. A systematic review of the literature demonstrated an age range of 1–69 years for patients undergoing surgical treatment of VPD,28 therefore, no cut-off age limit will be applied. The COS will cover all surgical and non-surgical interventions for the management of VPD. It is anticipated that the COS could be applied in all future studies that examine outcomes of interventions for VPD.
Methods and analysis
The study will be carried out according to the guidance of the Core Outcome Measures in Effectiveness Trials (COMET) initiative.31 The protocol for the study was developed in accordance with Core Outcome Set-STAndards for Development (COS-STAD) recommendations.32 The protocol is presented using the Core Outcome Set-STAndardised Protocol Items (COS-STAP) Statement for the content of a COS development study protocol.33
Stakeholders
Stakeholder groups to be involved in the COS development process include patients or their representatives, cleft surgeons, cleft speech and language therapists and researchers with a demonstrated interest in VPD. Clinician stakeholder eligibility is based on involvement in the clinical care of children with VPD. Clinicians experienced in managing VPD will be identified by convenience sampling by members of the steering group and/or through their membership of relevant societies or organisations, (eg, The Craniofacial Society of Great Britain and Ireland, The American Cleft Palate Association, Operation Smile). Furthermore, authors identified from the systematic review with a significant volume of publications in the field of VPD will be invited to participate.28 Adult patient and parent participants will be recruited by convenience sampling by members of the steering group and through patient representative organisations.
Steering group
A steering group will oversee the development of the COS. The steering group consists of academic cleft surgeons (CdeB, REK, KCYS and DJAO) and speech and language therapists (AB and DS), an expert in COS development (SS) and the parent of a patient with VPD (HEH). The steering group came together through the members’ attendance at the First and Second International Symposium on VPD meetings, held at the Nationwide Children’s Hospital in Columbus, Ohio, USA, in 2016 and 2018. All members of the steering group will be coauthors of the COS.
Public and patient involvement
The parent of a patient with VPD is included in the steering group of the study and in the authorship of this paper. Knowledge of the patient experience of VPD has been provided throughout the development of the protocol.
Information sources
A systematic review of 83 papers, which presented results of surgery for VPD, has been published.28 Outcomes recorded in papers included in this systematic review have been identified and recorded by the steering group. These outcomes have been presented and discussed at the Craniofacial Society of Great Britain and Ireland annual meeting (Birmingham, UK, 2018) and the Second International Symposium on VPD (Columbus, Ohio, USA, 2018). Both of these meetings provided the opportunity to receive contributions from a wide range of multidisciplinary experts in the field through a series of focused workshops. Participants in these workshops were asked to review outcomes relevant to their clinical field and to comment on the suitability of inclusion of the outcome in the COS. Based on this discussion, further outcomes were added to the initial list derived from the systematic review. This long list of outcomes will be reviewed independently by all members of the steering group. Duplicate outcomes will be removed. It is anticipated that certain outcomes may require separation into subcategories, whereas others may be grouped together. This process will be undertaken independently by at least two members of the steering group, with expertise in the specific area (eg, surgery, speech and language therapy). The resolution of conflicts will be carried out by the lead author. All outcomes will be reviewed and categorised independently by each member of the steering group into the following prespecified themes: patient-reported outcomes; speech outcomes; nasal airway outcomes (including SDB) and surgical care outcomes.
Consensus process
A Delphi process will be carried out among international clinical cleft teams and their patients/parents to achieve consensus on the outcomes to be included in the COS. The Delphi process is a commonly used consensus technique,34 which has frequently been used in COS development.35–38 The Delphi process ensures that anonymous opinions can be obtained in a way that gives equal influence to all who participate and avoids an individual participant being influenced by the opinions of any other participant.
Plain language documents, which have been developed by the COMET Initiative, will be used to explain COS development methodology to participants. The language of the outcomes themselves will be clarified and plain language definitions will be added where necessary. The same version of the questionnaire will be used for both clinical and patient/parent participants. The questionnaire will be pilot tested to assess usability by members of staff and patient representatives in the departments of the steering group and modified accordingly.
There is no robust method for calculating the required sample size for a Delphi survey and assumptions are based on COMET Initiative guidelines and previous studies.31 A balanced mix of stakeholders (adult patients, parents/guardians of patients, cleft surgeons, cleft speech and language therapists) will be identified by purposive sampling by the steering group, aiming for a total of 30 participants in the Delphi panel. Although the steering group is made up of Irish, UK and USA participants, the aim is to include a wider international representation in the Delphi panel, including representation from low-income and middle-income countries. In keeping with the previously described methodology, adult patient and parent participants will comprise 20% of the Delphi panel.39
Research Electronic Data Capture (REDCap) software will be used to deliver the Delphi survey to all participants.40 Potential participants will be emailed full details of the study and asked for their consent to participate in the Delphi panel. Having completed an online consent form, they will then be invited to complete an online Delphi questionnaire. The panel will answer questionnaires in two rounds. After each round, the steering group will provide a de-identified summary of the panel’s answers from the previous round. Thus, participants are encouraged to revise their earlier answers in light of the responses of other members of their panel. Participants will be asked to complete each round of the Delphi exercise within 3 weeks of receipt of the email and will be reminded of this at the start of each survey. A reminder email will be sent at the end of week two to prompt completion of the survey. One further reminder will be sent to non-responders at the end of the 3-week period.
At the beginning of the first (round 1) survey, participants will be presented with some plain language introductory information detailing the purpose and design of the study, as well as a glossary of terms. Round 1 content will comprise a long list of outcomes to be scored. Participants will also be provided with an option to add additional outcomes that they think are relevant. Any new outcomes identified by at least two Delphi participants will be included in round 2 of the process.
Consensus definition
Consensus criteria will be specified a priori. Any outcome with a rating of 7–9 by 70% or more of the panel and 1–3 by 15% or fewer will be included in the COS. Any outcome with a rating of 1–3 by 70% or more of the panel and 7–9 by 15% or fewer will be excluded.41 All other combinations indicated that no consensus had been achieved for the outcome. Round 2 may be analysed using more stringent criteria if a higher proportion of outcomes than expected is rated critical. Specifically, a higher threshold of 75% or more of the panel rating 7–9 and 25% or fewer rating 1–3 will be applied. This decision will be based on the steering group’s judgement and giving due consideration to current COMET recommendations regarding outcomes.31 All items retained after two rounds of the Delphi survey will be included in the final COS.
Outcomes scoring/feedback
Participants will be asked to score each of the outcomes listed in round 1 using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) scale of 1–9, with 1–3 labelled ‘not important’, 4–6 labelled ‘important but not critical’ and 7–9 labelled ‘critical’.42 Round 1 responses will be analysed according to the number of participants scoring each outcome within the GRADE criteria (1–3, 4–6 and 7–9) for the purpose of group feedback in round 2.
Round 2 will also be presented online and distributed via an electronic link embedded in an email. Round 2 will consist of all outcomes from Round one plus additional outcomes suggested by at least two Delphi participants in round 1. In round 2, participants will be shown their previous individual scores, together with group feedback (median score of the group per item), and asked to reconsider their own scores in light of the group response when scoring outcomes in round 2.
Missing data
The two main sources of missing data in a COS development consensus process are non-response (attrition) and partial response. As outlined previously, two reminders will be sent to invited participants in order to maximise the response rate. To address non-response to round 1, those who have not taken part in round 1 will not be invited to participate in round 2. To evaluate for attrition bias between rounds the following approach will be taken. For each survey item, the number of participants who have scored it and the distribution of scores will be summarised. The number of participants completing round 2 will be documented and the potential for attrition bias will be assessed by comparing the participant scores for those who completed both rounds with those who completed round 1 only. Changes in participant scores will be examined between rounds and the reasons given for these changes will be summarised.
Ethics and dissemination
The study has ethical approval through Our Lady’s Children’s Hospital Crumlin Dublin Research and Ethics Committee, Ref: GEN/683/18. Informed consent will be obtained from all participants via an online form that can be completed at the start of the round 1 Delphi survey. The study is registered with the COMET Initiative (http://www.cometinitiative.org/studies/details/1146?result=true).
The Core Outcome Set-STAndards for Reporting (COS-STAR) Equator Network guidelines will be used for the reporting of the COS.43 All members of the steering group will coauthor the final paper, which will be submitted for peer-review and publication in a journal of interest to the multidisciplinary cleft palate community. In order to reach as wide an audience as possible, the COS will also be submitted for presentation at a number of international meetings, including the Craniofacial Society of Great Britain and Ireland, the American Cleft Palate Association and the International Symposium on VPD. The COS will be distributed to patients and parents via patient representative groups.
Supplementary Material
Footnotes
Twitter: @cdeblacam, @susanmsmithm, @debbie_sell
Contributors: CdeB wrote the paper and conceived the project with the support of DJAO. HEH contributed knowledge of the patient experience of VPD. SMS contributed knowledge of core outcome set development. AB and DS contributed knowledge of speech and language therapy. REK, DJAO and KCYS contributed knowledge of VPD surgery and airway. All authors edited and critically revised the study protocol. All authors have read, contributed to and approved the manuscript.
Funding: The study received funding from the Children’s Medical & Research Foundation Crumlin and the Department of Surgical Affairs, Royal College of Surgeons in Ireland.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Kummer A. Resonance disorders and velopharyngeal dysfunction (VPD) : Kummer A, Cleft palate and craniofacial anomalies: effects on speech and resonance. 3rd edn NY, USA: Delamar, 2014: 182–224. [Google Scholar]
- 2.Bhuskute A, Skirko JR, Roth C, et al. Association of velopharyngeal insufficiency with quality of life and patient-reported outcomes after speech surgery. JAMA Facial Plast Surg 2017;19:406–12. 10.1001/jamafacial.2017.0639 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sell D, Grunwell P, Mildinhall S, et al. Cleft lip and palate care in the United Kingdom--the Clinical Standards Advisory Group (CSAG) study. Part 3: speech outcomes. Cleft Palate Craniofac J 2001;38:30–7. [DOI] [PubMed] [Google Scholar]
- 4.Cable BB, Canady JW, Karnell MP, et al. Pharyngeal flap surgery: long-term outcomes at the University of Iowa. Plast Reconstr Surg 2004;113:475–8. 10.1097/01.PRS.0000100806.45065.35 [DOI] [PubMed] [Google Scholar]
- 5.Britton L, Albery L, Bowden M, et al. A cross-sectional cohort study of speech in five-year-olds with cleft palate +/- lip to support development of national audit standards: benchmarking speech standards in the United Kingdom. Cleft Palate Craniofac J 2014;51:431–51. 10.1597/13-121 [DOI] [PubMed] [Google Scholar]
- 6.Hardy JC, Netsell R, Schweiger JW, et al. Management of velopharyngeal dysfunction in cerebral palsy. J Speech Hear Disord 1969;34:123–37. 10.1044/jshd.3402.123 [DOI] [PubMed] [Google Scholar]
- 7.Hillarp B, Ekberg O, Jacobsson S, et al. Myotonic dystrophy revealed at videoradiography of deglutition and speech in adult patients with velopharyngeal insufficiency: presentation of four cases. Cleft Palate Craniofac J 1994;31:125–33. 10.1597/1545-1569_1994_031_0125_mdravo_2.3.co_2 [DOI] [PubMed] [Google Scholar]
- 8.Horton SK, Murdoch BE, Theodoros DG, et al. Motor speech impairment in a case of childhood basilar artery stroke: treatment directions derived from physiological and perceptual assessment. Pediatr Rehabil 1997;1:163–77. 10.3109/17518429709167355 [DOI] [PubMed] [Google Scholar]
- 9.Huang MH, Lee ST, Rajendran K. Anatomic basis of cleft palate and velopharyngeal surgery: implications from a fresh cadaveric study. Plast Reconstr Surg 1998;101:613–27. 10.1097/00006534-199803000-00007 [DOI] [PubMed] [Google Scholar]
- 10.Sommerlad BC, Mehendale FV, Birch MJ, et al. Palate re-repair revisited. Cleft Palate Craniofac J 2002;39:295–307. [DOI] [PubMed] [Google Scholar]
- 11.Furlow LT, Jr. Cleft palate repair by double opposing Z-plasty. Plast Reconstr Surg 1986;78:724–36. 10.1097/00006534-198678060-00002 [DOI] [PubMed] [Google Scholar]
- 12.Hudson DA, Grobbelaar AO, Fernandes DB, et al. Treatment of velopharyngeal incompetence by the furlow Z-plasty. Ann Plast Surg 1995;34:23–6. 10.1097/00000637-199501000-00005 [DOI] [PubMed] [Google Scholar]
- 13.Chen PK, Wu JT, Chen YR, et al. Correction of secondary velopharyngeal insufficiency in cleft palate patients with the furlow palatoplasty. Plast Reconstr Surg 1994;94:933–43. 10.1097/00006534-199412000-00005 [DOI] [PubMed] [Google Scholar]
- 14.Chen PK, Wu JT, Chen YR, et al. Correction of secondary velopharyngeal insufficiency in cleft palate patients with the furlow palatoplasty. Plast Reconstr Surg 1994;94:933–41. 10.1097/00006534-199412000-00005 [DOI] [PubMed] [Google Scholar]
- 15.Hill C, Hayden C, Riaz M, et al. Buccinator sandwich pushback: a new technique for treatment of secondary velopharyngeal incompetence. Cleft Palate Craniofac J 2004;41:230–7. 10.1597/02-146.1 [DOI] [PubMed] [Google Scholar]
- 16.Mann RJ, Neaman KC, Armstrong SD, et al. The double-opposing buccal flap procedure for palatal lengthening. Plast Reconstr Surg 2011;127:2413–8. 10.1097/PRS.0b013e3182131d3e [DOI] [PubMed] [Google Scholar]
- 17.Hens G, Sell D, Pinkstone M, et al. Palate lengthening by buccinator myomucosal flaps for velopharyngeal insufficiency. Cleft Palate Craniofac J 2013;50:84–91. 10.1597/11-211 [DOI] [PubMed] [Google Scholar]
- 18.Hynes W. Pharyngoplasty by muscle transplantation. Br J Plast Surg 1950;3:128–35. 10.1016/S0007-1226(50)80020-6 [DOI] [PubMed] [Google Scholar]
- 19.Orticochea M. A review of 236 cleft palate patients treated with dynamic muscle sphincter. Plast Reconstr Surg 1983;71:180–6. 10.1097/00006534-198302000-00005 [DOI] [PubMed] [Google Scholar]
- 20.Jackson IT. Sphincter pharyngoplasty. Clin Plast Surg 1985;12:711–7. 10.1016/S0094-1298(20)31649-7 [DOI] [PubMed] [Google Scholar]
- 21.Sloan GM. Posterior pharyngeal flap and sphincter pharyngoplasty: the state of the art. Cleft Palate Craniofac J 2000;37:112–22. [DOI] [PubMed] [Google Scholar]
- 22.Filip C, Matzen M, Aagenæs I, et al. Autologous fat transplantation to the velopharynx for treating persistent velopharyngeal insufficiency of mild degree secondary to overt or submucous cleft palate. Journal of Plastic, Reconstructive & Aesthetic Surgery 2013;66:337–44. 10.1016/j.bjps.2012.11.006 [DOI] [PubMed] [Google Scholar]
- 23.Gray SD, Pinborough-zimmerman J, Catten M. Posterior wall augmentation for treatment of velopharyngeal insufficiency. Otolaryngol Head Neck Surg 1999;121:107–12. 10.1016/S0194-5998(99)70135-X [DOI] [PubMed] [Google Scholar]
- 24.Skoog T. The pharyngeal flap operation in cleft palate. A clinical study of eighty-two cases. Br J Plast Surg 1965;18:265–82. 10.1016/s0007-1226(65)80046-7 [DOI] [PubMed] [Google Scholar]
- 25.Honig CA. The treatment of velopharyngeal insufficiency after palatal repair. Arch Chir Neerl 1967;19:71–81. [PubMed] [Google Scholar]
- 26.Khuri SF, Daley J, Henderson W, et al. The department of veterans affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA surgical quality improvement program. Ann Surg 1998;228:491–507. 10.1097/00000658-199810000-00006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Marang-van de Mheen PJ, Stadlander MC, Kievit J. Adverse outcomes in surgical patients: implementation of a nationwide reporting system. Qual Saf Health Care 2006;15:320–4. 10.1136/qshc.2005.016220 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.de Blacam C, Smith S, Orr D. Surgery for velopharyngeal dysfunction: a systematic review of interventions and outcomes. Cleft Palate Craniofac J 2018;55:405–22. 10.1177/1055665617735102 [DOI] [PubMed] [Google Scholar]
- 29.Kirkham JJ, Boers M, Tugwell P, et al. Outcome measures in rheumatoid arthritis randomised trials over the last 50 years. Trials 2013;14:324 10.1186/1745-6215-14-324 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Bruce I, Harman N, Williamson P, et al. The management of otitis media with effusion in children with cleft palate (MOMENT): a feasibility study and economic evaluation. Health Technol Assess 2015;19:1–374. 10.3310/hta19680 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Williamson PR, Altman DG, Bagley H, et al. The comet Handbook: version 1.0. Trials 2017;18:280. 10.1186/s13063-017-1978-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kirkham JJ, Davis K, Altman DG, et al. Core outcome Set-STAndards for development: the COS-STAD recommendations. PLoS Med 2017;14:e1002447. 10.1371/journal.pmed.1002447 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Kirkham JJ, Gorst S, Altman DG, et al. Core outcome Set-STAndardised protocol items: the COS-STAP statement. Trials 2019;20 10.1186/s13063-019-3230-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Brown BB. Delphi process: a methodology used for the elicitation of opinions of experts. Santa Monica, CA: RAND Corporation, 1968. [Google Scholar]
- 35.Prinsen CAC, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a 'core outcome set'. Trials 2014;15:247. 10.1186/1745-6215-15-247 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Iyengar S, Williamson PR, Schmitt J, et al. Development of a core outcome set for clinical trials in rosacea: study protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials 2016;17:429. 10.1186/s13063-016-1554-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Harman NL, Bruce IA, Callery P, et al. MOMENT--Management of Otitis media with effusion in cleft palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials 2013;14:70. 10.1186/1745-6215-14-70 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Young A, Brookes S, Rumsey N, et al. Agreement on what to measure in randomised controlled trials in burn care: study protocol for the development of a core outcome set. BMJ Open 2017;7:e017267 10.1136/bmjopen-2017-017267 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Smith SM, Wallace E, Salisbury C, et al. A core outcome set for multimorbidity research (COSmm). Ann Fam Med 2018;16:132–8. 10.1370/afm.2178 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:132. 10.1186/1745-6215-13-132 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. framing the question and deciding on important outcomes. J Clin Epidemiol 2011;64:395–400. 10.1016/j.jclinepi.2010.09.012 [DOI] [PubMed] [Google Scholar]
- 43.Kirkham JJ, Gorst S, Altman DG, et al. Core outcome Set-STAndards for reporting: the COS-STAR statement. PLoS Med 2016;13:e1002148. 10.1371/journal.pmed.1002148 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.